Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.604699095s FAIL (2025-07-02 05:03:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.604741382s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.809161663s FAIL (2025-07-02 05:02:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.809197045s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.712038898s FAIL (2025-07-02 05:01:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.712065807s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.279716456s FAIL (2025-07-02 05:01:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.279772027s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.698162881s FAIL (2025-07-02 05:00:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.698194816s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.290662696s FAIL (2025-07-02 04:59:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.290696454s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.599042658s FAIL (2025-07-02 04:59:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.599068862s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.307687511s FAIL (2025-07-02 04:58:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.307719933s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.812076654s FAIL (2025-07-02 04:58:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.812096823s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.745114991s FAIL (2025-07-02 04:58:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.745143159s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.30022369s FAIL (2025-07-02 04:57:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.300258307s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.675881622s FAIL (2025-07-02 04:57:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.675979465s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.265791867s FAIL (2025-07-02 04:56:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.265830306s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.532066803s FAIL (2025-07-02 04:56:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.532095618s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.289458467s FAIL (2025-07-02 04:56:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.289486708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.24657942s FAIL (2025-07-02 04:55:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.246616861s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...............................................X...........] duration: 2.572218093s FAIL (2025-07-02 04:55:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.572245346s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X...............................................X..........] duration: 2.435473611s FAIL (2025-07-02 04:55:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.435506442s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X...............................................X.........] duration: 2.860564564s FAIL (2025-07-02 04:55:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.860596287s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.298021524s FAIL (2025-07-02 04:54:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.298050957s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X...............................................X........] duration: 2.574845543s FAIL (2025-07-02 04:54:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.574969878s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.333843805s FAIL (2025-07-02 04:53:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.333872077s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...............................................X.......] duration: 2.674864689s FAIL (2025-07-02 04:53:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.674892729s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.329123009s FAIL (2025-07-02 04:53:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.329177432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.283714189s FAIL (2025-07-02 04:52:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.283743879s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X...............................................X......] duration: 2.767239218s FAIL (2025-07-02 04:52:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.767264727s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X...............................................X.....] duration: 2.5614904s FAIL (2025-07-02 04:52:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.561526262s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.416074778s FAIL (2025-07-02 04:51:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.416110147s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X...............................................X....] duration: 2.842311373s FAIL (2025-07-02 04:51:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.842335611s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.334423959s FAIL (2025-07-02 04:50:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.334537636s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.258773694s FAIL (2025-07-02 04:50:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.258813617s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X...............................................X...] duration: 2.825282495s FAIL (2025-07-02 04:50:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.825322579s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.572100889s FAIL (2025-07-02 04:50:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.572198802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.293960699s FAIL (2025-07-02 04:49:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.29398921s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X...............................................X..] duration: 2.747131938s FAIL (2025-07-02 04:49:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.747158645s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X...............................................X.] duration: 2.775239632s FAIL (2025-07-02 04:48:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.775270894s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.300492979s FAIL (2025-07-02 04:48:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.300521317s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X...............................................X] duration: 30.000870142s FAIL (2025-07-02 04:48:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000901158s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.382237811s FAIL (2025-07-02 04:47:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.382266629s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.667976866s FAIL (2025-07-02 04:46:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.668009664s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.312049059s FAIL (2025-07-02 04:46:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.312085041s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.832375224s FAIL (2025-07-02 04:46:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.832411728s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.593631794s FAIL (2025-07-02 04:45:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.593668104s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.361043493s FAIL (2025-07-02 04:45:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.361099528s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.553354095s FAIL (2025-07-02 04:45:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.554796511s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.307771435s FAIL (2025-07-02 04:44:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.307801285s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.622833189s FAIL (2025-07-02 04:44:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.622929955s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.293779073s FAIL (2025-07-02 04:43:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.293807918s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.790650904s FAIL (2025-07-02 04:43:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.790752544s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.624879333s FAIL (2025-07-02 04:43:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.624906884s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)