Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.628449752s FAIL (2025-07-02 04:23:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.628990146s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.351675738s FAIL (2025-07-02 04:22:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.3517089s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.504462909s FAIL (2025-07-02 04:22:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.50449981s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.414090978s FAIL (2025-07-02 04:22:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.414121467s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.693560519s FAIL (2025-07-02 04:21:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.693590824s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.310596809s FAIL (2025-07-02 04:21:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.310700582s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.841337402s FAIL (2025-07-02 04:21:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.841370826s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.315480269s FAIL (2025-07-02 04:20:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.315528584s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.367202618s FAIL (2025-07-02 04:20:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.367228643s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.500446413s FAIL (2025-07-02 04:19:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.500476328s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.261789789s FAIL (2025-07-02 04:19:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.261820866s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.677455669s FAIL (2025-07-02 04:19:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.677479025s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.372583238s FAIL (2025-07-02 04:19:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.372611427s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.563428004s FAIL (2025-07-02 04:18:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.56350217s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.816443179s FAIL (2025-07-02 04:18:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.816471419s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.30414439s FAIL (2025-07-02 04:18:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.304175456s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.616112171s FAIL (2025-07-02 04:17:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.616137782s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.273089459s FAIL (2025-07-02 04:17:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.273117531s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.626594621s FAIL (2025-07-02 04:17:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.626623539s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.631374362s FAIL (2025-07-02 04:16:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.631447135s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.357813201s FAIL (2025-07-02 04:16:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.357851326s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.891465991s FAIL (2025-07-02 04:16:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.891492996s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.377501346s FAIL (2025-07-02 04:15:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.377530269s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.587535948s FAIL (2025-07-02 04:15:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.587555611s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.500268644s FAIL (2025-07-02 04:15:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.500308444s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.271657967s FAIL (2025-07-02 04:14:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.271684772s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.687490275s FAIL (2025-07-02 04:14:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.687589955s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.116341494s FAIL (2025-07-02 04:13:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== WWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.483583245 (211h2m10.366996878s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.486463918 (211h2m10.369877537s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487276164 (211h2m10.370689782s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.487931464 (211h2m10.371345085s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.488647054 (211h2m10.372060679s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.489381854 (211h2m10.372795479s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.491357802 (211h2m10.374771421s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.493286980 (211h2m10.376700599s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.494990033 (211h2m10.378403645s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.496640219 (211h2m10.380053839s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.497854847 (211h2m10.381268462s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.499240297 (211h2m10.38265392s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.503423699 (211h2m10.386837317s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.504688840 (211h2m10.38810246s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:10.507716926 (211h2m10.391130538s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Uploading piece... Jul 2 04:13:11.091150421 (211h2m10.974564038s) [upload=168801]: Next returned nil segment Jul 2 04:13:11.091210721 (211h2m10.974624351s) [upload=168801]: Waiting for error group managing segments... Jul 2 04:13:11.224127987 (211h2m11.107541604s) [noise=true , address=104.36.82.2:4731 , piece_id=ENXPHEJM6AA4ZVPAJ4LBDSDPVPNFXSOW4YEFQV7I67SO66OZEY3Q, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.227509915 (211h2m11.110923534s) [noise=true , address=104.36.82.10:4079 , piece_id=VNUZHNQE7XI4JXKRPYX6NDWKLGS6KW6ANRG65HCY2NWK7XWHOJZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.230251559 (211h2m11.113665175s) [noise=true , address=104.36.82.3:4767 , piece_id=DATEOSYAE7JO4RKN6WZYPPAYWDZDBSQXQSVT7IDV3Y4HLEI6SQDQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.232589626 (211h2m11.116003245s) [noise=true , address=104.36.82.6:4876 , piece_id=H2WWPUQN6X6YCQQK7HICBODSZZBAVGFVCS4AJK6JRFQBBKBJJTZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.234638228 (211h2m11.118051846s) [noise=true , address=104.36.82.8:4972 , piece_id=FUWF5Y3FLQLS4F4L2WTJUEZUXZO5ONHDWWHBKVOCR6GYAOA2JP4A, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.236463794 (211h2m11.119877413s) [noise=true , address=104.36.82.7:4911 , piece_id=5MGEOAA5NTGNZOWKW7MQWQ2YFFXCC5IIRP5OHN3ELYNUPLBQXOIQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.258669850 (211h2m11.142083468s) [noise=true , address=104.36.82.11:4113 , piece_id=RRW6N7XQQB3JON75JIBLGHH56NL6VFXN5GUBHGRPQFLXOTITY4ZQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:11.263325751 (211h2m11.146739368s) [noise=true , address=104.36.82.13:4185 , piece_id=GXRVOOJAASFLQ7EYBQZ5EIZFC33CPGG4CWUMIFNYYP6CA53W75YA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: <nil> Jul 2 04:13:40.094816384 (211h2m39.978230007s) [noise=true , address=45.91.116.10:4083 , piece_id=Z4TOZLOGSQRLL6HIBG3AEJZK356VKNCDSETE27RE46JQIS5RYRIA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59798->45.91.116.10:4083: use of closed network connection Jul 2 04:13:40.094892672 (211h2m39.978306289s) [noise=true , address=45.91.116.7:4010 , piece_id=DHCHV263SFC6C7JLZ5YDZYP6PRLC2MQDUM4NBKIAQGYQC53O66HQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44758->45.91.116.7:4010: use of closed network connection Jul 2 04:13:40.094960211 (211h2m39.978373836s) [noise=true , address=77.90.31.11:4553 , piece_id=MT7HUX3CPMXH5FLWFVI5GGOKITDJ2NJFYWJCBTRLVKLPVHAVBRXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:45046->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095013325 (211h2m39.978426942s) [noise=true , address=77.90.31.13:4627 , piece_id=TZN454N3WCEYETLKLQDSTHTY6SG7XM3NEXEG7CGZDFJZSTQHE7WQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 139.59.144.22:36200->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095022161 (211h2m39.978435781s) [noise=true , address=45.91.116.6:4974 , piece_id=NNRSU4RIQ5ICEF7XUIPSGSRNR26LHDK4QT6FPI3K6RU4RVQTHYXQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57574->45.91.116.6:4974: use of closed network connection Jul 2 04:13:40.095064847 (211h2m39.97847846s) [noise=true , address=45.91.116.9:4262 , piece_id=C6M2QO6CZQ53WMTDT5FGJPMPV6BILK5EV3YC4NZ6BVGUYTL3D4IA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47392->45.91.116.9:4262: use of closed network connection Jul 2 04:13:40.095114736 (211h2m39.978528352s) [noise=true , address=45.91.116.9:4261 , piece_id=MAXW6ORMRUEAB3XPSYXW23EIV3CI5KVKHMX4L6PQSKKPCMG4Y4ZA, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35362->45.91.116.9:4261: use of closed network connection Jul 2 04:13:40.095126048 (211h2m39.978539665s) [noise=true , address=77.90.31.19:4841 , piece_id=5OPMBXHPYGVLCDKDQCZ6Y374BBCO7ZPAXK3A5GCYMHJ2XTHGNJWQ, seg_pos={0 0}, upload=168801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:38726->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.095157559 (211h2m39.978571186s) [seg_pos={0 0}, upload=168801]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 2 04:13:40.208880963 (211h2m40.092294583s) [upload=168801]: Done uploading segments... Jul 2 04:13:40.208909447 (211h2m40.092323073s) [upload=168801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.116369889s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001047201s FAIL (2025-07-02 04:13:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001072294s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 869.525068ms FAIL (2025-07-02 02:46:29 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 869.569888ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 991.266349ms FAIL (2025-07-02 02:45:53 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 991.299077ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 925.478807ms FAIL (2025-07-02 02:44:41 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 925.563963ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.005166646s FAIL (2025-07-02 02:43:51 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.005199858s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 883.198167ms FAIL (2025-07-02 02:42:54 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 883.227138ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 916.516445ms FAIL (2025-07-02 02:42:26 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 916.548339ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.056825683s FAIL (2025-07-02 02:41:13 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.056860656s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.050661662s FAIL (2025-07-02 02:40:42 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.050694518s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.134497569s FAIL (2025-07-02 02:39:30 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.13453052s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.22918465s FAIL (2025-07-02 02:39:03 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.229463595s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 839.236467ms FAIL (2025-07-02 02:37:54 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 839.26577ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 973.213049ms FAIL (2025-07-02 02:36:40 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 973.244511ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 994.530799ms FAIL (2025-07-02 02:36:21 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 994.571397ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.042642169s FAIL (2025-07-02 02:36:02 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.042685622s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 903.495297ms FAIL (2025-07-02 02:34:47 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 903.523755ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 971.53784ms FAIL (2025-07-02 02:33:59 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 971.583545ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 950.24344ms FAIL (2025-07-02 02:33:28 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 950.278254ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.053224688s FAIL (2025-07-02 02:33:01 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.05326031s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.131170685s FAIL (2025-07-02 02:32:18 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.131205113s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 880.165258ms FAIL (2025-07-02 02:31:21 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 880.19957ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.195866715s FAIL (2025-07-02 02:30:59 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.195901912s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)