Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......................................XX....................] duration:
3.392234683s
FAIL (2025-09-19 04:02:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.392261468s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.466844058s
FAIL (2025-09-19 04:02:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.466875332s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......................................XX...................] duration:
3.518461335s
FAIL (2025-09-19 04:02:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.518571233s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.880700235s
FAIL (2025-09-19 04:01:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.880728941s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........................................XX..................] duration:
2.791635539s
FAIL (2025-09-19 04:01:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.791664633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........................................XX.................] duration:
2.948246359s
FAIL (2025-09-19 04:00:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.948282769s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.626684784s
FAIL (2025-09-19 04:00:23 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.62677883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........................................XX................] duration:
2.718475652s
FAIL (2025-09-19 03:59:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.718612328s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.506758117s
FAIL (2025-09-19 03:59:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.506791534s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........................................XX...............] duration:
2.69545601s
FAIL (2025-09-19 03:59:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.695490084s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.510610021s
FAIL (2025-09-19 03:58:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.510650476s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.526445554s
FAIL (2025-09-19 03:57:58 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.526478817s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............................................XX..............] duration:
3.271700254s
FAIL (2025-09-19 03:57:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.271748307s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............................................XX.............] duration:
2.961573746s
FAIL (2025-09-19 03:57:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.961607682s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.752539601s
FAIL (2025-09-19 03:56:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.752571365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............................................XX............] duration:
2.684627047s
FAIL (2025-09-19 03:56:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.684670405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.578278685s
FAIL (2025-09-19 03:55:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.578315298s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............................................XX...........] duration:
3.518842251s
FAIL (2025-09-19 03:55:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.51886767s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.383736564s
FAIL (2025-09-19 03:54:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.383759879s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
2.649257093s
FAIL (2025-09-19 03:54:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.650234805s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.422751872s
FAIL (2025-09-19 03:54:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.422884191s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
2.757316994s
FAIL (2025-09-19 03:53:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.757348363s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
3.05473526s
FAIL (2025-09-19 03:53:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.054766875s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.927615808s
FAIL (2025-09-19 03:53:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.927663558s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.640649799s
FAIL (2025-09-19 03:52:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.640698505s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
3.212835861s
FAIL (2025-09-19 03:52:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.212861234s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.444527764s
FAIL (2025-09-19 03:52:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.444556431s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
2.886312886s
FAIL (2025-09-19 03:51:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.886339297s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
3.457616226s
FAIL (2025-09-19 03:51:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 3.457645546s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
3.377634783s
FAIL (2025-09-19 03:51:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.377663445s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.685679745s
FAIL (2025-09-19 03:50:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.685710416s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
2.711589258s
FAIL (2025-09-19 03:50:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.711619919s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
3.494745534s
FAIL (2025-09-19 03:49:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.494773721s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.542639467s
FAIL (2025-09-19 03:49:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.542666406s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
2.719579669s
FAIL (2025-09-19 03:49:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.719609057s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.463251585s
FAIL (2025-09-19 03:48:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 2.463283646s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
2.72571059s
FAIL (2025-09-19 03:48:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.726282941s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
35.118726575s
FAIL (2025-09-19 03:48:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 35.118754866s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
1m0.000691397s
FAIL (2025-09-19 03:48:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000704576s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.106475522s
FAIL (2025-09-19 03:47:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
UQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171293002 (493h39m17.663725437s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.171975257 (493h39m17.664407693s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.174350984 (493h39m17.666783418s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.177310451 (493h39m17.669742886s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.179941279 (493h39m17.672373716s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.182528457 (493h39m17.674960885s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185193117 (493h39m17.67762555s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:46:57.185647080 (493h39m17.678079527s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Uploading piece...
Sep 19 03:47:56.702693018 (493h40m17.19512547s) [upload=434191]: Next segment err: context deadline exceeded
Sep 19 03:47:56.703255047 (493h40m17.195687499s) [noise=true , address=145.79.227.31:4828 , piece_id=TLVFYAPXMBXUD23PIDXIRLWPD7PO7ZJ6NDOOLYPLRQWP7KB46FQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: rpc: dial tcp 145.79.227.31:4828: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703336893 (493h40m17.195769339s) [noise=true , address=145.79.227.30:4787 , piece_id=RDRBR4VD37QPX7JTXPM5DHTVJTM5KDCAGHT7FYAFFZO34T5OM5SA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:145.79.227.30:4787): protocol: rpc: dial tcp 145.79.227.30:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703393958 (493h40m17.195826401s) [noise=true , address=145.79.227.51:4434 , piece_id=SFMRY3FWZWSL2VMHHZFXQKJQYJX4RVB2BCHGJMJASFKRKGTBQYUQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12kZH2amRgBmyujKY7Kk4yEQRY8CJMfaz2hw2XRJ8tzRiHExvtX, address:145.79.227.51:4434): protocol: rpc: dial tcp 145.79.227.51:4434: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703879282 (493h40m17.196311717s) [noise=true , address=145.79.227.42:4156 , piece_id=2UCOZTMB2HRR5RY4MZIY2C7FKQUL7ISG5VOSHSOBZ2AVAUHDVDKA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:145.79.227.42:4156): protocol: rpc: dial tcp 145.79.227.42:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.703967838 (493h40m17.196400274s) [noise=true , address=145.79.227.26:4667 , piece_id=VQCMIFD72WA7JPFZLFZNSZWRREURKQZR3YUHFN3OO7OFZJW62YIQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12iLz6HjQQ1HK7nbPen8szRrZywZcbPTs58FmCPDnAyAEg4AvNk, address:145.79.227.26:4667): protocol: rpc: dial tcp 145.79.227.26:4667: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704050076 (493h40m17.19648251s) [noise=true , address=145.79.227.23:4577 , piece_id=2LQWJBV4RYUSGNA3CPRRV24VO6AXHKNMH5K7EZ7ZVQIQPNUDH3GA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:145.79.227.23:4577): protocol: rpc: dial tcp 145.79.227.23:4577: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704137367 (493h40m17.196569806s) [noise=true , address=145.79.227.10:4412 , piece_id=LZXW5UUS7PHEIQQVIQNB5NNDMOYWXEEA2JTVYS22BJZB2VC4WIQQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12MyHzHi6j7ogzfSAgM4pWXq6SzA9tNTnWXXGAVcvYNZjxU3bcX, address:145.79.227.10:4412): protocol: rpc: dial tcp 145.79.227.10:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704216674 (493h40m17.196649106s) [noise=true , address=145.79.227.52:4475 , piece_id=YBEMM43JRDDPXC5VKGKMAJ2EFA4Z3332OETKIMHGCYWISM5LFFSQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704290852 (493h40m17.196723286s) [noise=true , address=145.79.227.12:4786 , piece_id=OWU5JCNF4U7K3FKHRMD255DHHDGBCR2BNEK2USTPMMDFCMF4345Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:127aTG7tofhLA5799hT6gvZZSkT1vvWWFejx7DS4Zcz8mBHHsuE, address:145.79.227.12:4786): protocol: rpc: dial tcp 145.79.227.12:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704363020 (493h40m17.196795452s) [noise=true , address=145.79.227.21:4482 , piece_id=4RXYM2XMYF4PZ5CMTXVVZCAYDXAEI7OJOGTHRFJ6FNYDKOPVYS4Q, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:145.79.227.21:4482): protocol: rpc: dial tcp 145.79.227.21:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704430045 (493h40m17.196862477s) [noise=true , address=145.79.227.14:4870 , piece_id=AYUGQFLUKQMKQ5473IM4THLOYQ2DG6YJEAXJL2N7UWMLB6NSO67A, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: rpc: dial tcp 145.79.227.14:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704444449 (493h40m17.196876902s) [noise=true , address=145.79.227.47:4283 , piece_id=EN7SUCIIULE2PGUOI2ON4G34225ZRQA3HFGUHDLYPDDQ7YE763CA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:145.79.227.47:4283): protocol: rpc: dial tcp 145.79.227.47:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704468026 (493h40m17.196900462s) [noise=true , address=145.79.227.4:4217 , piece_id=5QVTT7FIAVOH42YCDTSL6VJNE7WSWXWZSGOCIICJKRHK4GBVX7FQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704570213 (493h40m17.197002659s) [noise=true , address=145.79.227.28:4730 , piece_id=OTZOC46PESTUYYNUOZF7PJZSUQA3WJH6NNBMBHQLUJPSIRKUQDKQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704652142 (493h40m17.197084584s) [noise=true , address=145.79.227.17:4983 , piece_id=N2SUO43GSMQCNG34BXQMRVJO5IIXLTLZT3P6ZBGDDQEYP2REA2LQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:145.79.227.17:4983): protocol: rpc: dial tcp 145.79.227.17:4983: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704678787 (493h40m17.197111215s) [noise=true , address=145.79.227.29:4953 , piece_id=YLKSN7X4ZDJY4XXWSMCFGXO4SRRPKWQXQC3S6D2HJXB23T72ISPQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:145.79.227.29:4953): protocol: rpc: dial tcp 145.79.227.29:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704714590 (493h40m17.197147033s) [noise=true , address=145.79.227.24:4590 , piece_id=XYGLAI7GEPXL4USSJXEHXEK6B4XXKM2W7HXQ7YWYJDUAEX2J4CXQ, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:145.79.227.24:4590): protocol: rpc: dial tcp 145.79.227.24:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704768947 (493h40m17.19720139s) [noise=true , address=145.79.227.46:4613 , piece_id=QUJ6H6M4IOHBPKWHHAS7FRMJO43UGHXMI5EPHDZCNMDTNDFZ54IA, seg_pos={0 0}, upload=434191]: Done uploading piece. err: ecclient: upload failed (node:125xy2Y2QhNLeRQ9as4GccgVsyL1KUAtUChG4EJCDK8UYssFrR3, address:145.79.227.46:4613): protocol: rpc: dial tcp 145.79.227.46:4613: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 19 03:47:56.704799188 (493h40m17.197231637s) [seg_pos={0 0}, upload=434191]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 1m0.106504013s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.00149648s
FAIL (2025-09-19 03:47:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001521577s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.899108373s
FAIL (2025-09-18 22:34:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.899191183s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.819007845s
FAIL (2025-09-18 22:34:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.819257105s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.631303649s
FAIL (2025-09-18 22:34:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 18 21:51:34.707222432 (487h43m55.199654876s) [upload=428952]: Starting upload Sep 18 21:51:34.707338977 (487h43m55.199771422s) [upload=428952]: Done starting upload Sep 18 21:51:34.707496319 (487h43m55.199928756s) [upload=428952]: Uploading segments... Sep 18 21:51:34.707899906 (487h43m55.200332341s) [upload=428952]: Got next segment. Inline: false Sep 18 21:51:34.707991611 (487h43m55.200424057s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 18 21:51:34.708015128 (487h43m55.200447572s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Sep 18 21:52:04.423254886 (487h44m24.915687328s) [upload=428952]: Next segment err: context deadline exceeded Sep 18 21:52:04.423355400 (487h44m24.915787833s) [upload=428952]: Done uploading segments... Sep 18 21:52:04.423375151 (487h44m24.915807586s) [upload=428952]: Upload finished. err: context deadline exceeded
Duration: 2.631341288s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.982058123s
FAIL (2025-09-18 22:33:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.982471533s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.579262967s
FAIL (2025-09-18 22:33:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 18 21:51:34.707222432 (487h43m55.199654876s) [upload=428952]: Starting upload Sep 18 21:51:34.707338977 (487h43m55.199771422s) [upload=428952]: Done starting upload Sep 18 21:51:34.707496319 (487h43m55.199928756s) [upload=428952]: Uploading segments... Sep 18 21:51:34.707899906 (487h43m55.200332341s) [upload=428952]: Got next segment. Inline: false Sep 18 21:51:34.707991611 (487h43m55.200424057s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 18 21:51:34.708015128 (487h43m55.200447572s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Sep 18 21:52:04.423254886 (487h44m24.915687328s) [upload=428952]: Next segment err: context deadline exceeded Sep 18 21:52:04.423355400 (487h44m24.915787833s) [upload=428952]: Done uploading segments... Sep 18 21:52:04.423375151 (487h44m24.915807586s) [upload=428952]: Upload finished. err: context deadline exceeded
Duration: 2.579295486s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.902472672s
FAIL (2025-09-18 22:33:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.902506865s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.728404563s
FAIL (2025-09-18 22:32:05 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 18 21:51:34.707222432 (487h43m55.199654876s) [upload=428952]: Starting upload Sep 18 21:51:34.707338977 (487h43m55.199771422s) [upload=428952]: Done starting upload Sep 18 21:51:34.707496319 (487h43m55.199928756s) [upload=428952]: Uploading segments... Sep 18 21:51:34.707899906 (487h43m55.200332341s) [upload=428952]: Got next segment. Inline: false Sep 18 21:51:34.707991611 (487h43m55.200424057s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 18 21:51:34.708015128 (487h43m55.200447572s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Sep 18 21:52:04.423254886 (487h44m24.915687328s) [upload=428952]: Next segment err: context deadline exceeded Sep 18 21:52:04.423355400 (487h44m24.915787833s) [upload=428952]: Done uploading segments... Sep 18 21:52:04.423375151 (487h44m24.915807586s) [upload=428952]: Upload finished. err: context deadline exceeded
Duration: 2.728508767s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.632882692s
FAIL (2025-09-18 22:31:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.632975279s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.543940103s
FAIL (2025-09-18 22:31:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 18 21:51:34.707222432 (487h43m55.199654876s) [upload=428952]: Starting upload Sep 18 21:51:34.707338977 (487h43m55.199771422s) [upload=428952]: Done starting upload Sep 18 21:51:34.707496319 (487h43m55.199928756s) [upload=428952]: Uploading segments... Sep 18 21:51:34.707899906 (487h43m55.200332341s) [upload=428952]: Got next segment. Inline: false Sep 18 21:51:34.707991611 (487h43m55.200424057s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 18 21:51:34.708015128 (487h43m55.200447572s) [upload=428952]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Sep 18 21:52:04.423254886 (487h44m24.915687328s) [upload=428952]: Next segment err: context deadline exceeded Sep 18 21:52:04.423355400 (487h44m24.915787833s) [upload=428952]: Done uploading segments... Sep 18 21:52:04.423375151 (487h44m24.915807586s) [upload=428952]: Upload finished. err: context deadline exceeded
Duration: 2.543994775s
core
uplink
us-select-1
|