Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.473346338s
FAIL (2025-09-05 15:16:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.473399432s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.844690586s
FAIL (2025-09-05 15:16:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.844721073s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.458141079s
FAIL (2025-09-05 15:15:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.458174971s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.328124129s
FAIL (2025-09-05 15:15:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.328189874s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.747957927s
FAIL (2025-09-05 15:15:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.74802281s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.400703346s
FAIL (2025-09-05 15:14:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.400736664s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.635358322s
FAIL (2025-09-05 15:14:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.635393531s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.653359897s
FAIL (2025-09-05 15:13:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.653384058s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.391623882s
FAIL (2025-09-05 15:13:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.391654533s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.802545509s
FAIL (2025-09-05 15:13:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.802572571s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
8.457866561s
FAIL (2025-09-05 15:13:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.457891319s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.563769378s
FAIL (2025-09-05 15:12:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.563803868s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.903773959s
FAIL (2025-09-05 15:12:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.903815755s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.311619353s
FAIL (2025-09-05 15:12:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.311676982s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.896719065s
FAIL (2025-09-05 15:11:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.896780432s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.729981774s
FAIL (2025-09-05 15:11:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.730016233s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.421083376s
FAIL (2025-09-05 15:11:23 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.421116475s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.674683641s
FAIL (2025-09-05 15:10:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.674804886s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.653293521s
FAIL (2025-09-05 15:10:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.653332825s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.817834021s
FAIL (2025-09-05 15:10:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.817871922s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.734099888s
FAIL (2025-09-05 15:09:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.734119761s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.485836848s
FAIL (2025-09-05 15:09:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.485868283s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.715902744s
FAIL (2025-09-05 15:09:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.715940609s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.584154964s
FAIL (2025-09-05 15:08:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.584191606s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.641559886s
FAIL (2025-09-05 15:08:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.641592688s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.921635052s
FAIL (2025-09-05 15:07:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.921673021s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.524583914s
FAIL (2025-09-05 15:07:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.524614024s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.742424344s
FAIL (2025-09-05 15:07:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.742535316s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.766834726s
FAIL (2025-09-05 15:06:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.766860612s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.410090927s
FAIL (2025-09-05 15:06:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.410142508s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.981909202s
FAIL (2025-09-05 15:06:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.981976251s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.614249722s
FAIL (2025-09-05 15:05:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.614286082s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.939495457s
FAIL (2025-09-05 15:05:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.939531076s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.533996406s
FAIL (2025-09-05 15:05:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.534034891s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.756591241s
FAIL (2025-09-05 15:04:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.756641413s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.962754161s
FAIL (2025-09-05 15:04:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.962780057s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.386778249s
FAIL (2025-09-05 15:04:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.38681674s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.680392989s
FAIL (2025-09-05 15:03:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.680425817s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.775695551s
FAIL (2025-09-05 15:03:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.775759741s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.444688222s
FAIL (2025-09-05 15:03:28 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.444719625s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.794551912s
FAIL (2025-09-05 15:03:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.794600036s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.458822474s
FAIL (2025-09-05 15:02:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.458855618s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.768695179s
FAIL (2025-09-05 15:02:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.76871988s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.413487451s
FAIL (2025-09-05 15:02:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.413520859s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.840786466s
FAIL (2025-09-05 15:01:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.840855942s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.53708354s
FAIL (2025-09-05 15:01:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.537116779s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.566557382s
FAIL (2025-09-05 15:01:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.566590473s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.902078494s
FAIL (2025-09-05 15:01:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.902192544s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.61958798s
FAIL (2025-09-05 15:00:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
849 (168h37m5.717708303s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.225700575 (168h37m5.718133025s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226321425 (168h37m5.718753873s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.226758873 (168h37m5.719191327s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227460172 (168h37m5.719892626s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.227949873 (168h37m5.72038233s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228373793 (168h37m5.720806245s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.228792442 (168h37m5.721224948s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229196927 (168h37m5.721629376s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.229623399 (168h37m5.722055848s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:44:45.230143670 (168h37m5.722576124s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Uploading piece...
Sep 5 14:45:14.784768820 (168h37m35.277201269s) [upload=148399]: Next segment err: context deadline exceeded
Sep 5 14:45:14.786287635 (168h37m35.27872011s) [noise=true , address=45.91.116.4:4938 , piece_id=LM6TM2EDSLAYAMJBXJSPBZZONSLBVV3S3M27OPDDO3BQNKU646CA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.786963165 (168h37m35.279395622s) [noise=true , address=45.91.116.9:4261 , piece_id=LUFPDCQDE6CDQMWLHYO6OUECFYV6QKQIAJEV2TSPIS7TGSQL5EUQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787092184 (168h37m35.279524632s) [noise=true , address=77.90.31.16:4733 , piece_id=DYLL6KQQZJR64UNYTTA7ZKP42YKDZJDXQALQMBSPITIBXCWRWMWA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787168610 (168h37m35.279601046s) [noise=true , address=77.90.31.18:4805 , piece_id=ATKM2BAFYJ2XPSNCQ2M54BET6ZJL4KASUPKYCHQDAM52CILTB27Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 139.59.144.22:59698->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787225233 (168h37m35.279657677s) [noise=true , address=77.90.31.10:4517 , piece_id=LROS5ZFHY2WQHC5XKQAHFGP4J7F5OBE3RLCA66J7ECMSXQMEPUOQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787256431 (168h37m35.279688875s) [noise=true , address=45.91.116.14:4191 , piece_id=T4JEXJ5GAIOJJOOI7U75IVAG323ZPC6G4N4ALGY64SXETV4P6SGQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:55510->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787349336 (168h37m35.279781781s) [noise=true , address=45.91.116.11:4117 , piece_id=SE2F6VWCSKYNUTIP4MMTOBVQNRQWJOCGMYA5IBBGRV6XDJVS4T4Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:36062->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787412006 (168h37m35.279844451s) [noise=true , address=77.90.31.10:4520 , piece_id=RGCWCGHEIQ7PEAQ36WCDNSKJB2FVD4XH6HS4JPXCBIAOSRRYB45A, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:51966->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787419735 (168h37m35.279852169s) [noise=true , address=45.91.116.6:4973 , piece_id=YKZ2H6AUO2ZO6743KA4FEW45JQY24DANVD5LIKVOJ5EITMXY2XVA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787466329 (168h37m35.279898774s) [noise=true , address=77.90.31.16:4734 , piece_id=377KPWDXZP2WJBOYA44EG6ZC4KAO7A453KDJRIPIQTB5RA4Q6WDA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:46232->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787524844 (168h37m35.279957292s) [noise=true , address=45.91.116.13:4155 , piece_id=H4QY7D7WHNDJQO4XTR6CMAKPS5V5POO4VG7E3SGOMSNZPM6W3HBA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:40352->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787580533 (168h37m35.280012978s) [noise=true , address=45.91.116.10:4081 , piece_id=HYDZ7W6DJYUQROQBS5353QCSILQPSJ5PQNKGF2BRBG3JU2PQJ6KQ, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:56344->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787619479 (168h37m35.280051914s) [noise=true , address=45.91.116.11:4118 , piece_id=HYKKV5XZD43O4VM4UMXO2QH7NW3KBMIE7PNKKEE6SFCKSBUMNRCA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:44240->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787702763 (168h37m35.280135199s) [noise=true , address=45.91.116.7:4011 , piece_id=C2E3J5DCNGBTSSYE3FTXSO2SACDSYRY37ONJZCFJKMWPDKHBU7GA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:52152->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787727215 (168h37m35.280159659s) [noise=true , address=45.91.116.12:4227 , piece_id=ATMBAUCWAWNT4GORSLRVRMOACL2I4VSVH6X3RENVWPISFFEM3I3Q, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:47026->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787754874 (168h37m35.280187311s) [noise=true , address=45.91.116.13:4154 , piece_id=OY6BUHQIMIJUPTYAP35ECAG3IWUXUCVQYH4QTXTV7ZLLER37LLLA, seg_pos={0 0}, upload=148399]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:53570->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 5 14:45:14.787771903 (168h37m35.280204337s) [seg_pos={0 0}, upload=148399]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 2.619656723s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.960716291s
FAIL (2025-09-05 15:00:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.960740593s
edge
gateway
us-select-1
|