Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................X...................] duration:
1.791741146s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:58:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.791761296s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................X..................] duration:
1.63060346s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:57:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.630660834s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
1.961773363s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:56:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.961805402s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................X................] duration:
1.722608546s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:56:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.722717213s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................................X...............] duration:
1.66542952s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:54:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.6655271s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................................X..............] duration:
1.758637863s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:54:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.758667516s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
1.703686646s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:53:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.703720325s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
2.177965971s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:52:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.177996361s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
2.285404121s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:51:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.285513552s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
2.682172598s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:50:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.682198937s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
2.232068082s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:49:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.232106768s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.710476935s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:49:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.710506s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
2.657119195s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:48:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.657147221s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
2.080638506s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:47:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.080670977s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
2.783216239s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:46:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.783242113s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
2.038552493s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:45:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.038584187s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
2.200790147s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:45:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.200820525s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.890147351s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:44:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.890384713s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
1.745934346s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:43:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.745957842s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.000701505s trace:
https://jaeger.dev.storj.io/trace/3e5ed298be967693
FAIL (2026-02-12 22:42:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000725305s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.449041788s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:23:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.449077606s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.387090803s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:22:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.387163845s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.459744528s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:21:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.459791122s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.454515867s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:20:49 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.454547674s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.519210572s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:20:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.51923949s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.496705186s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:19:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.496734787s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.527467371s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:18:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.527493976s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.469841414s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:18:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.469923986s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.441104093s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:17:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.441127397s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.478444113s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:16:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.478543807s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.479710812s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:15:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.479740598s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.40419838s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:14:58 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.404228038s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.465569504s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:13:49 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.465599749s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.376770838s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:12:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.376798402s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.496317774s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:12:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.496351874s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.446645095s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:11:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.446676218s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.316370363s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:11:28 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.316548233s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.394140839s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:10:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.394208354s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.427282491s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:09:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.427309007s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.458817176s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:08:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.458838117s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.460212111s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:07:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.460263578s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.458467095s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:07:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.458511554s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.45639499s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:06:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.456423422s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.412793335s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:05:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.412821666s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.459580716s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:05:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.459605238s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.389281989s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:04:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.389311883s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.410574324s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:03:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.410608611s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.381499958s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:02:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.381533066s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.37896373s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:02:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.37899324s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.402988581s trace:
https://jaeger.dev.storj.io/trace/40150cfcfd7e63e0
FAIL (2026-02-12 22:01:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, Piece ID: 4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA) context deadline exceeded; piecestore: (Node ID: 12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, Piece ID: SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A) context deadline exceeded; piecestore: (Node ID: 12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, Piece ID: JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA) context deadline exceeded; piecestore: (Node ID: 19rx4nU8dUTtPXvMonzZSBkn5Z5cnBXrndvU7t8ewUmfYLBqid, Piece ID: JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ) context deadline exceeded; piecestore: (Node ID: 1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, Piece ID: FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ) context deadline exceeded
======LOGS======
435517288s) [upload=50547]: Uploading segments...
Feb 12 21:38:14.002584088 (57h23m14.435846278s) [upload=50547]: Got next segment. Inline: false
Feb 12 21:38:14.002676626 (57h23m14.43593878s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 12 21:38:14.002684892 (57h23m14.435947022s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 12 21:38:14.116829594 (57h23m14.550091747s) [seg_pos={0 0}, upload=50547]: Begin upload segment...
Feb 12 21:38:14.116924460 (57h23m14.550186572s) [seg_pos={0 0}, upload=50547]: Done begin upload segment.
Feb 12 21:38:14.116946106 (57h23m14.550208232s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117138421 (57h23m14.550400547s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.117794188 (57h23m14.551056324s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.118335993 (57h23m14.55159814s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119340762 (57h23m14.552602896s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.119764540 (57h23m14.553026668s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120198405 (57h23m14.553460532s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.120697148 (57h23m14.553959273s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.121410403 (57h23m14.554672538s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122127071 (57h23m14.555389199s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.122677467 (57h23m14.555939592s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.124387365 (57h23m14.557649497s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.126253889 (57h23m14.55951602s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.128444349 (57h23m14.561706477s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.130234388 (57h23m14.563496519s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.131765784 (57h23m14.565027925s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Uploading piece...
Feb 12 21:38:14.716427455 (57h23m15.14968959s) [upload=50547]: Next returned nil segment
Feb 12 21:38:14.716446821 (57h23m15.149708956s) [upload=50547]: Waiting for error group managing segments...
Feb 12 21:38:14.846725705 (57h23m15.279987843s) [noise=true , address=77.90.31.13:4627 , piece_id=RDLX3XN6256R4H5ARNY6ERZSBHFSDYXRV7QYAGOLW4SFJMOJNLXQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.883514541 (57h23m15.316776673s) [noise=true , address=77.90.31.10:4517 , piece_id=MURNJXAXHNETMTFGQYJSMZM5KOINIB4ZN5LPDQMFQBZATROXZR7Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:14.897566468 (57h23m15.330828593s) [noise=true , address=79.127.219.41:4925 , piece_id=WJKWGIDLA3MVZMUJ32CJEAW6AQE5X2QD62YPJZIEARB7F52Y7EMQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.006318993 (57h23m15.439581181s) [noise=true , address=79.127.219.38:4822 , piece_id=VBSOEA27PC4ZIUA64LGWNIZF6JMBE67JHWPM77DA2WZ5GB5ILCTA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.507295907 (57h23m15.940558044s) [noise=true , address=145.79.227.19:4082 , piece_id=GKKRKVBAVVJPNX6OLB7R5UXFGLSHPASZEQXVVNMMW45ELHJKU6MQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:15.560742983 (57h23m15.994005108s) [noise=true , address=145.79.227.3:4154 , piece_id=HA37NSGFZQRLWH7ETAXIWFWTMCEY73X5XU2RBGFMGENZSBAPQ2FA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:16.194465369 (57h23m16.627727495s) [noise=true , address=145.79.227.7:4297 , piece_id=FXP7ZKUZIXRGFWQTOYSAQV3TVMAWXTPA4CESTRHBUSHVUJ7LM64Q, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:17.158722003 (57h23m17.591984127s) [noise=true , address=145.79.227.50:4379 , piece_id=JFFZMMU4YN4LBC25ZJZQZDHKA4YAOIWA6ARVRAC5YRLJ3WHEROZQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:36.622501269 (57h23m37.055763381s) [noise=true , address=145.79.227.35:4485 , piece_id=JE436GWUI5FW3FZIBPBHKOQKVNPFSN7EMRNPTTKKDF4OWV4LCEKA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:39.328146131 (57h23m39.761408267s) [noise=true , address=145.79.227.36:4520 , piece_id=4PYAT622CP4M6NVIY3KKN44CDED3G7DN5BMJ6GGWTT7MZ3BFPGQA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:40.486408635 (57h23m40.919670774s) [noise=true , address=145.79.227.25:4621 , piece_id=W2ASV4KBBKE5HAYTO4ZJPJ6EKH2J3WMY5II6S2NU5XACSLHC64YQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:42.732720561 (57h23m43.165982686s) [noise=true , address=145.79.227.13:4805 , piece_id=SE2AMRFTBR2PCQRUJ76IZD6MXLNKOV6BFVHQDYNFIOAW66GZTT7A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.005976134 (57h23m43.439238259s) [noise=true , address=145.79.227.23:4548 , piece_id=JEMBCTMY36VVD7BGIDLSRPPBGGAEJL3BBFPZQKLKTYE7KBF3IWTQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: <nil>
Feb 12 21:38:43.006002803 (57h23m43.439265001s) [seg_pos={0 0}, upload=50547]: Satisfied cohort requirements after 13 pieces
Feb 12 21:38:43.006267808 (57h23m43.439529943s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms): protocol: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42070->145.79.227.52:4450: use of closed network connection
Feb 12 21:38:43.006279373 (57h23m43.439541508s) [noise=true , address=145.79.227.52:4450 , piece_id=ZP7SGNOBEC3L7R4TOXKRAHY7URSSJ3KXQGGIXTXUSBNXK443TEGA, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006294630 (57h23m43.439556778s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq): protocol: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53122->145.79.227.26:4657: use of closed network connection
Feb 12 21:38:43.006325887 (57h23m43.439588017s) [noise=true , address=145.79.227.26:4657 , piece_id=QLAKD2TELXRWBKXOTGMMW4E7WWDEK52EJ5K25JNMW3S474M5EQUQ, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006665522 (57h23m43.439927656s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: write tcp 139.59.144.22:55994->145.79.227.38:4977: use of closed network connection; context canceled
Feb 12 21:38:43.006676662 (57h23m43.439938787s) [noise=true , address=145.79.227.38:4977 , piece_id=N76T57LIUHGZA5OQXE5AFF4AKUWGN7WZGJXVKQCZEBG5MH6KQB5A, seg_pos={0 0}, upload=50547]: Long tail cancellation due to optimal threshold
Feb 12 21:38:43.006697807 (57h23m43.439959922s) [seg_pos={0 0}, upload=50547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 12 21:38:43.006774081 (57h23m43.440036206s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 12 21:38:43.006779257 (57h23m43.440041382s) [upload=50547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 12 21:38:43.143421669 (57h23m43.576683785s) [upload=50547]: Done uploading segments...
Feb 12 21:38:43.143453083 (57h23m43.576715208s) [upload=50547]: Upload finished. err: <nil>
Duration: 2.403012413s
core
uplink
us-select-1
|