Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.515907237s
FAIL (2025-10-09 09:38:24 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.515936412s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.547631368s
FAIL (2025-10-09 09:37:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.54767178s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.642726313s
FAIL (2025-10-09 09:36:35 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.642762291s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.605741646s
FAIL (2025-10-09 09:35:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.605771301s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.64731715s
FAIL (2025-10-09 09:34:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.647375682s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.472710516s
FAIL (2025-10-09 09:33:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.47275353s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.736938729s
FAIL (2025-10-09 09:32:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.736966244s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.636403281s
FAIL (2025-10-09 09:31:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.636449677s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.777481806s
FAIL (2025-10-09 09:31:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.777626839s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.486282311s
FAIL (2025-10-09 09:31:08 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 2.486313546s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000977687s
FAIL (2025-10-09 09:30:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 9 09:30:04.279586783 (379h59m52.083051362s) [upload=334301]: Starting upload Oct 9 09:30:04.279686300 (379h59m52.083150885s) [upload=334301]: Done starting upload Oct 9 09:30:04.279802761 (379h59m52.083267359s) [upload=334301]: Uploading segments... Oct 9 09:30:04.280163317 (379h59m52.083627926s) [upload=334301]: Got next segment. Inline: false Oct 9 09:30:04.280254776 (379h59m52.083719357s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 9 09:30:04.280262417 (379h59m52.083727009s) [upload=334301]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 9 09:30:33.964911659 (380h0m21.768376256s) [upload=334301]: Next segment err: context deadline exceeded Oct 9 09:30:33.965063344 (380h0m21.768527967s) [upload=334301]: Done uploading segments... Oct 9 09:30:33.965085062 (380h0m21.768549639s) [upload=334301]: Upload finished. err: context deadline exceeded
Duration: 30.001008137s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.534237511s
FAIL (2025-10-09 00:31:22 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.534268298s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.498858983s
FAIL (2025-10-09 00:30:48 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.49889008s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.45568252s
FAIL (2025-10-09 00:29:50 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.455709158s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.458069369s
FAIL (2025-10-09 00:28:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.458101947s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.518025431s
FAIL (2025-10-09 00:27:58 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.5180602s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.545166496s
FAIL (2025-10-09 00:27:37 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.545204169s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.683076005s
FAIL (2025-10-09 00:26:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.68326328s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.579571957s
FAIL (2025-10-09 00:25:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.579671111s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.596566289s
FAIL (2025-10-09 00:25:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.596599897s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.608605489s
FAIL (2025-10-09 00:24:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.608632408s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.522312841s
FAIL (2025-10-09 00:23:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.522345669s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.502615399s
FAIL (2025-10-09 00:22:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.502705406s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.684706717s
FAIL (2025-10-09 00:22:09 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.684794447s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.580807259s
FAIL (2025-10-09 00:21:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.580905951s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.492801821s
FAIL (2025-10-09 00:21:03 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.492897707s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.515353618s
FAIL (2025-10-09 00:20:00 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.515391262s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.566664901s
FAIL (2025-10-09 00:18:47 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.566697599s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.509413904s
FAIL (2025-10-09 00:18:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.509521326s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.50414758s
FAIL (2025-10-09 00:17:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.504186711s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.547494606s
FAIL (2025-10-09 00:16:50 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.547530342s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.534220078s
FAIL (2025-10-09 00:16:27 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.534259062s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.595300549s
FAIL (2025-10-09 00:15:32 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.59534083s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.512227643s
FAIL (2025-10-09 00:14:50 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.51225801s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.516953932s
FAIL (2025-10-09 00:13:36 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.517119246s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.474209153s
FAIL (2025-10-09 00:13:08 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.474243288s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.451077932s
FAIL (2025-10-09 00:11:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.451111692s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.515758748s
FAIL (2025-10-09 00:11:04 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.515799866s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.612418433s
FAIL (2025-10-09 00:10:17 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.612478827s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.546418902s
FAIL (2025-10-09 00:09:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.546453568s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.504838988s
FAIL (2025-10-09 00:08:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.504938829s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.777714051s
FAIL (2025-10-09 00:07:34 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.777772211s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.568998896s
FAIL (2025-10-09 00:06:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.569031568s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.556844419s
FAIL (2025-10-09 00:06:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.556959337s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.595937533s
FAIL (2025-10-09 00:05:21 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.595973742s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.609415589s
FAIL (2025-10-09 00:04:46 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.60944945s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.56154872s
FAIL (2025-10-09 00:04:00 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.561586803s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.536962049s
FAIL (2025-10-09 00:03:33 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.537089103s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.693407784s
FAIL (2025-10-09 00:02:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.69343859s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.792420206s
FAIL (2025-10-09 00:01:11 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Oct 8 23:44:46.921985642 (370h14m34.725450224s) [upload=325729]: Starting upload
Oct 8 23:44:46.922065428 (370h14m34.725530028s) [upload=325729]: Done starting upload
Oct 8 23:44:46.922142420 (370h14m34.725607004s) [upload=325729]: Uploading segments...
Oct 8 23:44:46.922425032 (370h14m34.725889624s) [upload=325729]: Got next segment. Inline: false
Oct 8 23:44:46.922479564 (370h14m34.725944159s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 23:44:46.922495772 (370h14m34.725960368s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 23:44:47.034162922 (370h14m34.837627521s) [seg_pos={0 0}, upload=325729]: Begin upload segment...
Oct 8 23:44:47.034254844 (370h14m34.837719443s) [seg_pos={0 0}, upload=325729]: Done begin upload segment.
Oct 8 23:44:47.034283487 (370h14m34.837748088s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034298536 (370h14m34.83776313s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034418373 (370h14m34.837882965s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.034964915 (370h14m34.838429516s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.035520165 (370h14m34.838984766s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036117717 (370h14m34.839582318s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.036877301 (370h14m34.84034189s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.039108249 (370h14m34.842572846s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.040992741 (370h14m34.844457341s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.042971230 (370h14m34.84643583s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.045131872 (370h14m34.848596472s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.046982560 (370h14m34.85044716s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.048622564 (370h14m34.852087165s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.052953512 (370h14m34.856418111s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.055683022 (370h14m34.859147645s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.057343404 (370h14m34.860808003s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Uploading piece...
Oct 8 23:44:47.732471340 (370h14m35.535935932s) [upload=325729]: Next returned nil segment
Oct 8 23:44:47.732496477 (370h14m35.535961081s) [upload=325729]: Waiting for error group managing segments...
Oct 8 23:44:47.844246193 (370h14m35.647710771s) [noise=true , address=145.79.227.17:4971 , piece_id=25KKFVIUAJWVW6KLWBCGVVLGYFRPIPTZXTPEDWXWE6NRQ2OIXQDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.857316735 (370h14m35.66078132s) [noise=true , address=145.79.227.14:4839 , piece_id=KZ7UCSYWUG5S6MDB5KUMZMX22ER4FSYZCZHXUVCRNTNEPRFPU6SQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.858594726 (370h14m35.662059319s) [noise=true , address=145.79.227.5:4222 , piece_id=AE5YN46TA3OOIDO2LHMCBEBLPMRMRIHDR64QXD6HGCINHJWAIJEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.879701307 (370h14m35.683165907s) [noise=true , address=145.79.227.16:4912 , piece_id=F2RTQY4ALBGCGVIZPPZ3WKZRMC4CVIHV2UD4VZNW43MGHITJ3BFA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.883631165 (370h14m35.687095757s) [noise=true , address=145.79.227.13:4803 , piece_id=BKYV46SC6A2KYMHU7HQY2BN36PLGGC7YN2DYKGA7I6F5VDZDODKQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:44:47.905105726 (370h14m35.708570308s) [noise=true , address=145.79.227.6:4259 , piece_id=M6GSWSDYCCS4ZDGPH2UZ6E655F7V74S57KCX55KE4OCTRWETEB4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.189089108 (370h14m54.992553703s) [noise=true , address=45.91.116.12:4225 , piece_id=NVKCJLKFD5QAL3DOQCW7BPTUT3YUZHOGLL7R52XKLP6K3CY76Z7A, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.312744942 (370h14m55.116209531s) [noise=true , address=77.90.31.11:4554 , piece_id=BULPH6CRE5XQ3UHOF2OQX4B557E4KB27CMSVEGWGS7F3K4L4WYSA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:07.899473224 (370h14m55.702937825s) [noise=true , address=45.91.116.7:4011 , piece_id=DB3PSB6POMJPCELXHK3JZ2WK36RLMZYBSIKLC2HXDBWUILLVELDA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:08.498620127 (370h14m56.302084724s) [noise=true , address=45.91.116.14:4191 , piece_id=LDAGYID3QXC52REXG4KQGDMVEVODGC324HXDDLE5RG6RDHNC2I4Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:09.179678264 (370h14m56.983142848s) [noise=true , address=77.90.31.19:4842 , piece_id=4EHS3GMJFJ2PDWWMKGPQOF7FVHBQOFYMALEEY77CK7BCPTLPCXEA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:12.172576799 (370h14m59.976041381s) [noise=true , address=45.91.116.9:4263 , piece_id=5DNIFZYH5XQ4JPXSSBEH43QP4ZYG5JSXMTQRSOGVIDPM6EJFZS6Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427171301 (370h15m4.230635886s) [noise=true , address=45.91.116.11:4119 , piece_id=4YRQZZEKYOAJ4DGPCTHNUWBEH5ZR6UMFN6222XCXIDHEU4RGPURA, seg_pos={0 0}, upload=325729]: Done uploading piece. err: <nil>
Oct 8 23:45:16.427206387 (370h15m4.230670978s) [seg_pos={0 0}, upload=325729]: Segment reached optimal threshold of 13 pieces.
Oct 8 23:45:16.427507762 (370h15m4.230972341s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427519892 (370h15m4.230984489s) [noise=true , address=77.90.31.11:4555 , piece_id=E3NDVLO6MKC6QVFNP4WDBSP7TGFL3PBHF2H5XRAQJ5QH37SIJ3RQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427615585 (370h15m4.231080163s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Oct 8 23:45:16.427621042 (370h15m4.231085632s) [noise=true , address=45.91.116.10:4083 , piece_id=2H6PTKTZYM5BUB53QUVQZ6TTI6ZUXNNUOCCP7SPKJYJ7LDLDAV7Q, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427627817 (370h15m4.231092409s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 139.59.144.22:51230->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 23:45:16.427638240 (370h15m4.231102842s) [noise=true , address=45.91.116.4:4939 , piece_id=LCTMD4KPYZCKAWXGY7HUKDTMYY4MSNYFRSI6PLCFBN4IXY6ICBMQ, seg_pos={0 0}, upload=325729]: Long tail cancellation due to optimal threshold
Oct 8 23:45:16.427661455 (370h15m4.231126048s) [seg_pos={0 0}, upload=325729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 8 23:45:16.427765388 (370h15m4.23122998s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 8 23:45:16.427772109 (370h15m4.2312367s) [upload=325729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 8 23:45:16.578651028 (370h15m4.382115632s) [upload=325729]: Done uploading segments...
Oct 8 23:45:16.578675673 (370h15m4.382140257s) [upload=325729]: Upload finished. err: <nil>
Duration: 2.792482368s
core
uplink
us-select-1
|