Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
948.509944ms
FAIL (2025-07-03 11:43:37 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 948.538298ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.026552645s
FAIL (2025-07-03 11:42:32 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.02667774s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
915.747644ms
FAIL (2025-07-03 11:42:04 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 915.840979ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
906.182727ms
FAIL (2025-07-03 11:41:25 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 906.217405ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
932.026636ms
FAIL (2025-07-03 11:40:53 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 932.059882ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
882.628187ms
FAIL (2025-07-03 11:40:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 882.662138ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
896.580994ms
FAIL (2025-07-03 11:39:41 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 896.608973ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.083974328s
FAIL (2025-07-03 11:38:48 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.084005844s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
911.36399ms
FAIL (2025-07-03 11:38:05 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 911.392878ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
955.412647ms
FAIL (2025-07-03 11:37:05 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 955.598711ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
868.363901ms
FAIL (2025-07-03 11:36:29 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 868.391963ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
889.331339ms
FAIL (2025-07-03 11:35:45 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 889.361306ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
871.400314ms
FAIL (2025-07-03 11:34:39 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 871.428279ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
959.607141ms
FAIL (2025-07-03 11:34:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 959.633536ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
924.67264ms
FAIL (2025-07-03 11:33:42 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 924.690771ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
942.473288ms
FAIL (2025-07-03 11:33:19 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 942.500229ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
910.454804ms
FAIL (2025-07-03 11:32:25 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 910.483178ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
979.437132ms
FAIL (2025-07-03 11:31:15 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 979.479182ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
939.067957ms
FAIL (2025-07-03 11:30:03 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 939.197731ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
918.964789ms
FAIL (2025-07-03 11:29:38 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 919.000242ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.151356464s
FAIL (2025-07-03 11:28:38 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.151385882s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
963.819798ms
FAIL (2025-07-03 11:28:01 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 963.844268ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.142825591s
FAIL (2025-07-03 11:27:17 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.142858648s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
869.962749ms
FAIL (2025-07-03 11:26:49 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 869.996222ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
887.58653ms
FAIL (2025-07-03 11:26:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 887.616206ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
903.618779ms
FAIL (2025-07-03 11:26:00 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 903.649163ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
995.466895ms
FAIL (2025-07-03 11:25:26 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 995.494187ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
903.256231ms
FAIL (2025-07-03 11:25:06 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 903.305053ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
889.408212ms
FAIL (2025-07-03 11:24:06 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 889.462476ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
886.79116ms
FAIL (2025-07-03 11:23:15 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 886.831298ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
902.10278ms
FAIL (2025-07-03 11:22:44 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 902.138885ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
879.98562ms
FAIL (2025-07-03 11:22:10 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 880.013188ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
862.392815ms
FAIL (2025-07-03 11:21:27 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 862.424257ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
926.127684ms
FAIL (2025-07-03 11:20:41 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 926.160434ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
891.100797ms
FAIL (2025-07-03 11:20:13 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 891.133808ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
917.611167ms
FAIL (2025-07-03 11:19:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 917.638904ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
982.451274ms
FAIL (2025-07-03 11:18:12 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 982.483013ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
898.476393ms
FAIL (2025-07-03 11:17:51 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 898.504302ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
866.034929ms
FAIL (2025-07-03 11:16:42 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 866.062559ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
896.940075ms
FAIL (2025-07-03 11:16:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 896.964953ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.064750188s
FAIL (2025-07-03 11:15:26 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.064777331s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.104122923s
FAIL (2025-07-03 11:14:37 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.104147675s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
900.702759ms
FAIL (2025-07-03 11:14:12 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 900.737743ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.113355955s
FAIL (2025-07-03 11:13:57 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.113386705s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
875.745874ms
FAIL (2025-07-03 11:13:18 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 875.771998ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.059009917s
FAIL (2025-07-03 11:12:27 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.059033013s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.155944444s
FAIL (2025-07-03 11:12:03 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.155980049s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
881.117863ms
FAIL (2025-07-03 11:11:46 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 881.150371ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
895.809432ms
FAIL (2025-07-03 11:10:54 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 895.838699ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
888.076662ms
FAIL (2025-07-03 11:09:52 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 888.103791ms
core
uplink
us-select-1
|