Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
869.962749ms
FAIL (2025-07-03 11:26:49 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 869.996222ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
887.58653ms
FAIL (2025-07-03 11:26:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 887.616206ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
903.618779ms
FAIL (2025-07-03 11:26:00 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 903.649163ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
995.466895ms
FAIL (2025-07-03 11:25:26 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 995.494187ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
903.256231ms
FAIL (2025-07-03 11:25:06 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 903.305053ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
889.408212ms
FAIL (2025-07-03 11:24:06 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 889.462476ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
886.79116ms
FAIL (2025-07-03 11:23:15 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 886.831298ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
902.10278ms
FAIL (2025-07-03 11:22:44 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 902.138885ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
879.98562ms
FAIL (2025-07-03 11:22:10 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 880.013188ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
862.392815ms
FAIL (2025-07-03 11:21:27 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 862.424257ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
926.127684ms
FAIL (2025-07-03 11:20:41 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 926.160434ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
891.100797ms
FAIL (2025-07-03 11:20:13 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 891.133808ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
917.611167ms
FAIL (2025-07-03 11:19:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 917.638904ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
982.451274ms
FAIL (2025-07-03 11:18:12 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 982.483013ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
898.476393ms
FAIL (2025-07-03 11:17:51 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 898.504302ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
866.034929ms
FAIL (2025-07-03 11:16:42 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 866.062559ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
896.940075ms
FAIL (2025-07-03 11:16:24 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 896.964953ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.064750188s
FAIL (2025-07-03 11:15:26 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.064777331s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.104122923s
FAIL (2025-07-03 11:14:37 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.104147675s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
900.702759ms
FAIL (2025-07-03 11:14:12 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 900.737743ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.113355955s
FAIL (2025-07-03 11:13:57 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.113386705s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
875.745874ms
FAIL (2025-07-03 11:13:18 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 875.771998ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.059009917s
FAIL (2025-07-03 11:12:27 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.059033013s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.155944444s
FAIL (2025-07-03 11:12:03 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.155980049s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
881.117863ms
FAIL (2025-07-03 11:11:46 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 881.150371ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
895.809432ms
FAIL (2025-07-03 11:10:54 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 895.838699ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
888.076662ms
FAIL (2025-07-03 11:09:52 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 888.103791ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
946.737841ms
FAIL (2025-07-03 11:09:33 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 946.771248ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
935.137109ms
FAIL (2025-07-03 11:08:19 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 935.174432ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
887.927782ms
FAIL (2025-07-03 11:07:59 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 888.035923ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
928.172813ms
FAIL (2025-07-03 11:07:21 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 928.211753ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
911.990327ms
FAIL (2025-07-03 11:06:40 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 912.020911ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.00476702s
FAIL (2025-07-03 11:05:48 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.004801125s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
918.458048ms
FAIL (2025-07-03 11:05:01 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 918.509093ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
940.759829ms
FAIL (2025-07-03 11:03:55 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 940.793443ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.002628813s
FAIL (2025-07-03 11:03:27 UTC)
orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 92141280 (241h45m47.575554915s) [upload=193315]: Done starting upload Jul 3 10:56:47.692210194 (241h45m47.575623815s) [upload=193315]: Uploading segments... Jul 3 10:56:47.692245102 (241h45m47.57565874s) [upload=193315]: Got next segment. Inline: true Jul 3 10:56:47.692263212 (241h45m47.575676841s) [upload=193315]: Waiting for error group managing segments... Jul 3 10:56:47.692302063 (241h45m47.575715676s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:56:47.692307690 (241h45m47.575721305s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:56:47.692311364 (241h45m47.575724977s) [upload=193315]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:56:47.877125903 (241h45m47.76053953s) [upload=193315]: Done uploading segments... Jul 3 10:56:47.877145693 (241h45m47.760559313s) [upload=193315]: Upload finished. err: <nil> Jul 3 10:57:29.864771860 (241h46m29.748185487s) [upload=193328]: Starting upload Jul 3 10:57:29.864828141 (241h46m29.748241771s) [upload=193328]: Done starting upload Jul 3 10:57:29.864943650 (241h46m29.748357266s) [upload=193328]: Uploading segments... Jul 3 10:57:29.864985764 (241h46m29.748399399s) [upload=193328]: Got next segment. Inline: true Jul 3 10:57:29.864993204 (241h46m29.74840683s) [upload=193328]: Waiting for error group managing segments... Jul 3 10:57:29.865040088 (241h46m29.748453714s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:57:29.865045828 (241h46m29.748459455s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:57:29.865049313 (241h46m29.748462939s) [upload=193328]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:57:30.016209856 (241h46m29.899623491s) [upload=193328]: Done uploading segments... Jul 3 10:57:30.016240023 (241h46m29.899653635s) [upload=193328]: Upload finished. err: <nil> Jul 3 10:58:18.141993741 (241h47m18.025407353s) [upload=193338]: Starting upload Jul 3 10:58:18.142061061 (241h47m18.02547469s) [upload=193338]: Done starting upload Jul 3 10:58:18.142135095 (241h47m18.025548723s) [upload=193338]: Uploading segments... Jul 3 10:58:18.142182869 (241h47m18.025596504s) [upload=193338]: Got next segment. Inline: true Jul 3 10:58:18.142187133 (241h47m18.02560076s) [upload=193338]: Waiting for error group managing segments... Jul 3 10:58:18.142224562 (241h47m18.025638174s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:18.142238844 (241h47m18.025652472s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:18.142241281 (241h47m18.025654907s) [upload=193338]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:18.299567757 (241h47m18.182981371s) [upload=193338]: Done uploading segments... Jul 3 10:58:18.299610191 (241h47m18.183023822s) [upload=193338]: Upload finished. err: <nil> Jul 3 10:58:47.777766979 (241h47m47.661180605s) [upload=193343]: Starting upload Jul 3 10:58:47.777836832 (241h47m47.661250464s) [upload=193343]: Done starting upload Jul 3 10:58:47.777979958 (241h47m47.661393586s) [upload=193343]: Uploading segments... Jul 3 10:58:47.778043884 (241h47m47.661457517s) [upload=193343]: Got next segment. Inline: true Jul 3 10:58:47.778049596 (241h47m47.661463222s) [upload=193343]: Waiting for error group managing segments... Jul 3 10:58:47.778104147 (241h47m47.661517773s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 10:58:47.778110475 (241h47m47.661524103s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 10:58:47.778116139 (241h47m47.661529765s) [upload=193343]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 10:58:47.927715833 (241h47m47.81112945s) [upload=193343]: Done uploading segments... Jul 3 10:58:47.927742125 (241h47m47.811155742s) [upload=193343]: Upload finished. err: <nil> Jul 3 11:00:03.142483257 (241h49m3.025896893s) [upload=193356]: Starting upload Jul 3 11:00:03.142574638 (241h49m3.025988277s) [upload=193356]: Done starting upload Jul 3 11:00:03.142656605 (241h49m3.026070226s) [upload=193356]: Uploading segments... Jul 3 11:00:03.142714076 (241h49m3.026127718s) [upload=193356]: Got next segment. Inline: true Jul 3 11:00:03.142719656 (241h49m3.026133274s) [upload=193356]: Waiting for error group managing segments... Jul 3 11:00:03.142775399 (241h49m3.02618904s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:03.142784075 (241h49m3.026197718s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:03.142788014 (241h49m3.026201632s) [upload=193356]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:03.413413596 (241h49m3.296827246s) [upload=193356]: Done uploading segments... Jul 3 11:00:03.413460073 (241h49m3.296873703s) [upload=193356]: Upload finished. err: <nil> Jul 3 11:00:47.511786974 (241h49m47.39520059s) [upload=193367]: Starting upload Jul 3 11:00:47.511861701 (241h49m47.395275319s) [upload=193367]: Done starting upload Jul 3 11:00:47.511957156 (241h49m47.39537077s) [upload=193367]: Uploading segments... Jul 3 11:00:47.512038777 (241h49m47.395452425s) [upload=193367]: Got next segment. Inline: true Jul 3 11:00:47.512046607 (241h49m47.395460232s) [upload=193367]: Waiting for error group managing segments... Jul 3 11:00:47.512104973 (241h49m47.395518599s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:00:47.512118685 (241h49m47.395532304s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:00:47.512123632 (241h49m47.395537245s) [upload=193367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:00:47.851149392 (241h49m47.734563055s) [upload=193367]: Done uploading segments... Jul 3 11:00:47.851175680 (241h49m47.734589316s) [upload=193367]: Upload finished. err: <nil> Jul 3 11:01:43.222334376 (241h50m43.105747996s) [upload=193381]: Starting upload Jul 3 11:01:43.222398447 (241h50m43.105812064s) [upload=193381]: Done starting upload Jul 3 11:01:43.222464323 (241h50m43.105877942s) [upload=193381]: Uploading segments... Jul 3 11:01:43.222511742 (241h50m43.105925383s) [upload=193381]: Got next segment. Inline: true Jul 3 11:01:43.222519229 (241h50m43.105932969s) [upload=193381]: Waiting for error group managing segments... Jul 3 11:01:43.222571369 (241h50m43.105984996s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:01:43.222577365 (241h50m43.105990995s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:01:43.222580905 (241h50m43.105994532s) [upload=193381]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:01:43.383478176 (241h50m43.266891789s) [upload=193381]: Done uploading segments... Jul 3 11:01:43.383495943 (241h50m43.266909556s) [upload=193381]: Upload finished. err: <nil> Jul 3 11:02:04.710352041 (241h51m4.593765678s) [upload=193387]: Starting upload Jul 3 11:02:04.710460350 (241h51m4.593874169s) [upload=193387]: Done starting upload Jul 3 11:02:04.710524219 (241h51m4.593937847s) [upload=193387]: Uploading segments... Jul 3 11:02:04.710614733 (241h51m4.594028369s) [upload=193387]: Got next segment. Inline: true Jul 3 11:02:04.710621857 (241h51m4.594035534s) [upload=193387]: Waiting for error group managing segments... Jul 3 11:02:04.710694506 (241h51m4.594108134s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:04.710702105 (241h51m4.594115732s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:04.710706207 (241h51m4.594119834s) [upload=193387]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:04.871874956 (241h51m4.755288591s) [upload=193387]: Done uploading segments... Jul 3 11:02:04.871901585 (241h51m4.755315212s) [upload=193387]: Upload finished. err: <nil> Jul 3 11:02:31.431592759 (241h51m31.315006373s) [upload=193392]: Starting upload Jul 3 11:02:31.431744767 (241h51m31.315158386s) [upload=193392]: Done starting upload Jul 3 11:02:31.431860032 (241h51m31.315273649s) [upload=193392]: Uploading segments... Jul 3 11:02:31.431927625 (241h51m31.31534126s) [upload=193392]: Got next segment. Inline: true Jul 3 11:02:31.431934169 (241h51m31.315347796s) [upload=193392]: Waiting for error group managing segments... Jul 3 11:02:31.431978297 (241h51m31.31539191s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:02:31.431984868 (241h51m31.31539848s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:02:31.431987895 (241h51m31.31540152s) [upload=193392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:02:31.656380106 (241h51m31.539793758s) [upload=193392]: Done uploading segments... Jul 3 11:02:31.656425956 (241h51m31.539839603s) [upload=193392]: Upload finished. err: <nil> Jul 3 11:03:27.384087695 (241h52m27.267501328s) [upload=193403]: Starting upload Jul 3 11:03:27.384183209 (241h52m27.267596845s) [upload=193403]: Done starting upload Jul 3 11:03:27.384272136 (241h52m27.267685768s) [upload=193403]: Uploading segments... Jul 3 11:03:27.384343834 (241h52m27.267757471s) [upload=193403]: Got next segment. Inline: true Jul 3 11:03:27.384351337 (241h52m27.267764966s) [upload=193403]: Waiting for error group managing segments... Jul 3 11:03:27.384408842 (241h52m27.267822474s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 3 11:03:27.384417106 (241h52m27.267830842s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 3 11:03:27.384421732 (241h52m27.267835368s) [upload=193403]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 3 11:03:27.541285766 (241h52m27.424699395s) [upload=193403]: Done uploading segments... Jul 3 11:03:27.541312525 (241h52m27.424726153s) [upload=193403]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.002659685s
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
325.897307ms
FAIL (2025-07-03 05:17:07 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 325.929204ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
344.429373ms
FAIL (2025-07-03 05:16:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 344.464639ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
864.962588ms
FAIL (2025-07-03 05:15:34 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 864.988108ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
352.923001ms
FAIL (2025-07-03 05:14:43 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 352.952339ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
305.036636ms
FAIL (2025-07-03 05:14:21 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 305.082394ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
346.952341ms
FAIL (2025-07-03 05:13:40 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 346.97087ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
304.443812ms
FAIL (2025-07-03 05:12:37 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 304.470263ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
437.236004ms
FAIL (2025-07-03 05:11:39 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 437.266047ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
331.989028ms
FAIL (2025-07-03 05:10:35 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 332.019344ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
336.364357ms
FAIL (2025-07-03 05:10:01 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 336.401199ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
366.059535ms
FAIL (2025-07-03 05:09:24 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 366.09244ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
300.689704ms
FAIL (2025-07-03 05:08:57 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 300.722011ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
374.106706ms
FAIL (2025-07-03 05:08:28 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 374.143986ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
317.886851ms
FAIL (2025-07-03 05:07:37 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 317.917271ms
edge
authservice
http
us-select-1
|