Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.02848458s FAIL (2025-07-12 10:22:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 1.028524336s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 874.90992ms FAIL (2025-07-12 10:21:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 874.942789ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 872.198974ms FAIL (2025-07-12 10:20:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 872.226694ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 980.365904ms FAIL (2025-07-12 10:20:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 980.392962ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 967.709698ms FAIL (2025-07-12 10:19:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 967.738443ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 888.372669ms FAIL (2025-07-12 10:18:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 888.398931ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 876.845685ms FAIL (2025-07-12 10:18:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 876.878112ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 900.397126ms FAIL (2025-07-12 10:17:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 900.447796ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 940.632367ms FAIL (2025-07-12 10:16:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 940.657639ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 894.785759ms FAIL (2025-07-12 10:16:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 894.817365ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 867.214551ms FAIL (2025-07-12 10:15:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 867.246322ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 868.850589ms FAIL (2025-07-12 10:14:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 868.888601ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 885.203255ms FAIL (2025-07-12 10:13:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 885.231775ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 933.711357ms FAIL (2025-07-12 10:13:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 933.742667ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 912.37053ms FAIL (2025-07-12 10:12:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 912.400957ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 889.564221ms FAIL (2025-07-12 10:11:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 889.591199ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 925.448273ms FAIL (2025-07-12 10:10:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 925.474496ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 839.781138ms FAIL (2025-07-12 10:09:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 839.813571ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 915.755576ms FAIL (2025-07-12 10:08:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 915.786118ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 915.895827ms FAIL (2025-07-12 10:08:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 915.9265ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 935.157932ms FAIL (2025-07-12 10:07:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 935.214619ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 852.538665ms FAIL (2025-07-12 10:06:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 852.566481ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 835.929297ms FAIL (2025-07-12 10:05:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 836.009563ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 984.18514ms FAIL (2025-07-12 10:05:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 984.287431ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 890.669197ms FAIL (2025-07-12 10:04:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 890.703078ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 926.719584ms FAIL (2025-07-12 10:03:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 926.762653ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 894.906937ms FAIL (2025-07-12 10:03:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 894.932686ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001043789s FAIL (2025-07-12 10:03:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : Starting upload Jul 12 09:54:55.149617136 (456h43m55.033030761s) [upload=365226]: Done starting upload Jul 12 09:54:55.149694303 (456h43m55.03310792s) [upload=365226]: Uploading segments... Jul 12 09:54:55.149745747 (456h43m55.033159375s) [upload=365226]: Got next segment. Inline: true Jul 12 09:54:55.149750696 (456h43m55.033164309s) [upload=365226]: Waiting for error group managing segments... Jul 12 09:54:55.149786551 (456h43m55.033200164s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:54:55.149792069 (456h43m55.033205682s) [upload=365226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:54:55.149794752 (456h43m55.033208364s) [upload=365226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:54:55.304909122 (456h43m55.188322754s) [upload=365226]: Done uploading segments... Jul 12 09:54:55.304931312 (456h43m55.188344948s) [upload=365226]: Upload finished. err: <nil> Jul 12 09:55:31.688941506 (456h44m31.572355136s) [upload=365232]: Starting upload Jul 12 09:55:31.689018817 (456h44m31.572432446s) [upload=365232]: Done starting upload Jul 12 09:55:31.689099781 (456h44m31.5725134s) [upload=365232]: Uploading segments... Jul 12 09:55:31.689150694 (456h44m31.572564323s) [upload=365232]: Got next segment. Inline: true Jul 12 09:55:31.689156625 (456h44m31.572570252s) [upload=365232]: Waiting for error group managing segments... Jul 12 09:55:31.689199973 (456h44m31.5726136s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:55:31.689222045 (456h44m31.572635674s) [upload=365232]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:55:31.689225646 (456h44m31.572639272s) [upload=365232]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:55:31.900996264 (456h44m31.784409891s) [upload=365232]: Done uploading segments... Jul 12 09:55:31.901018299 (456h44m31.784431912s) [upload=365232]: Upload finished. err: <nil> Jul 12 09:56:41.350926655 (456h45m41.234340281s) [upload=365249]: Starting upload Jul 12 09:56:41.350993235 (456h45m41.234406861s) [upload=365249]: Done starting upload Jul 12 09:56:41.351081276 (456h45m41.234494888s) [upload=365249]: Uploading segments... Jul 12 09:56:41.351129662 (456h45m41.234543297s) [upload=365249]: Got next segment. Inline: true Jul 12 09:56:41.351134559 (456h45m41.234548171s) [upload=365249]: Waiting for error group managing segments... Jul 12 09:56:41.351170020 (456h45m41.234583633s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:56:41.351174665 (456h45m41.234588277s) [upload=365249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:56:41.351177347 (456h45m41.234590959s) [upload=365249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:56:41.509201073 (456h45m41.392614687s) [upload=365249]: Done uploading segments... Jul 12 09:56:41.509245862 (456h45m41.392659478s) [upload=365249]: Upload finished. err: <nil> Jul 12 09:57:03.893802711 (456h46m3.777216347s) [upload=365255]: Starting upload Jul 12 09:57:03.893886509 (456h46m3.777300147s) [upload=365255]: Done starting upload Jul 12 09:57:03.894244374 (456h46m3.777658005s) [upload=365255]: Uploading segments... Jul 12 09:57:03.894320826 (456h46m3.777734464s) [upload=365255]: Got next segment. Inline: true Jul 12 09:57:03.894328744 (456h46m3.777742386s) [upload=365255]: Waiting for error group managing segments... Jul 12 09:57:03.894388055 (456h46m3.777801683s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:03.894398402 (456h46m3.777812038s) [upload=365255]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:03.894402222 (456h46m3.777815855s) [upload=365255]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:04.031145630 (456h46m3.91455925s) [upload=365255]: Done uploading segments... Jul 12 09:57:04.031162832 (456h46m3.914576451s) [upload=365255]: Upload finished. err: <nil> Jul 12 09:57:46.235008108 (456h46m46.118421723s) [upload=365264]: Starting upload Jul 12 09:57:46.235134681 (456h46m46.118548315s) [upload=365264]: Done starting upload Jul 12 09:57:46.235259195 (456h46m46.118672835s) [upload=365264]: Uploading segments... Jul 12 09:57:46.235342410 (456h46m46.118756048s) [upload=365264]: Got next segment. Inline: true Jul 12 09:57:46.235349710 (456h46m46.118763338s) [upload=365264]: Waiting for error group managing segments... Jul 12 09:57:46.235406294 (456h46m46.118819908s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:57:46.235413329 (456h46m46.118826956s) [upload=365264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:57:46.235418042 (456h46m46.118831671s) [upload=365264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:57:46.411244334 (456h46m46.294657962s) [upload=365264]: Done uploading segments... Jul 12 09:57:46.411263385 (456h46m46.294676998s) [upload=365264]: Upload finished. err: <nil> Jul 12 09:58:45.645147799 (456h47m45.528561428s) [upload=365276]: Starting upload Jul 12 09:58:45.645209345 (456h47m45.528622971s) [upload=365276]: Done starting upload Jul 12 09:58:45.645269169 (456h47m45.528682783s) [upload=365276]: Uploading segments... Jul 12 09:58:45.645308209 (456h47m45.528721844s) [upload=365276]: Got next segment. Inline: true Jul 12 09:58:45.645312533 (456h47m45.528726159s) [upload=365276]: Waiting for error group managing segments... Jul 12 09:58:45.645349909 (456h47m45.528763535s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:58:45.645354770 (456h47m45.528768395s) [upload=365276]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:58:45.645357086 (456h47m45.528770712s) [upload=365276]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:58:45.786444665 (456h47m45.66985829s) [upload=365276]: Done uploading segments... Jul 12 09:58:45.786465636 (456h47m45.669879263s) [upload=365276]: Upload finished. err: <nil> Jul 12 09:59:48.801535619 (456h48m48.684949239s) [upload=365292]: Starting upload Jul 12 09:59:48.801605032 (456h48m48.685018659s) [upload=365292]: Done starting upload Jul 12 09:59:48.801690907 (456h48m48.685104527s) [upload=365292]: Uploading segments... Jul 12 09:59:48.801733893 (456h48m48.68514753s) [upload=365292]: Got next segment. Inline: true Jul 12 09:59:48.801740622 (456h48m48.685154253s) [upload=365292]: Waiting for error group managing segments... Jul 12 09:59:48.801790074 (456h48m48.6852037s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 09:59:48.801795809 (456h48m48.685209435s) [upload=365292]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 09:59:48.801799046 (456h48m48.685212672s) [upload=365292]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 09:59:49.295085477 (456h48m49.17849911s) [upload=365292]: Done uploading segments... Jul 12 09:59:49.295125412 (456h48m49.178539028s) [upload=365292]: Upload finished. err: <nil> Jul 12 10:00:43.422496312 (456h49m43.305909941s) [upload=365306]: Starting upload Jul 12 10:00:43.422566837 (456h49m43.305980454s) [upload=365306]: Done starting upload Jul 12 10:00:43.422648044 (456h49m43.306061658s) [upload=365306]: Uploading segments... Jul 12 10:00:43.422734937 (456h49m43.306148574s) [upload=365306]: Got next segment. Inline: true Jul 12 10:00:43.422743166 (456h49m43.306156804s) [upload=365306]: Waiting for error group managing segments... Jul 12 10:00:43.422803041 (456h49m43.306216658s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:00:43.422809093 (456h49m43.306222711s) [upload=365306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:00:43.422812342 (456h49m43.306225959s) [upload=365306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:00:43.572364279 (456h49m43.455777914s) [upload=365306]: Done uploading segments... Jul 12 10:00:43.572398490 (456h49m43.455812104s) [upload=365306]: Upload finished. err: <nil> Jul 12 10:01:08.565165981 (456h50m8.448579607s) [upload=365312]: Starting upload Jul 12 10:01:08.565223259 (456h50m8.448636891s) [upload=365312]: Done starting upload Jul 12 10:01:08.565284941 (456h50m8.448698575s) [upload=365312]: Uploading segments... Jul 12 10:01:08.565324255 (456h50m8.44873789s) [upload=365312]: Got next segment. Inline: true Jul 12 10:01:08.565329767 (456h50m8.448743393s) [upload=365312]: Waiting for error group managing segments... Jul 12 10:01:08.565370213 (456h50m8.44878384s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:08.565375755 (456h50m8.44878938s) [upload=365312]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:08.565379028 (456h50m8.448792654s) [upload=365312]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:08.856266160 (456h50m8.739679798s) [upload=365312]: Done uploading segments... Jul 12 10:01:08.856285860 (456h50m8.739699488s) [upload=365312]: Upload finished. err: <nil> Jul 12 10:01:36.367080769 (456h50m36.250494387s) [upload=365320]: Starting upload Jul 12 10:01:36.367147081 (456h50m36.25056071s) [upload=365320]: Done starting upload Jul 12 10:01:36.367227641 (456h50m36.250641265s) [upload=365320]: Uploading segments... Jul 12 10:01:36.367281403 (456h50m36.250695041s) [upload=365320]: Got next segment. Inline: true Jul 12 10:01:36.367288287 (456h50m36.250701925s) [upload=365320]: Waiting for error group managing segments... Jul 12 10:01:36.367335784 (456h50m36.250749399s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 10:01:36.367341428 (456h50m36.250755058s) [upload=365320]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 10:01:36.367344620 (456h50m36.25075825s) [upload=365320]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 10:01:36.517869938 (456h50m36.401283565s) [upload=365320]: Done uploading segments... Jul 12 10:01:36.517887070 (456h50m36.401300682s) [upload=365320]: Upload finished. err: <nil> Duration: 30.001069032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.146645309s FAIL (2025-07-12 06:46:59 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 1.146673576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 886.678776ms FAIL (2025-07-12 06:45:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 886.726084ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 925.447521ms FAIL (2025-07-12 06:45:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 925.489955ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.109376051s FAIL (2025-07-12 06:44:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 1.109406939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 854.353523ms FAIL (2025-07-12 06:43:59 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 854.39335ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 896.473753ms FAIL (2025-07-12 06:43:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 896.505558ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 902.718621ms FAIL (2025-07-12 06:43:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 902.751184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 951.46436ms FAIL (2025-07-12 06:42:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 951.491178ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 934.241658ms FAIL (2025-07-12 06:41:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 934.274268ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.01022088s FAIL (2025-07-12 06:40:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 1.01025493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 918.439429ms FAIL (2025-07-12 06:39:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 918.470475ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.006460652s FAIL (2025-07-12 06:39:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 1.006510115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 896.315822ms FAIL (2025-07-12 06:38:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 896.349705ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 877.820722ms FAIL (2025-07-12 06:37:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 877.848589ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.023385135s FAIL (2025-07-12 06:36:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 1.023415405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 885.226455ms FAIL (2025-07-12 06:35:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 885.253769ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 935.716908ms FAIL (2025-07-12 06:35:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 935.743466ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 885.392477ms FAIL (2025-07-12 06:34:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 885.429595ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 882.652137ms FAIL (2025-07-12 06:33:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 882.685853ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 886.549959ms FAIL (2025-07-12 06:32:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 886.597579ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 997.918396ms FAIL (2025-07-12 06:31:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 997.948456ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 903.753367ms FAIL (2025-07-12 06:31:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 12 05:54:05.214783366 (452h43m5.098196999s) [upload=362006]: Done starting upload Jul 12 05:54:05.214943524 (452h43m5.098357152s) [upload=362006]: Uploading segments... Jul 12 05:54:05.215028467 (452h43m5.098442102s) [upload=362006]: Got next segment. Inline: true Jul 12 05:54:05.215037994 (452h43m5.098451632s) [upload=362006]: Waiting for error group managing segments... Jul 12 05:54:05.215124043 (452h43m5.098537659s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:54:05.215133464 (452h43m5.098547098s) [upload=362006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:54:05.215139439 (452h43m5.098553073s) [upload=362006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:54:05.350144515 (452h43m5.233558135s) [upload=362006]: Done uploading segments... Jul 12 05:54:05.350163715 (452h43m5.233577333s) [upload=362006]: Upload finished. err: <nil> Jul 12 05:55:07.569452170 (452h44m7.452865799s) [upload=362020]: Starting upload Jul 12 05:55:07.569518042 (452h44m7.452931674s) [upload=362020]: Done starting upload Jul 12 05:55:07.569617328 (452h44m7.453030949s) [upload=362020]: Uploading segments... Jul 12 05:55:07.569709616 (452h44m7.453123251s) [upload=362020]: Got next segment. Inline: true Jul 12 05:55:07.569715872 (452h44m7.453129498s) [upload=362020]: Waiting for error group managing segments... Jul 12 05:55:07.569786811 (452h44m7.453200428s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:07.569794452 (452h44m7.453208079s) [upload=362020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:07.569798041 (452h44m7.453211667s) [upload=362020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:07.734866378 (452h44m7.618280007s) [upload=362020]: Done uploading segments... Jul 12 05:55:07.734890776 (452h44m7.618304402s) [upload=362020]: Upload finished. err: <nil> Jul 12 05:55:42.300457718 (452h44m42.183871344s) [upload=362028]: Starting upload Jul 12 05:55:42.300520891 (452h44m42.183934517s) [upload=362028]: Done starting upload Jul 12 05:55:42.300586769 (452h44m42.184000387s) [upload=362028]: Uploading segments... Jul 12 05:55:42.300630845 (452h44m42.184044481s) [upload=362028]: Got next segment. Inline: true Jul 12 05:55:42.300636740 (452h44m42.184050367s) [upload=362028]: Waiting for error group managing segments... Jul 12 05:55:42.300679682 (452h44m42.184093308s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:55:42.300685330 (452h44m42.184098956s) [upload=362028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:55:42.300690591 (452h44m42.184104217s) [upload=362028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:55:42.463812846 (452h44m42.34722647s) [upload=362028]: Done uploading segments... Jul 12 05:55:42.463838009 (452h44m42.347251625s) [upload=362028]: Upload finished. err: <nil> Jul 12 05:56:57.219717464 (452h45m57.103131091s) [upload=362044]: Starting upload Jul 12 05:56:57.219777850 (452h45m57.103191475s) [upload=362044]: Done starting upload Jul 12 05:56:57.219834673 (452h45m57.103248285s) [upload=362044]: Uploading segments... Jul 12 05:56:57.219873712 (452h45m57.103287344s) [upload=362044]: Got next segment. Inline: true Jul 12 05:56:57.219878315 (452h45m57.103291941s) [upload=362044]: Waiting for error group managing segments... Jul 12 05:56:57.219916750 (452h45m57.103330376s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:56:57.219921361 (452h45m57.103334987s) [upload=362044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:56:57.219923846 (452h45m57.103337473s) [upload=362044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:56:57.360992207 (452h45m57.244405821s) [upload=362044]: Done uploading segments... Jul 12 05:56:57.361011314 (452h45m57.244424927s) [upload=362044]: Upload finished. err: <nil> Jul 12 05:57:50.271258395 (452h46m50.154672025s) [upload=362059]: Starting upload Jul 12 05:57:50.271354465 (452h46m50.154768101s) [upload=362059]: Done starting upload Jul 12 05:57:50.271493330 (452h46m50.154906943s) [upload=362059]: Uploading segments... Jul 12 05:57:50.271555529 (452h46m50.154969169s) [upload=362059]: Got next segment. Inline: true Jul 12 05:57:50.271562118 (452h46m50.154975748s) [upload=362059]: Waiting for error group managing segments... Jul 12 05:57:50.271628175 (452h46m50.155041802s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:57:50.271634604 (452h46m50.155048231s) [upload=362059]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:57:50.271638113 (452h46m50.155051741s) [upload=362059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:57:50.409186354 (452h46m50.292599971s) [upload=362059]: Done uploading segments... Jul 12 05:57:50.409205433 (452h46m50.292619049s) [upload=362059]: Upload finished. err: <nil> Jul 12 05:58:40.678652401 (452h47m40.562066013s) [upload=362068]: Starting upload Jul 12 05:58:40.678738970 (452h47m40.562152595s) [upload=362068]: Done starting upload Jul 12 05:58:40.678889497 (452h47m40.562303117s) [upload=362068]: Uploading segments... Jul 12 05:58:40.678996054 (452h47m40.56240969s) [upload=362068]: Got next segment. Inline: true Jul 12 05:58:40.679005554 (452h47m40.562419189s) [upload=362068]: Waiting for error group managing segments... Jul 12 05:58:40.679070474 (452h47m40.562484111s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:58:40.679078913 (452h47m40.562492552s) [upload=362068]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:58:40.679082385 (452h47m40.562495998s) [upload=362068]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:58:40.833826911 (452h47m40.717240533s) [upload=362068]: Done uploading segments... Jul 12 05:58:40.833854832 (452h47m40.717268447s) [upload=362068]: Upload finished. err: <nil> Jul 12 05:59:49.818630674 (452h48m49.702044293s) [upload=362084]: Starting upload Jul 12 05:59:49.818706003 (452h48m49.702119629s) [upload=362084]: Done starting upload Jul 12 05:59:49.818776130 (452h48m49.702189749s) [upload=362084]: Uploading segments... Jul 12 05:59:49.818817995 (452h48m49.702231631s) [upload=362084]: Got next segment. Inline: true Jul 12 05:59:49.818823484 (452h48m49.702237102s) [upload=362084]: Waiting for error group managing segments... Jul 12 05:59:49.818866953 (452h48m49.702280572s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 05:59:49.818872937 (452h48m49.702286559s) [upload=362084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 05:59:49.818876156 (452h48m49.702289774s) [upload=362084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 05:59:49.954529961 (452h48m49.837943576s) [upload=362084]: Done uploading segments... Jul 12 05:59:49.954554066 (452h48m49.837967679s) [upload=362084]: Upload finished. err: <nil> Jul 12 06:00:43.495779457 (452h49m43.379193083s) [upload=362094]: Starting upload Jul 12 06:00:43.495850292 (452h49m43.379263908s) [upload=362094]: Done starting upload Jul 12 06:00:43.495941322 (452h49m43.379354938s) [upload=362094]: Uploading segments... Jul 12 06:00:43.495995120 (452h49m43.379408751s) [upload=362094]: Got next segment. Inline: true Jul 12 06:00:43.496003010 (452h49m43.379416636s) [upload=362094]: Waiting for error group managing segments... Jul 12 06:00:43.496057483 (452h49m43.379471109s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:00:43.496063525 (452h49m43.379477152s) [upload=362094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:00:43.496067040 (452h49m43.379480666s) [upload=362094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:00:43.751058267 (452h49m43.634471902s) [upload=362094]: Done uploading segments... Jul 12 06:00:43.751086051 (452h49m43.634499678s) [upload=362094]: Upload finished. err: <nil> Jul 12 06:01:18.554885710 (452h50m18.438299339s) [upload=362102]: Starting upload Jul 12 06:01:18.554992486 (452h50m18.438406114s) [upload=362102]: Done starting upload Jul 12 06:01:18.555064908 (452h50m18.438478539s) [upload=362102]: Uploading segments... Jul 12 06:01:18.555161420 (452h50m18.438575056s) [upload=362102]: Got next segment. Inline: true Jul 12 06:01:18.555187002 (452h50m18.43860063s) [upload=362102]: Waiting for error group managing segments... Jul 12 06:01:18.555244054 (452h50m18.438657683s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:18.555267160 (452h50m18.43868079s) [upload=362102]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:18.555270932 (452h50m18.438684562s) [upload=362102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:18.943647504 (452h50m18.827061125s) [upload=362102]: Done uploading segments... Jul 12 06:01:18.943667489 (452h50m18.827081107s) [upload=362102]: Upload finished. err: <nil> Jul 12 06:01:35.111458191 (452h50m34.99487181s) [upload=362106]: Starting upload Jul 12 06:01:35.111534761 (452h50m34.994948387s) [upload=362106]: Done starting upload Jul 12 06:01:35.111619484 (452h50m34.995033103s) [upload=362106]: Uploading segments... Jul 12 06:01:35.111668242 (452h50m34.995081877s) [upload=362106]: Got next segment. Inline: true Jul 12 06:01:35.111674298 (452h50m34.995087925s) [upload=362106]: Waiting for error group managing segments... Jul 12 06:01:35.111731613 (452h50m34.995145231s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 12 06:01:35.111738659 (452h50m34.995152284s) [upload=362106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 12 06:01:35.111741973 (452h50m34.9951556s) [upload=362106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 12 06:01:35.268624606 (452h50m35.152038225s) [upload=362106]: Done uploading segments... Jul 12 06:01:35.268672244 (452h50m35.152085874s) [upload=362106]: Upload finished. err: <nil> Duration: 903.789122ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)