Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 3.439232853s FAIL (2025-07-06 22:55:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.439276769s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.649778647s FAIL (2025-07-06 22:54:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.649814699s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 3.47970515s FAIL (2025-07-06 22:54:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.479727205s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 3.442438396s FAIL (2025-07-06 22:53:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.442467608s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.785988886s FAIL (2025-07-06 22:53:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.786008056s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.602730366s FAIL (2025-07-06 22:53:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.602745957s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.911482269s FAIL (2025-07-06 22:52:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.911499481s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.716874887s FAIL (2025-07-06 22:51:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.716901236s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.630404346s FAIL (2025-07-06 22:51:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.630432052s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.878521743s FAIL (2025-07-06 22:50:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.878548834s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.787536398s FAIL (2025-07-06 22:49:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.78756368s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.896466584s FAIL (2025-07-06 22:49:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.896501635s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.868346124s FAIL (2025-07-06 22:49:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.868425513s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.100927036s FAIL (2025-07-06 22:48:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.101555242s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.70744385s FAIL (2025-07-06 22:47:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.707481734s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.349066937s FAIL (2025-07-06 22:46:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.349095438s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.223901957s FAIL (2025-07-06 22:45:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.223929713s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.849896561s FAIL (2025-07-06 22:44:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.849936925s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 4.99587093s FAIL (2025-07-06 22:43:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.995970331s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.897355922s FAIL (2025-07-06 22:42:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.897383648s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.679440274s FAIL (2025-07-06 22:41:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.67946719s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000585394s FAIL (2025-07-06 22:41:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00060978s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 867.324991ms FAIL (2025-07-06 20:45:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 867.364498ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 859.892569ms FAIL (2025-07-06 20:45:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 859.923779ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 849.729555ms FAIL (2025-07-06 20:45:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 849.765228ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 857.42976ms FAIL (2025-07-06 20:44:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 857.470697ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 900.829534ms FAIL (2025-07-06 20:43:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 901.062464ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 847.672419ms FAIL (2025-07-06 20:42:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 847.780177ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 900.428904ms FAIL (2025-07-06 20:41:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 900.460698ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 860.258757ms FAIL (2025-07-06 20:40:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 860.307121ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 941.135302ms FAIL (2025-07-06 20:39:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 941.16643ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 881.929899ms FAIL (2025-07-06 20:38:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 881.975706ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 846.95211ms FAIL (2025-07-06 20:38:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 846.990952ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 864.288197ms FAIL (2025-07-06 20:36:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 864.31736ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 960.371634ms FAIL (2025-07-06 20:36:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 960.474556ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 846.696277ms FAIL (2025-07-06 20:35:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 846.799627ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 863.296452ms FAIL (2025-07-06 20:34:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 863.33417ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 929.481658ms FAIL (2025-07-06 20:33:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 929.541763ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 877.941563ms FAIL (2025-07-06 20:33:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 877.990516ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 878.916559ms FAIL (2025-07-06 20:32:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 878.944948ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 953.870183ms FAIL (2025-07-06 20:32:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 953.984242ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 886.709629ms FAIL (2025-07-06 20:31:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 886.79849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.067784619s FAIL (2025-07-06 20:31:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 1.06781739s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 867.231171ms FAIL (2025-07-06 20:30:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 867.279256ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 859.400975ms FAIL (2025-07-06 20:29:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 859.438858ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 868.208005ms FAIL (2025-07-06 20:29:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 868.243054ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 955.419816ms FAIL (2025-07-06 20:28:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 955.469989ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 888.394065ms FAIL (2025-07-06 20:28:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 888.49346ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 889.989102ms FAIL (2025-07-06 20:27:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 890.022978ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 860.371744ms FAIL (2025-07-06 20:27:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 3]: Starting upload Jul 6 19:53:17.696182921 (322h42m17.57959654s) [upload=258043]: Done starting upload Jul 6 19:53:17.696284637 (322h42m17.579698255s) [upload=258043]: Uploading segments... Jul 6 19:53:17.696369389 (322h42m17.579783026s) [upload=258043]: Got next segment. Inline: true Jul 6 19:53:17.696377435 (322h42m17.579791049s) [upload=258043]: Waiting for error group managing segments... Jul 6 19:53:17.696466044 (322h42m17.579879688s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:53:17.696473147 (322h42m17.57988678s) [upload=258043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:53:17.696477145 (322h42m17.579890775s) [upload=258043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:53:17.970432335 (322h42m17.853845953s) [upload=258043]: Done uploading segments... Jul 6 19:53:17.970463037 (322h42m17.853876654s) [upload=258043]: Upload finished. err: <nil> Jul 6 19:54:28.117890598 (322h43m28.001304235s) [upload=258056]: Starting upload Jul 6 19:54:28.118040723 (322h43m28.001454351s) [upload=258056]: Done starting upload Jul 6 19:54:28.118137584 (322h43m28.001551215s) [upload=258056]: Uploading segments... Jul 6 19:54:28.118206157 (322h43m28.001619804s) [upload=258056]: Got next segment. Inline: true Jul 6 19:54:28.118213125 (322h43m28.001626752s) [upload=258056]: Waiting for error group managing segments... Jul 6 19:54:28.118322406 (322h43m28.001736037s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:54:28.118330930 (322h43m28.001744559s) [upload=258056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:54:28.118338639 (322h43m28.001752271s) [upload=258056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:54:28.261414387 (322h43m28.144828007s) [upload=258056]: Done uploading segments... Jul 6 19:54:28.261455875 (322h43m28.144869502s) [upload=258056]: Upload finished. err: <nil> Jul 6 19:55:33.095648966 (322h44m32.979062602s) [upload=258072]: Starting upload Jul 6 19:55:33.095770766 (322h44m32.979184378s) [upload=258072]: Done starting upload Jul 6 19:55:33.095872846 (322h44m32.979286476s) [upload=258072]: Uploading segments... Jul 6 19:55:33.095943124 (322h44m32.979356764s) [upload=258072]: Got next segment. Inline: true Jul 6 19:55:33.095950360 (322h44m32.979363996s) [upload=258072]: Waiting for error group managing segments... Jul 6 19:55:33.096028365 (322h44m32.979442003s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:55:33.096034532 (322h44m32.979448172s) [upload=258072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:55:33.096037940 (322h44m32.979451573s) [upload=258072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:55:33.243940836 (322h44m33.127354474s) [upload=258072]: Done uploading segments... Jul 6 19:55:33.243964591 (322h44m33.127378222s) [upload=258072]: Upload finished. err: <nil> Jul 6 19:56:06.282827178 (322h45m6.166240816s) [upload=258080]: Starting upload Jul 6 19:56:06.282915389 (322h45m6.166329024s) [upload=258080]: Done starting upload Jul 6 19:56:06.282997114 (322h45m6.16641074s) [upload=258080]: Uploading segments... Jul 6 19:56:06.283051835 (322h45m6.166465472s) [upload=258080]: Got next segment. Inline: true Jul 6 19:56:06.283058076 (322h45m6.166471714s) [upload=258080]: Waiting for error group managing segments... Jul 6 19:56:06.283128071 (322h45m6.166541699s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:56:06.283135188 (322h45m6.166548848s) [upload=258080]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:56:06.283139003 (322h45m6.166552639s) [upload=258080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:56:06.558416011 (322h45m6.441829649s) [upload=258080]: Done uploading segments... Jul 6 19:56:06.558456611 (322h45m6.441870248s) [upload=258080]: Upload finished. err: <nil> Jul 6 19:57:03.243636619 (322h46m3.127050256s) [upload=258090]: Starting upload Jul 6 19:57:03.243739469 (322h46m3.1271531s) [upload=258090]: Done starting upload Jul 6 19:57:03.243829805 (322h46m3.127243425s) [upload=258090]: Uploading segments... Jul 6 19:57:03.243899209 (322h46m3.127312846s) [upload=258090]: Got next segment. Inline: true Jul 6 19:57:03.243906118 (322h46m3.127319734s) [upload=258090]: Waiting for error group managing segments... Jul 6 19:57:03.243972280 (322h46m3.127385914s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:57:03.243978941 (322h46m3.127392571s) [upload=258090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:57:03.243984529 (322h46m3.127398156s) [upload=258090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:57:03.391652577 (322h46m3.275066197s) [upload=258090]: Done uploading segments... Jul 6 19:57:03.391672667 (322h46m3.275086288s) [upload=258090]: Upload finished. err: <nil> Jul 6 19:58:13.697516843 (322h47m13.580930472s) [upload=258104]: Starting upload Jul 6 19:58:13.697603707 (322h47m13.581021484s) [upload=258104]: Done starting upload Jul 6 19:58:13.697699228 (322h47m13.581112841s) [upload=258104]: Uploading segments... Jul 6 19:58:13.697752865 (322h47m13.5811665s) [upload=258104]: Got next segment. Inline: true Jul 6 19:58:13.697757885 (322h47m13.581171497s) [upload=258104]: Waiting for error group managing segments... Jul 6 19:58:13.697806149 (322h47m13.581219761s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:58:13.697811341 (322h47m13.581224953s) [upload=258104]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:58:13.697814082 (322h47m13.581227695s) [upload=258104]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:58:13.841091745 (322h47m13.724505364s) [upload=258104]: Done uploading segments... Jul 6 19:58:13.841124196 (322h47m13.724537815s) [upload=258104]: Upload finished. err: <nil> Jul 6 19:59:16.950570012 (322h48m16.833983639s) [upload=258120]: Starting upload Jul 6 19:59:16.950641780 (322h48m16.834055397s) [upload=258120]: Done starting upload Jul 6 19:59:16.950725338 (322h48m16.834138954s) [upload=258120]: Uploading segments... Jul 6 19:59:16.950764438 (322h48m16.834178073s) [upload=258120]: Got next segment. Inline: true Jul 6 19:59:16.950770321 (322h48m16.834183938s) [upload=258120]: Waiting for error group managing segments... Jul 6 19:59:16.950824426 (322h48m16.834238042s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 19:59:16.950830204 (322h48m16.834243831s) [upload=258120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 19:59:16.950833478 (322h48m16.834247097s) [upload=258120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 19:59:17.104935299 (322h48m16.988348934s) [upload=258120]: Done uploading segments... Jul 6 19:59:17.104971364 (322h48m16.988384977s) [upload=258120]: Upload finished. err: <nil> Jul 6 20:00:02.341189836 (322h49m2.224603454s) [upload=258131]: Starting upload Jul 6 20:00:02.341318462 (322h49m2.22473208s) [upload=258131]: Done starting upload Jul 6 20:00:02.341413922 (322h49m2.224827543s) [upload=258131]: Uploading segments... Jul 6 20:00:02.341477306 (322h49m2.224890936s) [upload=258131]: Got next segment. Inline: true Jul 6 20:00:02.341483239 (322h49m2.224896856s) [upload=258131]: Waiting for error group managing segments... Jul 6 20:00:02.341549241 (322h49m2.224962858s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:02.341556652 (322h49m2.224970271s) [upload=258131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:02.341560867 (322h49m2.224974485s) [upload=258131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:02.486864441 (322h49m2.370278063s) [upload=258131]: Done uploading segments... Jul 6 20:00:02.486899877 (322h49m2.370313501s) [upload=258131]: Upload finished. err: <nil> Jul 6 20:00:37.613195250 (322h49m37.496608882s) [upload=258141]: Starting upload Jul 6 20:00:37.613301342 (322h49m37.496714961s) [upload=258141]: Done starting upload Jul 6 20:00:37.613439708 (322h49m37.496853324s) [upload=258141]: Uploading segments... Jul 6 20:00:37.613500917 (322h49m37.496914546s) [upload=258141]: Got next segment. Inline: true Jul 6 20:00:37.613507575 (322h49m37.496921194s) [upload=258141]: Waiting for error group managing segments... Jul 6 20:00:37.613557129 (322h49m37.496970764s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:00:37.613563526 (322h49m37.496977156s) [upload=258141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:00:37.613580070 (322h49m37.496993701s) [upload=258141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:00:37.859063809 (322h49m37.742477461s) [upload=258141]: Done uploading segments... Jul 6 20:00:37.859116029 (322h49m37.742529671s) [upload=258141]: Upload finished. err: <nil> Jul 6 20:01:41.758461123 (322h50m41.641874742s) [upload=258153]: Starting upload Jul 6 20:01:41.758528267 (322h50m41.641941885s) [upload=258153]: Done starting upload Jul 6 20:01:41.758593360 (322h50m41.642006976s) [upload=258153]: Uploading segments... Jul 6 20:01:41.758636282 (322h50m41.642049918s) [upload=258153]: Got next segment. Inline: true Jul 6 20:01:41.758643400 (322h50m41.642057026s) [upload=258153]: Waiting for error group managing segments... Jul 6 20:01:41.758684317 (322h50m41.642097933s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 20:01:41.758689708 (322h50m41.642103327s) [upload=258153]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 6 20:01:41.758692964 (322h50m41.642106583s) [upload=258153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 20:01:41.911079543 (322h50m41.794493162s) [upload=258153]: Done uploading segments... Jul 6 20:01:41.911099828 (322h50m41.794513447s) [upload=258153]: Upload finished. err: <nil> Duration: 860.411039ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)