Orbiter History

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.....................XX..............XX.X...................] duration: 318.180915ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:52:22 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 318.209831ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 40 out of 60 is failed (threshold=3) [.....X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.............] duration: 2.690260613s FAIL (2025-08-07 21:52:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.690286106s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [..........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.............] duration: 1.052848972s FAIL (2025-08-07 21:52:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.052872587s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X......] duration: 2.940479279s FAIL (2025-08-07 21:52:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.940506102s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 2.771185924s FAIL (2025-08-07 21:52:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.771218698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.....] duration: 2.267474209s FAIL (2025-08-07 21:52:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.26750647s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [...........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X............] duration: 1.043355468s FAIL (2025-08-07 21:52:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.043380362s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [........................X............X.X.X..................] duration: 104.615892ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:51:58 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 104.684968ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXX....X.........] duration: 2.481834051s FAIL (2025-08-07 21:51:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.481862156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 38 out of 60 is failed (threshold=3) [..........XX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........] duration: 903.996418ms FAIL (2025-08-07 21:51:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 904.027272ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [......................XX..............XX.X..................] duration: 325.16441ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:51:48 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 325.212115ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 957.282783ms FAIL (2025-08-07 21:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 957.310409ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.......................XX.X..........X.....X......] duration: 3.296025491s FAIL (2025-08-07 21:51:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.931857584 (12h7m22.675914448s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.932258428 (12h7m22.676315289s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933368803 (12h7m22.677425673s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933961999 (12h7m22.678018845s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934101417 (12h7m22.67815828s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934371093 (12h7m22.678427954s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934756333 (12h7m22.678813196s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934888956 (12h7m22.678945807s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935299805 (12h7m22.679356668s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935539944 (12h7m22.679596796s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935895434 (12h7m22.679952289s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936414259 (12h7m22.680471112s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936836219 (12h7m22.680893083s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937535515 (12h7m22.68159243s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937642268 (12h7m22.681699132s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938008889 (12h7m22.682065754s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938565671 (12h7m22.682622538s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:01.515934007 (12h7m23.259990877s) [upload=12498]: Next returned nil segment Aug 7 21:47:01.515957554 (12h7m23.260014424s) [upload=12498]: Waiting for error group managing segments... Aug 7 21:47:01.689221575 (12h7m23.433278437s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.696245173 (12h7m23.440302028s) [noise=true , address=104.36.82.45:4931 , piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.698487716 (12h7m23.442544583s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.699902328 (12h7m23.443959181s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.704569768 (12h7m23.448626616s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.713265789 (12h7m23.457322661s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.733750309 (12h7m23.477807164s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.746947712 (12h7m23.491004568s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.755976189 (12h7m23.500033045s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.795757727 (12h7m23.539814619s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.901256341 (12h7m23.645313207s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.922821824 (12h7m23.666878693s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.950282380 (12h7m23.694339244s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145128156 (12h7m23.889185029s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145174967 (12h7m23.889231834s) [seg_pos={0 0}, upload=12498]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:02.145375024 (12h7m23.889431872s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MYs8DF38z84JzwWUYRL7rxPZWNxfyH4WnUdT1L58BiLkQsUTW): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.145801827 (12h7m23.889858706s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bpEYLHxV7TM3KdheTwmwBxpPvm3PKccMVFLfLc5fTXtJZX5VY): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146021528 (12h7m23.890078399s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CTzrehDEVX9VJdfpPHUwmixehj18F1hsPLhifikqU9B3Gt3Hy): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146292773 (12h7m23.890349638s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XPFXQaLxDzhZjcaACurTejzdA8S4szrXVJjc5pgJm9UMreNiy): protocol: read tcp 139.59.144.22:49992->104.36.82.61:4405: use of closed network connection; context canceled Aug 7 21:47:02.146329902 (12h7m23.890386769s) [seg_pos={0 0}, upload=12498]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:02.146430465 (12h7m23.890487825s) [upload=12498]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:02.146437380 (12h7m23.890494243s) [upload=12498]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:02.446739950 (12h7m24.190796819s) [upload=12498]: Done uploading segments... Aug 7 21:47:02.446788220 (12h7m24.190845097s) [upload=12498]: Upload finished. err: <nil> Aug 7 21:47:52.318171063 (12h8m14.06222791s) [upload=12512]: Starting upload Aug 7 21:47:52.318240558 (12h8m14.062297427s) [upload=12512]: Done starting upload Aug 7 21:47:52.318352095 (12h8m14.062408942s) [upload=12512]: Uploading segments... Aug 7 21:47:52.318602540 (12h8m14.062659412s) [upload=12512]: Got next segment. Inline: false Aug 7 21:47:52.318662425 (12h8m14.062719297s) [upload=12512]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:52.318669510 (12h8m14.062726381s) [upload=12512]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:52.406892213 (12h8m14.150949079s) [upload=12512]: Next segment err: metaclient: Too Many Requests Aug 7 21:47:52.406918034 (12h8m14.150974895s) [upload=12512]: Done uploading segments... Aug 7 21:47:52.406934956 (12h8m14.150991807s) [upload=12512]: Upload finished. err: metaclient: Too Many Requests Duration: 3.296064069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 2.949577863s FAIL (2025-08-07 21:51:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.949615253s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 40 out of 60 is failed (threshold=3) [........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX.........] duration: 924.167278ms FAIL (2025-08-07 21:51:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 924.23259ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXXXXXX....X........] duration: 2.483173918s FAIL (2025-08-07 21:51:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.483208069s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 40 out of 60 is failed (threshold=3) [......X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX............] duration: 2.707480067s FAIL (2025-08-07 21:51:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.707512367s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.......................XX.X..........X.....X.....] duration: 2.905794345s FAIL (2025-08-07 21:51:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.931857584 (12h7m22.675914448s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.932258428 (12h7m22.676315289s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933368803 (12h7m22.677425673s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933961999 (12h7m22.678018845s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934101417 (12h7m22.67815828s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934371093 (12h7m22.678427954s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934756333 (12h7m22.678813196s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934888956 (12h7m22.678945807s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935299805 (12h7m22.679356668s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935539944 (12h7m22.679596796s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935895434 (12h7m22.679952289s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936414259 (12h7m22.680471112s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936836219 (12h7m22.680893083s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937535515 (12h7m22.68159243s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937642268 (12h7m22.681699132s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938008889 (12h7m22.682065754s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938565671 (12h7m22.682622538s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:01.515934007 (12h7m23.259990877s) [upload=12498]: Next returned nil segment Aug 7 21:47:01.515957554 (12h7m23.260014424s) [upload=12498]: Waiting for error group managing segments... Aug 7 21:47:01.689221575 (12h7m23.433278437s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.696245173 (12h7m23.440302028s) [noise=true , address=104.36.82.45:4931 , piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.698487716 (12h7m23.442544583s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.699902328 (12h7m23.443959181s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.704569768 (12h7m23.448626616s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.713265789 (12h7m23.457322661s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.733750309 (12h7m23.477807164s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.746947712 (12h7m23.491004568s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.755976189 (12h7m23.500033045s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.795757727 (12h7m23.539814619s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.901256341 (12h7m23.645313207s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.922821824 (12h7m23.666878693s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.950282380 (12h7m23.694339244s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145128156 (12h7m23.889185029s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145174967 (12h7m23.889231834s) [seg_pos={0 0}, upload=12498]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:02.145375024 (12h7m23.889431872s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MYs8DF38z84JzwWUYRL7rxPZWNxfyH4WnUdT1L58BiLkQsUTW): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.145801827 (12h7m23.889858706s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bpEYLHxV7TM3KdheTwmwBxpPvm3PKccMVFLfLc5fTXtJZX5VY): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146021528 (12h7m23.890078399s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CTzrehDEVX9VJdfpPHUwmixehj18F1hsPLhifikqU9B3Gt3Hy): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146292773 (12h7m23.890349638s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XPFXQaLxDzhZjcaACurTejzdA8S4szrXVJjc5pgJm9UMreNiy): protocol: read tcp 139.59.144.22:49992->104.36.82.61:4405: use of closed network connection; context canceled Aug 7 21:47:02.146329902 (12h7m23.890386769s) [seg_pos={0 0}, upload=12498]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:02.146430465 (12h7m23.890487825s) [upload=12498]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:02.146437380 (12h7m23.890494243s) [upload=12498]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:02.446739950 (12h7m24.190796819s) [upload=12498]: Done uploading segments... Aug 7 21:47:02.446788220 (12h7m24.190845097s) [upload=12498]: Upload finished. err: <nil> Aug 7 21:47:52.318171063 (12h8m14.06222791s) [upload=12512]: Starting upload Aug 7 21:47:52.318240558 (12h8m14.062297427s) [upload=12512]: Done starting upload Aug 7 21:47:52.318352095 (12h8m14.062408942s) [upload=12512]: Uploading segments... Aug 7 21:47:52.318602540 (12h8m14.062659412s) [upload=12512]: Got next segment. Inline: false Aug 7 21:47:52.318662425 (12h8m14.062719297s) [upload=12512]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:52.318669510 (12h8m14.062726381s) [upload=12512]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:52.406892213 (12h8m14.150949079s) [upload=12512]: Next segment err: metaclient: Too Many Requests Aug 7 21:47:52.406918034 (12h8m14.150974895s) [upload=12512]: Done uploading segments... Aug 7 21:47:52.406934956 (12h8m14.150991807s) [upload=12512]: Upload finished. err: metaclient: Too Many Requests Duration: 2.905825815s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.........................X............X.X.X.................] duration: 121.588337ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:51:26 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 121.612563ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X.....] duration: 2.882515463s FAIL (2025-08-07 21:51:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.882540558s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 815.287681ms FAIL (2025-08-07 21:51:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 815.322502ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [....................X.XXXXXXXXXXXXXXXXXXXXXXXX......X.......] duration: 850.937588ms FAIL (2025-08-07 21:51:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 850.969712ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.......................XX..............XX.X.................] duration: 284.943481ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:51:18 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 284.972832ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX....] duration: 2.332780417s FAIL (2025-08-07 21:51:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.332812321s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...........] duration: 1.05753242s FAIL (2025-08-07 21:51:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.057571367s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXXX....] duration: 3.255639592s FAIL (2025-08-07 21:50:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 3.255666815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 38 out of 60 is failed (threshold=3) [...........XX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........] duration: 964.039432ms FAIL (2025-08-07 21:50:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 964.068908ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [........................XXXXXXXXXXXXXXXXXXXXXXXX....X.......] duration: 2.449071974s FAIL (2025-08-07 21:50:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.44910174s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X....] duration: 2.988968036s FAIL (2025-08-07 21:50:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.988998686s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 40 out of 60 is failed (threshold=3) [.......X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........] duration: 2.725697668s FAIL (2025-08-07 21:50:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.725726867s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [........................XX..............XX.X................] duration: 309.362621ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:50:49 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 309.386818ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 863.56348ms FAIL (2025-08-07 21:50:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 863.592964ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 40 out of 60 is failed (threshold=3) [.........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX........] duration: 1.186005604s FAIL (2025-08-07 21:50:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.18610998s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXXXXXXXXXXXXXXX....X......] duration: 2.553541214s FAIL (2025-08-07 21:50:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.553583532s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X.......................XX.X..........X.....X....] duration: 2.693360605s FAIL (2025-08-07 21:50:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.931857584 (12h7m22.675914448s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.932258428 (12h7m22.676315289s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933368803 (12h7m22.677425673s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933961999 (12h7m22.678018845s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934101417 (12h7m22.67815828s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934371093 (12h7m22.678427954s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934756333 (12h7m22.678813196s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934888956 (12h7m22.678945807s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935299805 (12h7m22.679356668s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935539944 (12h7m22.679596796s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935895434 (12h7m22.679952289s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936414259 (12h7m22.680471112s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936836219 (12h7m22.680893083s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937535515 (12h7m22.68159243s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937642268 (12h7m22.681699132s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938008889 (12h7m22.682065754s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938565671 (12h7m22.682622538s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:01.515934007 (12h7m23.259990877s) [upload=12498]: Next returned nil segment Aug 7 21:47:01.515957554 (12h7m23.260014424s) [upload=12498]: Waiting for error group managing segments... Aug 7 21:47:01.689221575 (12h7m23.433278437s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.696245173 (12h7m23.440302028s) [noise=true , address=104.36.82.45:4931 , piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.698487716 (12h7m23.442544583s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.699902328 (12h7m23.443959181s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.704569768 (12h7m23.448626616s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.713265789 (12h7m23.457322661s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.733750309 (12h7m23.477807164s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.746947712 (12h7m23.491004568s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.755976189 (12h7m23.500033045s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.795757727 (12h7m23.539814619s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.901256341 (12h7m23.645313207s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.922821824 (12h7m23.666878693s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.950282380 (12h7m23.694339244s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145128156 (12h7m23.889185029s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145174967 (12h7m23.889231834s) [seg_pos={0 0}, upload=12498]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:02.145375024 (12h7m23.889431872s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MYs8DF38z84JzwWUYRL7rxPZWNxfyH4WnUdT1L58BiLkQsUTW): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.145801827 (12h7m23.889858706s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bpEYLHxV7TM3KdheTwmwBxpPvm3PKccMVFLfLc5fTXtJZX5VY): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146021528 (12h7m23.890078399s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CTzrehDEVX9VJdfpPHUwmixehj18F1hsPLhifikqU9B3Gt3Hy): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146292773 (12h7m23.890349638s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XPFXQaLxDzhZjcaACurTejzdA8S4szrXVJjc5pgJm9UMreNiy): protocol: read tcp 139.59.144.22:49992->104.36.82.61:4405: use of closed network connection; context canceled Aug 7 21:47:02.146329902 (12h7m23.890386769s) [seg_pos={0 0}, upload=12498]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:02.146430465 (12h7m23.890487825s) [upload=12498]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:02.146437380 (12h7m23.890494243s) [upload=12498]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:02.446739950 (12h7m24.190796819s) [upload=12498]: Done uploading segments... Aug 7 21:47:02.446788220 (12h7m24.190845097s) [upload=12498]: Upload finished. err: <nil> Aug 7 21:47:52.318171063 (12h8m14.06222791s) [upload=12512]: Starting upload Aug 7 21:47:52.318240558 (12h8m14.062297427s) [upload=12512]: Done starting upload Aug 7 21:47:52.318352095 (12h8m14.062408942s) [upload=12512]: Uploading segments... Aug 7 21:47:52.318602540 (12h8m14.062659412s) [upload=12512]: Got next segment. Inline: false Aug 7 21:47:52.318662425 (12h8m14.062719297s) [upload=12512]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:52.318669510 (12h8m14.062726381s) [upload=12512]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:52.406892213 (12h8m14.150949079s) [upload=12512]: Next segment err: metaclient: Too Many Requests Aug 7 21:47:52.406918034 (12h8m14.150974895s) [upload=12512]: Done uploading segments... Aug 7 21:47:52.406934956 (12h8m14.150991807s) [upload=12512]: Upload finished. err: metaclient: Too Many Requests Duration: 2.693389065s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [..........................X............X.X.X................] duration: 104.487933ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:50:18 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 104.516069ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX...] duration: 2.499864792s FAIL (2025-08-07 21:50:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.49989708s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [.....................X.XXXXXXXXXXXXXXXXXXXXXXXX......X......] duration: 886.578765ms FAIL (2025-08-07 21:50:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 886.608013ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 40 out of 60 is failed (threshold=3) [..........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX.......] duration: 893.011338ms FAIL (2025-08-07 21:50:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 893.035002ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XXXX...] duration: 2.783913824s FAIL (2025-08-07 21:50:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.784012578s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 38 out of 60 is failed (threshold=3) [............XX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.........] duration: 884.043396ms FAIL (2025-08-07 21:50:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 884.252112ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X.......................XX.X..........X.....X...] duration: 3.261321617s FAIL (2025-08-07 21:50:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.931857584 (12h7m22.675914448s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.932258428 (12h7m22.676315289s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933368803 (12h7m22.677425673s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.933961999 (12h7m22.678018845s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934101417 (12h7m22.67815828s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934371093 (12h7m22.678427954s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934756333 (12h7m22.678813196s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.934888956 (12h7m22.678945807s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935299805 (12h7m22.679356668s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935539944 (12h7m22.679596796s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.935895434 (12h7m22.679952289s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936414259 (12h7m22.680471112s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.936836219 (12h7m22.680893083s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937535515 (12h7m22.68159243s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.937642268 (12h7m22.681699132s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938008889 (12h7m22.682065754s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:00.938565671 (12h7m22.682622538s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Uploading piece... Aug 7 21:47:01.515934007 (12h7m23.259990877s) [upload=12498]: Next returned nil segment Aug 7 21:47:01.515957554 (12h7m23.260014424s) [upload=12498]: Waiting for error group managing segments... Aug 7 21:47:01.689221575 (12h7m23.433278437s) [noise=true , address=104.36.82.58:4283 , piece_id=AEQQA4INRDBRLNAYRVDD4PSLOCBFM5HP4GOCWFOX47AHHJAEBWZA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.696245173 (12h7m23.440302028s) [noise=true , address=104.36.82.45:4931 , piece_id=RXXWJ22ZFGJ42YRFAX4HRE7E2VPGAKWKOEBWWTBBEKXPLRQFF54A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.698487716 (12h7m23.442544583s) [noise=true , address=104.36.82.23:4452 , piece_id=4XKVMVVLGCJRPKBC5HVJSX6M6UUI4QGFOCCKRHCMPNIOI3ULLWSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.699902328 (12h7m23.443959181s) [noise=true , address=104.36.82.55:4224 , piece_id=DVH3QKF4OWIMYY4EEMSNOIXW5CC54KN2M54CHUTU6YFC5MHRQFOA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.704569768 (12h7m23.448626616s) [noise=true , address=104.36.82.62:4431 , piece_id=CRGUBEWHDRPMKGY47AYWHGCX4D3RKCP5OW6UV6TD77DIIVNBVZBA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.713265789 (12h7m23.457322661s) [noise=true , address=104.36.82.39:4754 , piece_id=B64XNARGUDMJTQHTSEYX6LYDVYEPSFESYSHFJ2YATHOWE7V3JTSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.733750309 (12h7m23.477807164s) [noise=true , address=104.36.82.36:4640 , piece_id=KCDKD2VKLEHD6V52CQBLYI4KRNDDMLDGGYBQPLX27JBDAIV64XQQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.746947712 (12h7m23.491004568s) [noise=true , address=104.36.82.13:4214 , piece_id=4X2ZSNKGKTJ4OZWJVFDYTR2RBRPOJRSOOVTXSBUPCIO3K53U5HJQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.755976189 (12h7m23.500033045s) [noise=true , address=104.36.82.43:4853 , piece_id=WBEOH7537PO55GJLYGC4YHMTKULN2T6BCCYAEVBRR5X4AQ5F4XSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.795757727 (12h7m23.539814619s) [noise=true , address=104.36.82.9:4027 , piece_id=R3U7T2YCU7EKQULIM6KKZYBWQP2ZGNLHUUWQ7B4KJUJJIRAPZEPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.901256341 (12h7m23.645313207s) [noise=true , address=104.36.82.44:4877 , piece_id=EDM3CTKQYKOSOP3JU3UUWN63LVG7BA6FGOAFF2IRKRNRUPK7GTVQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.922821824 (12h7m23.666878693s) [noise=true , address=45.91.116.9:4288 , piece_id=ILYKFA6LDTNRBRGOVOR25H2VIGOUHQZQ3WPZDV3DZMZKH6ENKVNA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:01.950282380 (12h7m23.694339244s) [noise=true , address=104.36.82.37:4688 , piece_id=LBVMHMZJFVI5HD3P33LOA2L5KA7EUB64TQWVFX3PSYUUSLCCLAPA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145128156 (12h7m23.889185029s) [noise=true , address=104.36.82.35:4601 , piece_id=FIWLJE2E4X22266LQ44U7IBBZ4D7XLUGN6LRGENTOTC7H5WAIKMQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: <nil> Aug 7 21:47:02.145174967 (12h7m23.889231834s) [seg_pos={0 0}, upload=12498]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:02.145375024 (12h7m23.889431872s) [noise=true , address=79.127.205.233:4642 , piece_id=YGV6RGALGLVITSWGUWQFJVQNCPHXWS67G33QWZZWAPIFE6RYQQSA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MYs8DF38z84JzwWUYRL7rxPZWNxfyH4WnUdT1L58BiLkQsUTW): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.145801827 (12h7m23.889858706s) [noise=true , address=64.49.13.154:4926 , piece_id=X5CLC7JS7PCLNSPLHTDA7VC4BXKIAZON3A2EVZCBZ3NZYYZD42HA, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bpEYLHxV7TM3KdheTwmwBxpPvm3PKccMVFLfLc5fTXtJZX5VY): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146021528 (12h7m23.890078399s) [noise=true , address=64.49.13.169:4708 , piece_id=HTW7Y2VQQF5VDJPPLE265SZ7JU24T2KSEIUCSPEFQKZA2HNI2QWQ, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CTzrehDEVX9VJdfpPHUwmixehj18F1hsPLhifikqU9B3Gt3Hy): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:02.146292773 (12h7m23.890349638s) [noise=true , address=104.36.82.61:4405 , piece_id=BA25J7JHA2KUSFTJDRPBPBHFVIUPTD7FE7IKSTJUEU3IGBMIB72A, seg_pos={0 0}, upload=12498]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XPFXQaLxDzhZjcaACurTejzdA8S4szrXVJjc5pgJm9UMreNiy): protocol: read tcp 139.59.144.22:49992->104.36.82.61:4405: use of closed network connection; context canceled Aug 7 21:47:02.146329902 (12h7m23.890386769s) [seg_pos={0 0}, upload=12498]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:02.146430465 (12h7m23.890487825s) [upload=12498]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:02.146437380 (12h7m23.890494243s) [upload=12498]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:02.446739950 (12h7m24.190796819s) [upload=12498]: Done uploading segments... Aug 7 21:47:02.446788220 (12h7m24.190845097s) [upload=12498]: Upload finished. err: <nil> Aug 7 21:47:52.318171063 (12h8m14.06222791s) [upload=12512]: Starting upload Aug 7 21:47:52.318240558 (12h8m14.062297427s) [upload=12512]: Done starting upload Aug 7 21:47:52.318352095 (12h8m14.062408942s) [upload=12512]: Uploading segments... Aug 7 21:47:52.318602540 (12h8m14.062659412s) [upload=12512]: Got next segment. Inline: false Aug 7 21:47:52.318662425 (12h8m14.062719297s) [upload=12512]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:52.318669510 (12h8m14.062726381s) [upload=12512]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:52.406892213 (12h8m14.150949079s) [upload=12512]: Next segment err: metaclient: Too Many Requests Aug 7 21:47:52.406918034 (12h8m14.150974895s) [upload=12512]: Done uploading segments... Aug 7 21:47:52.406934956 (12h8m14.150991807s) [upload=12512]: Upload finished. err: metaclient: Too Many Requests Duration: 3.261391329s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 861.93113ms FAIL (2025-08-07 21:50:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 862.034208ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 40 out of 60 is failed (threshold=3) [........X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........] duration: 2.586321889s FAIL (2025-08-07 21:49:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.586346129s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X...] duration: 2.984038789s FAIL (2025-08-07 21:49:50 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.984066417s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [.............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..........] duration: 1.018844102s FAIL (2025-08-07 21:49:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.018880968s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX..] duration: 2.547166589s FAIL (2025-08-07 21:49:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.547271778s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X..] duration: 42.790720057s FAIL (2025-08-07 21:49:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 42.790782809s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.........................XX..............XX.X...............] duration: 298.176643ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:49:38 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 298.2036ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXX....X.....] duration: 2.445244055s FAIL (2025-08-07 21:49:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.445271464s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)