Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [..X.XXXXXXXXXXXXXXXXXXXXXXXX......X.........................] duration: 918.24306ms FAIL (2025-08-07 22:05:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 918.279765ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [......XX..............XX.X..................................] duration: 380.308049ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:05:29 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 380.331881ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XXXX......................] duration: 3.609718414s FAIL (2025-08-07 22:05:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 3.609761912s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............XX.X..........X.....X.........X..X...........X] duration: 2.719505062s FAIL (2025-08-07 22:05:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 5.727759364s) [upload=12820]: Starting upload Aug 7 22:05:23.983754833 (12h25m45.727811679s) [upload=12820]: Done starting upload Aug 7 22:05:23.983988533 (12h25m45.728045379s) [upload=12820]: Uploading segments... Aug 7 22:05:23.984022348 (12h25m45.728079214s) [upload=12820]: Got next segment. Inline: false Aug 7 22:05:23.984055700 (12h25m45.728112548s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:23.984060538 (12h25m45.728117398s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:05:24.185392691 (12h25m45.929449561s) [seg_pos={0 0}, upload=12820]: Begin upload segment... Aug 7 22:05:24.185557956 (12h25m45.929614817s) [seg_pos={0 0}, upload=12820]: Done begin upload segment. Aug 7 22:05:24.185603301 (12h25m45.929660177s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186037040 (12h25m45.930093906s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186570414 (12h25m45.930627287s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186824559 (12h25m45.930881425s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187208122 (12h25m45.931264991s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187658761 (12h25m45.931715627s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187940002 (12h25m45.931996872s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188116203 (12h25m45.932173069s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188531115 (12h25m45.932587976s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188637273 (12h25m45.932694145s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188961006 (12h25m45.933017872s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189183712 (12h25m45.933240578s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189352560 (12h25m45.933409422s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189640677 (12h25m45.93369755s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189855650 (12h25m45.933912517s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190187162 (12h25m45.934244033s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190334028 (12h25m45.934390898s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190661949 (12h25m45.934718816s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.754845434 (12h25m46.498902284s) [upload=12820]: Next returned nil segment Aug 7 22:05:24.754862619 (12h25m46.49891948s) [upload=12820]: Waiting for error group managing segments... Aug 7 22:05:24.937656798 (12h25m46.681713649s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.980443483 (12h25m46.724500333s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.995491860 (12h25m46.739551452s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.083772095 (12h25m46.827828964s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.147179204 (12h25m46.891236066s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.274393636 (12h25m47.018450492s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.452784464 (12h25m47.196841314s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.474459740 (12h25m47.218516586s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.481247102 (12h25m47.225303949s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666066275 (12h25m47.410123138s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666429351 (12h25m47.410486202s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.782101005 (12h25m47.526157853s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.863469768 (12h25m47.607526622s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865162168 (12h25m47.609219032s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865201343 (12h25m47.609258205s) [seg_pos={0 0}, upload=12820]: Segment reached optimal threshold of 14 pieces. Aug 7 22:05:25.865362755 (12h25m47.609419601s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865412734 (12h25m47.609469599s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nhwT2THUu6ptumqU9Bz1Nd34FHLewx9vWjGFC34j3Lz3RWfkw): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865468394 (12h25m47.609525241s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865629517 (12h25m47.609686385s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v): protocol: write tcp 139.59.144.22:45386->104.36.82.22:4530: use of closed network connection; context canceled Aug 7 22:05:25.865657252 (12h25m47.609714121s) [seg_pos={0 0}, upload=12820]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:05:25.865814825 (12h25m47.609871688s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:05:25.865822925 (12h25m47.609879795s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:26.138690358 (12h25m47.882747231s) [upload=12820]: Done uploading segments... Aug 7 22:05:26.138713087 (12h25m47.882769948s) [upload=12820]: Upload finished. err: <nil> Duration: 2.719556367s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..............................] duration: 861.987324ms FAIL (2025-08-07 22:05:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 862.019994ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [..........XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.....................] duration: 2.239365395s FAIL (2025-08-07 22:05:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.239413901s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...XXXXXXXXXXXXXXXXXXXXXXXX....X............................] duration: 2.802093609s FAIL (2025-08-07 22:05:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 2.802139468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............XX.X..........X.....X.........X..X...........] duration: 3.489804996s FAIL (2025-08-07 22:05:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 2679]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:41.477492343 (12h18m3.221549204s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:57:41.672062399 (12h18m3.416119246s) [seg_pos={0 0}, upload=12679]: Begin upload segment... Aug 7 21:57:41.672109341 (12h18m3.416166188s) [seg_pos={0 0}, upload=12679]: Done begin upload segment. Aug 7 21:57:41.672137062 (12h18m3.416193926s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672169185 (12h18m3.416226053s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672462219 (12h18m3.416519083s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672724395 (12h18m3.416781259s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672734877 (12h18m3.416791747s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672974216 (12h18m3.41703108s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673211496 (12h18m3.41726836s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673247825 (12h18m3.417304692s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673482119 (12h18m3.417538982s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673729154 (12h18m3.417786013s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675300194 (12h18m3.419357058s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675937494 (12h18m3.419994342s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676204488 (12h18m3.420261334s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676430906 (12h18m3.420487772s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676657651 (12h18m3.420714511s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676895247 (12h18m3.420952107s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677161207 (12h18m3.421218069s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677432270 (12h18m3.421489131s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:42.251495665 (12h18m3.995552516s) [upload=12679]: Next returned nil segment Aug 7 21:57:42.251513937 (12h18m3.995570787s) [upload=12679]: Waiting for error group managing segments... Aug 7 21:57:42.431114374 (12h18m4.175171221s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.434754441 (12h18m4.178811288s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457422325 (12h18m4.201479172s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457520666 (12h18m4.201577512s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.470582115 (12h18m4.214638961s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.474337979 (12h18m4.218394829s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.546312273 (12h18m4.290369142s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.554650633 (12h18m4.298707502s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.561724122 (12h18m4.305780975s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.575632750 (12h18m4.319689621s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.600836061 (12h18m4.344892927s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.852181705 (12h18m4.596238565s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.923085668 (12h18m4.667142529s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924189057 (12h18m4.668245921s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924227116 (12h18m4.668283979s) [seg_pos={0 0}, upload=12679]: Segment reached optimal threshold of 14 pieces. Aug 7 21:57:42.924427387 (12h18m4.668484248s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924565879 (12h18m4.668622735s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ektWku5yW6MRm2Rph8kiAAuifJJWZUMopaA3gcwStUfY6Nax8): protocol: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection Aug 7 21:57:42.924659012 (12h18m4.668715873s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924686911 (12h18m4.668743763s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fB7rqiDhMNQ4j9smkctbKdnGCXXdfFPWTgTxRrwQVZ9fKEFeZ): protocol: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection Aug 7 21:57:42.924713955 (12h18m4.668770815s) [seg_pos={0 0}, upload=12679]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:57:42.924812314 (12h18m4.668869174s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:57:42.924819863 (12h18m4.668876723s) [upload=12679]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:43.111275435 (12h18m4.855332295s) [upload=12679]: Done uploading segments... Aug 7 21:57:43.111294851 (12h18m4.855351702s) [upload=12679]: Upload finished. err: <nil> Duration: 3.489849466s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XXXX.....................] duration: 3.564590419s FAIL (2025-08-07 22:04:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 3.564632388s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [...........XXXXXXXXXXXXXXXXXXXXXXXXXXXXX....................] duration: 2.424388976s FAIL (2025-08-07 22:04:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.424430502s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.............................] duration: 884.045812ms FAIL (2025-08-07 22:04:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 884.073051ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX............................] duration: 1.068899458s FAIL (2025-08-07 22:04:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.068932534s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.............................] duration: 2.823392092s FAIL (2025-08-07 22:04:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.823420275s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [....XXXXXXXXXXXXXXXXXXXXXXXX....X...........................] duration: 3.265874528s FAIL (2025-08-07 22:04:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 3.265905468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X......................................X...X....] duration: 898.601832ms FAIL (2025-08-07 22:04:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 898.650047ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [......X............X.X.X....................................] duration: 201.68353ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:04:39 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 201.708779ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [....XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X.....................] duration: 3.040149078s FAIL (2025-08-07 22:04:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.040183958s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.......XX..............XX.X.................................] duration: 311.545034ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:04:32 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 311.571084ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..............................] duration: 1.034912936s FAIL (2025-08-07 22:04:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.034954946s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X......................................X...X...] duration: 828.743543ms FAIL (2025-08-07 22:04:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 828.795849ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [...X.XXXXXXXXXXXXXXXXXXXXXXXX......X........................] duration: 1.105650384s FAIL (2025-08-07 22:04:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.105695647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XXXX....................] duration: 4.311307893s FAIL (2025-08-07 22:04:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 4.311339289s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.....XXXXXXXXXXXXXXXXXXXXXXXX....X..........................] duration: 4.401914672s FAIL (2025-08-07 22:04:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 4.401949141s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [........XX..............XX.X................................] duration: 302.638052ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:04:02 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 302.676104ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX............................] duration: 2.64502409s FAIL (2025-08-07 22:04:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.645119395s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [................XX.X..........X.....X.........X..X..........] duration: 3.981803888s FAIL (2025-08-07 22:03:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 2679]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:41.477492343 (12h18m3.221549204s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:57:41.672062399 (12h18m3.416119246s) [seg_pos={0 0}, upload=12679]: Begin upload segment... Aug 7 21:57:41.672109341 (12h18m3.416166188s) [seg_pos={0 0}, upload=12679]: Done begin upload segment. Aug 7 21:57:41.672137062 (12h18m3.416193926s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672169185 (12h18m3.416226053s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672462219 (12h18m3.416519083s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672724395 (12h18m3.416781259s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672734877 (12h18m3.416791747s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672974216 (12h18m3.41703108s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673211496 (12h18m3.41726836s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673247825 (12h18m3.417304692s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673482119 (12h18m3.417538982s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673729154 (12h18m3.417786013s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675300194 (12h18m3.419357058s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675937494 (12h18m3.419994342s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676204488 (12h18m3.420261334s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676430906 (12h18m3.420487772s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676657651 (12h18m3.420714511s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676895247 (12h18m3.420952107s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677161207 (12h18m3.421218069s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677432270 (12h18m3.421489131s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:42.251495665 (12h18m3.995552516s) [upload=12679]: Next returned nil segment Aug 7 21:57:42.251513937 (12h18m3.995570787s) [upload=12679]: Waiting for error group managing segments... Aug 7 21:57:42.431114374 (12h18m4.175171221s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.434754441 (12h18m4.178811288s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457422325 (12h18m4.201479172s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457520666 (12h18m4.201577512s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.470582115 (12h18m4.214638961s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.474337979 (12h18m4.218394829s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.546312273 (12h18m4.290369142s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.554650633 (12h18m4.298707502s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.561724122 (12h18m4.305780975s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.575632750 (12h18m4.319689621s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.600836061 (12h18m4.344892927s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.852181705 (12h18m4.596238565s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.923085668 (12h18m4.667142529s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924189057 (12h18m4.668245921s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924227116 (12h18m4.668283979s) [seg_pos={0 0}, upload=12679]: Segment reached optimal threshold of 14 pieces. Aug 7 21:57:42.924427387 (12h18m4.668484248s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924565879 (12h18m4.668622735s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ektWku5yW6MRm2Rph8kiAAuifJJWZUMopaA3gcwStUfY6Nax8): protocol: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection Aug 7 21:57:42.924659012 (12h18m4.668715873s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924686911 (12h18m4.668743763s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fB7rqiDhMNQ4j9smkctbKdnGCXXdfFPWTgTxRrwQVZ9fKEFeZ): protocol: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection Aug 7 21:57:42.924713955 (12h18m4.668770815s) [seg_pos={0 0}, upload=12679]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:57:42.924812314 (12h18m4.668869174s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:57:42.924819863 (12h18m4.668876723s) [upload=12679]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:43.111275435 (12h18m4.855332295s) [upload=12679]: Done uploading segments... Aug 7 21:57:43.111294851 (12h18m4.855351702s) [upload=12679]: Upload finished. err: <nil> Duration: 3.981838784s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [....X.XXXXXXXXXXXXXXXXXXXXXXXX......X.......................] duration: 924.310862ms FAIL (2025-08-07 22:03:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 924.34323ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [............XXXXXXXXXXXXXXXXXXXXXXXXXXXXX...................] duration: 2.198133972s FAIL (2025-08-07 22:03:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.19816515s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X......................................X...X..] duration: 872.736281ms FAIL (2025-08-07 22:03:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 872.803849ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX............................] duration: 950.611354ms FAIL (2025-08-07 22:03:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 950.64411ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX...........................] duration: 889.783912ms FAIL (2025-08-07 22:03:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 889.835592ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........................] duration: 2.933522199s FAIL (2025-08-07 22:03:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.93354626s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.............................] duration: 1.111224843s FAIL (2025-08-07 22:03:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.111258859s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.......X............X.X.X...................................] duration: 106.998005ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:03:28 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 107.087442ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.................XX.X..........X.....X.........X..X.........] duration: 4.195171913s FAIL (2025-08-07 22:03:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 2679]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:41.477492343 (12h18m3.221549204s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:57:41.672062399 (12h18m3.416119246s) [seg_pos={0 0}, upload=12679]: Begin upload segment... Aug 7 21:57:41.672109341 (12h18m3.416166188s) [seg_pos={0 0}, upload=12679]: Done begin upload segment. Aug 7 21:57:41.672137062 (12h18m3.416193926s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672169185 (12h18m3.416226053s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672462219 (12h18m3.416519083s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672724395 (12h18m3.416781259s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672734877 (12h18m3.416791747s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672974216 (12h18m3.41703108s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673211496 (12h18m3.41726836s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673247825 (12h18m3.417304692s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673482119 (12h18m3.417538982s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673729154 (12h18m3.417786013s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675300194 (12h18m3.419357058s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675937494 (12h18m3.419994342s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676204488 (12h18m3.420261334s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676430906 (12h18m3.420487772s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676657651 (12h18m3.420714511s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676895247 (12h18m3.420952107s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677161207 (12h18m3.421218069s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677432270 (12h18m3.421489131s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:42.251495665 (12h18m3.995552516s) [upload=12679]: Next returned nil segment Aug 7 21:57:42.251513937 (12h18m3.995570787s) [upload=12679]: Waiting for error group managing segments... Aug 7 21:57:42.431114374 (12h18m4.175171221s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.434754441 (12h18m4.178811288s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457422325 (12h18m4.201479172s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457520666 (12h18m4.201577512s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.470582115 (12h18m4.214638961s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.474337979 (12h18m4.218394829s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.546312273 (12h18m4.290369142s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.554650633 (12h18m4.298707502s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.561724122 (12h18m4.305780975s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.575632750 (12h18m4.319689621s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.600836061 (12h18m4.344892927s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.852181705 (12h18m4.596238565s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.923085668 (12h18m4.667142529s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924189057 (12h18m4.668245921s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924227116 (12h18m4.668283979s) [seg_pos={0 0}, upload=12679]: Segment reached optimal threshold of 14 pieces. Aug 7 21:57:42.924427387 (12h18m4.668484248s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924565879 (12h18m4.668622735s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ektWku5yW6MRm2Rph8kiAAuifJJWZUMopaA3gcwStUfY6Nax8): protocol: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection Aug 7 21:57:42.924659012 (12h18m4.668715873s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924686911 (12h18m4.668743763s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fB7rqiDhMNQ4j9smkctbKdnGCXXdfFPWTgTxRrwQVZ9fKEFeZ): protocol: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection Aug 7 21:57:42.924713955 (12h18m4.668770815s) [seg_pos={0 0}, upload=12679]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:57:42.924812314 (12h18m4.668869174s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:57:42.924819863 (12h18m4.668876723s) [upload=12679]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:43.111275435 (12h18m4.855332295s) [upload=12679]: Done uploading segments... Aug 7 21:57:43.111294851 (12h18m4.855351702s) [upload=12679]: Upload finished. err: <nil> Duration: 4.195214243s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [.....XXXXXXXXXXXXXXXXXXXXXXXXXX......X.X....................] duration: 3.007370085s FAIL (2025-08-07 22:03:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.00741099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [......XXXXXXXXXXXXXXXXXXXXXXXX....X.........................] duration: 3.670400205s FAIL (2025-08-07 22:03:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 3.67043056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 26 out of 60 is failed (threshold=3) [.....X.XXXXXXXXXXXXXXXXXXXXXXXX......X......................] duration: 892.877834ms FAIL (2025-08-07 22:03:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 202662 (11h58m6.212259509s) [upload=12350]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:44.877004541 (11h58m6.621061411s) [upload=12363]: Starting upload Aug 7 21:37:44.877093129 (11h58m6.621150048s) [upload=12363]: Done starting upload Aug 7 21:37:44.877179828 (11h58m6.621236677s) [upload=12363]: Uploading segments... Aug 7 21:37:44.877253719 (11h58m6.621310589s) [upload=12363]: Got next segment. Inline: true Aug 7 21:37:44.877262492 (11h58m6.621319365s) [upload=12363]: Waiting for error group managing segments... Aug 7 21:37:44.877313668 (11h58m6.621370528s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:44.877319037 (11h58m6.621375897s) [upload=12363]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:44.877322512 (11h58m6.621379375s) [upload=12363]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:44.603781017 (11h59m6.347837867s) [upload=12363]: Done uploading segments... Aug 7 21:38:44.603802220 (11h59m6.347859069s) [upload=12363]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:45.032933533 (11h59m6.776990379s) [upload=12375]: Starting upload Aug 7 21:38:45.033010670 (11h59m6.777067533s) [upload=12375]: Done starting upload Aug 7 21:38:45.033084965 (11h59m6.777141811s) [upload=12375]: Uploading segments... Aug 7 21:38:45.033156998 (11h59m6.777213868s) [upload=12375]: Got next segment. Inline: true Aug 7 21:38:45.033162279 (11h59m6.777219127s) [upload=12375]: Waiting for error group managing segments... Aug 7 21:38:45.033211860 (11h59m6.777268707s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:45.033216660 (11h59m6.77727352s) [upload=12375]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:45.033219329 (11h59m6.777276175s) [upload=12375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:44.758881739 (12h0m6.502938587s) [upload=12375]: Done uploading segments... Aug 7 21:39:44.758902501 (12h0m6.502959346s) [upload=12375]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:45.185696259 (12h0m6.929753109s) [upload=12388]: Starting upload Aug 7 21:39:45.185761209 (12h0m6.929818072s) [upload=12388]: Done starting upload Aug 7 21:39:45.185834033 (12h0m6.929890896s) [upload=12388]: Uploading segments... Aug 7 21:39:45.185982126 (12h0m6.930038997s) [upload=12388]: Got next segment. Inline: true Aug 7 21:39:45.185990859 (12h0m6.930047726s) [upload=12388]: Waiting for error group managing segments... Aug 7 21:39:45.186067966 (12h0m6.930124827s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:45.186074972 (12h0m6.930131844s) [upload=12388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:45.186078706 (12h0m6.930135567s) [upload=12388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:44.901948337 (12h1m6.646005185s) [upload=12388]: Done uploading segments... Aug 7 21:40:44.901969954 (12h1m6.646026803s) [upload=12388]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:53.544245995 (12h1m15.288302845s) [upload=12404]: Starting upload Aug 7 21:40:53.544357406 (12h1m15.288414275s) [upload=12404]: Done starting upload Aug 7 21:40:53.544447469 (12h1m15.288504323s) [upload=12404]: Uploading segments... Aug 7 21:40:53.544512878 (12h1m15.28856975s) [upload=12404]: Got next segment. Inline: true Aug 7 21:40:53.544519444 (12h1m15.288576314s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:40:53.544579423 (12h1m15.288636288s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:53.544585620 (12h1m15.288642482s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:53.544589678 (12h1m15.288646542s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:05.856628151 (12h1m27.600685011s) [upload=12404]: Done uploading segments... Aug 7 21:41:05.856646351 (12h1m27.6007032s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:41:43.946896616 (12h2m5.690953463s) [upload=12420]: Starting upload Aug 7 21:41:43.946973751 (12h2m5.69103062s) [upload=12420]: Done starting upload Aug 7 21:41:43.947068595 (12h2m5.691125457s) [upload=12420]: Uploading segments... Aug 7 21:41:43.947136055 (12h2m5.691192916s) [upload=12420]: Got next segment. Inline: true Aug 7 21:41:43.947141676 (12h2m5.691198537s) [upload=12420]: Waiting for error group managing segments... Aug 7 21:41:43.947184608 (12h2m5.691241458s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:43.947190015 (12h2m5.691246865s) [upload=12420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:43.947193244 (12h2m5.691250106s) [upload=12420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:31.804258579 (12h2m53.548315429s) [upload=12420]: Done uploading segments... Aug 7 21:42:31.804277273 (12h2m53.548334124s) [upload=12420]: Upload finished. err: <nil> Aug 7 21:42:52.266861509 (12h3m14.01091837s) [upload=12435]: Starting upload Aug 7 21:42:52.266938245 (12h3m14.010995105s) [upload=12435]: Done starting upload Aug 7 21:42:52.267013941 (12h3m14.011070791s) [upload=12435]: Uploading segments... Aug 7 21:42:52.267054220 (12h3m14.01111109s) [upload=12435]: Got next segment. Inline: true Aug 7 21:42:52.267059450 (12h3m14.011116311s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:42:52.267099382 (12h3m14.011156243s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:52.267105199 (12h3m14.011162059s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:52.267108901 (12h3m14.011165762s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:01.858360925 (12h3m23.602417792s) [upload=12435]: Done uploading segments... Aug 7 21:43:01.858412136 (12h3m23.602468987s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:43:32.197096129 (12h3m53.941152979s) [upload=12448]: Starting upload Aug 7 21:43:32.197186644 (12h3m53.941243495s) [upload=12448]: Done starting upload Aug 7 21:43:32.197270859 (12h3m53.941327709s) [upload=12448]: Uploading segments... Aug 7 21:43:32.197324582 (12h3m53.941381451s) [upload=12448]: Got next segment. Inline: true Aug 7 21:43:32.197329997 (12h3m53.941386847s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:43:32.197385019 (12h3m53.941441879s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:32.197390582 (12h3m53.941447443s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:32.197394073 (12h3m53.941450934s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:32.363120575 (12h3m54.107177435s) [upload=12448]: Done uploading segments... Aug 7 21:43:32.363140640 (12h3m54.107197486s) [upload=12448]: Upload finished. err: <nil> Aug 7 21:44:28.281971344 (12h4m50.026028222s) [upload=12459]: Starting upload Aug 7 21:44:28.282137259 (12h4m50.026194124s) [upload=12459]: Done starting upload Aug 7 21:44:28.282276848 (12h4m50.026333712s) [upload=12459]: Uploading segments... Aug 7 21:44:28.282367088 (12h4m50.026423964s) [upload=12459]: Got next segment. Inline: true Aug 7 21:44:28.282375073 (12h4m50.026431948s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:44:28.282442525 (12h4m50.026499391s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:28.282449059 (12h4m50.026505928s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:28.282452578 (12h4m50.026509443s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:35.643697666 (12h4m57.387754517s) [upload=12459]: Done uploading segments... Aug 7 21:44:35.643716871 (12h4m57.387773725s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:45:33.044130503 (12h5m54.788187353s) [upload=12474]: Starting upload Aug 7 21:45:33.044219677 (12h5m54.788276538s) [upload=12474]: Done starting upload Aug 7 21:45:33.044292138 (12h5m54.788348988s) [upload=12474]: Uploading segments... Aug 7 21:45:33.044343424 (12h5m54.788400293s) [upload=12474]: Got next segment. Inline: true Aug 7 21:45:33.044349210 (12h5m54.78840607s) [upload=12474]: Waiting for error group managing segments... Aug 7 21:45:33.044391490 (12h5m54.78844834s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:33.044407486 (12h5m54.788464346s) [upload=12474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:33.044413529 (12h5m54.788470389s) [upload=12474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:33.198940222 (12h5m54.942997072s) [upload=12474]: Done uploading segments... Aug 7 21:45:33.198960825 (12h5m54.943017674s) [upload=12474]: Upload finished. err: <nil> Aug 7 21:45:57.787118330 (12h6m19.53117519s) [upload=12482]: Starting upload Aug 7 21:45:57.787191262 (12h6m19.531248122s) [upload=12482]: Done starting upload Aug 7 21:45:57.787268709 (12h6m19.53132556s) [upload=12482]: Uploading segments... Aug 7 21:45:57.787331847 (12h6m19.531388712s) [upload=12482]: Got next segment. Inline: true Aug 7 21:45:57.787339509 (12h6m19.53139637s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:45:57.787386486 (12h6m19.531443346s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:57.787403824 (12h6m19.531460685s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:57.787407765 (12h6m19.531464626s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:57.502016096 (12h7m19.246072946s) [upload=12482]: Done uploading segments... Aug 7 21:46:57.502068445 (12h7m19.246125296s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Duration: 892.910156ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 4.320284796s FAIL (2025-08-07 22:03:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:24:45.813454815 (11h45m7.557511676s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:43.107142097 (11h46m4.851198968s) [seg_pos={0 0}, upload=12171]: Begin upload segment... Aug 7 21:25:43.107205391 (11h46m4.851262241s) [seg_pos={0 0}, upload=12171]: Done begin upload segment. Aug 7 21:25:43.107237087 (11h46m4.851293957s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107267905 (11h46m4.851324768s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.107830175 (11h46m4.851887046s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108780038 (11h46m4.852836904s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.108959605 (11h46m4.853016472s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109220359 (11h46m4.853277218s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109377214 (11h46m4.853434079s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109626289 (11h46m4.853683159s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.109743295 (11h46m4.853800166s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110014430 (11h46m4.854071298s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110156657 (11h46m4.854213527s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110413863 (11h46m4.854470734s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.110778464 (11h46m4.854835327s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111117136 (11h46m4.855173999s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111560812 (11h46m4.855617682s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.111976240 (11h46m4.856033107s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112141257 (11h46m4.856198127s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.112560277 (11h46m4.856617148s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Uploading piece... Aug 7 21:25:43.675139110 (11h46m5.419195972s) [upload=12171]: Next returned nil segment Aug 7 21:25:43.675161049 (11h46m5.419217914s) [upload=12171]: Waiting for error group managing segments... Aug 7 21:25:43.872519019 (11h46m5.616575886s) [noise=true , address=104.36.82.62:4412 , piece_id=6NAJYYYU3AX7KJNFBXTPEYRCFFTMV3ESLVFUWTEGKGNOEHFHJJKA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.876126423 (11h46m5.62018327s) [noise=true , address=104.36.82.51:4073 , piece_id=HPC5WAIPHCOMTZDKPDPGZ7UJ43KPZZ2IFL5VFH7ILXHCF63D5NXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.882439205 (11h46m5.626496076s) [noise=true , address=104.36.82.35:4602 , piece_id=42ZEPU7YI52FBKWCUG2ZRDLPOLG5IYXSPR4HJ2QUSURO65WIAICQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.886290836 (11h46m5.63034769s) [noise=true , address=104.36.82.22:4511 , piece_id=WWW6R3JFLEQCGIAIZU6VXOC6XCPXFC3AWJ74PBL4PQXKTTSF535A, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888219892 (11h46m5.632276738s) [noise=true , address=104.36.82.15:4269 , piece_id=HTXIUN5HOU4B66MPMEXJJSUL6JNBE6FI6EB62IEG7P3WTZLAH47Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.888421317 (11h46m5.632478185s) [noise=true , address=45.91.116.14:4197 , piece_id=52LI7AMEDE2OBW6TAIOP5Y2M4J4HAWEITEKIYPZZ7XQ2Z5CGYIBQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.889366258 (11h46m5.633423104s) [noise=true , address=104.36.82.35:4607 , piece_id=DQKF7V3EW6OT5C4IFITEQNOXUUHDOMG5AESWXUYJP3SERMWUF5VQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.890288566 (11h46m5.634345437s) [noise=true , address=104.36.82.43:4852 , piece_id=NBGNTQUWEOVY72D3XFFO74EFUL52BZP222Z6FNIYCFB43FOBIW2Q, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.893016005 (11h46m5.637072875s) [noise=true , address=45.91.116.8:4056 , piece_id=DOFJMEXQQBOTVMCWC77YLLDCUY4XYYRM3X5BOBSXXAPB7T4YVHZQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.903527905 (11h46m5.647584771s) [noise=true , address=104.36.82.38:4708 , piece_id=JUSE5AFFHKPHCDVDRN5WWWXQK6VHIY3CXQWMJ4EXEJXVMICMZIXA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.912802006 (11h46m5.656858856s) [noise=true , address=45.91.116.7:4013 , piece_id=JEWYLXASDWIZ6G3LSHWMUAFAZIKLWYO7KOZLEQEGNZB3HMNVSKYA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:43.974200245 (11h46m5.718257092s) [noise=true , address=104.36.82.8:4983 , piece_id=ZDR4NMXM2MJTWTLT5776GQIUXCIY7TMRNBQ43L73XOFFBLSA6KQQ, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.128893332 (11h46m5.872950195s) [noise=true , address=79.127.201.214:4296 , piece_id=SK5JKFXZVF5XRW6STC5KDH5WNI7JHB6X5X5MLZ6WLLHG5SO6APSA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143902251 (11h46m5.887959098s) [noise=true , address=104.36.82.37:4662 , piece_id=KZG6JYEDQUTQGQCTUAZK7R3JRD7GKFLKORVEC5LIZ47GVJ6MFZRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: <nil> Aug 7 21:25:44.143991645 (11h46m5.888048495s) [seg_pos={0 0}, upload=12171]: Segment reached optimal threshold of 14 pieces. Aug 7 21:25:44.144122598 (11h46m5.888179458s) [noise=true , address=64.49.13.138:4487 , piece_id=MXQL4B6PL3FOQS4WCCGCPF5DT5C3JDLYJNOEIE7GKNE5FRQJZ7EA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144194345 (11h46m5.888251205s) [noise=true , address=79.127.205.245:4234 , piece_id=RADEPAKA2ZEQSI5UUX5BHRAR5JHNMOQI7MGZW42A4ZW52KJOXBRA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJWq44d6y4Ai4tyZpCUvCgZeMHrVdiiSSngTR75YZZXbsgq2d): protocol: expected piece hash; context canceled; EOF Aug 7 21:25:44.144213145 (11h46m5.888269992s) [noise=true , address=104.36.82.13:4195 , piece_id=YVPFOIDCG5F4LET7V5SZU2X6WKD4MCZWIQ6HUE7NVIOIVBRGDNHA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mmoBC7kBL5PVuzYvsq8Qb2kFNJNWQBTSCVq6xjkeyJxXPdPBH): protocol: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41808->104.36.82.13:4195: use of closed network connection Aug 7 21:25:44.144301767 (11h46m5.888358613s) [noise=true , address=104.36.82.16:4319 , piece_id=36CAJ2JJZF3CLN67PLA7VUM2F6NULTFSQ3WG6EDE734G3LNFZJTA, seg_pos={0 0}, upload=12171]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51894->104.36.82.16:4319: use of closed network connection Aug 7 21:25:44.144323393 (11h46m5.88838024s) [seg_pos={0 0}, upload=12171]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:25:44.144394152 (11h46m5.888450998s) [upload=12171]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:25:44.144398730 (11h46m5.888455591s) [upload=12171]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:47.207768845 (12h9m8.951825696s) [upload=12171]: Done uploading segments... Aug 7 21:48:47.207799297 (12h9m8.951856149s) [upload=12171]: Upload finished. err: metaclient: context deadline exceeded Duration:* 4.320331935s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........................] duration: 1.162869297s FAIL (2025-08-07 22:03:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.162889552s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [........X............X.X.X..................................] duration: 114.04041ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:03:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 114.058658ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX..X.XX..........................] duration: 1.08132719s FAIL (2025-08-07 22:03:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.081359997s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X............................] duration: 1.17678524s FAIL (2025-08-07 22:02:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.176822736s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [.............XXXXXXXXXXXXXXXXXXXXXXXXXXXXX..................] duration: 2.260414743s FAIL (2025-08-07 22:02:55 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.260503132s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.........XX..............XX.X...............................] duration: 336.233811ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:02:50 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 336.266317ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.......XXXXXXXXXXXXXXXXXXXXXXXX....X........................] duration: 3.242384156s FAIL (2025-08-07 22:02:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== :35.590125129 (12h5m57.33418199s) [upload=12475]: Got next segment. Inline: false Aug 7 21:45:35.590176001 (12h5m57.33423285s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:35.590181518 (12h5m57.334238369s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:35.699013188 (12h5m57.443070052s) [seg_pos={0 0}, upload=12475]: Begin upload segment... Aug 7 21:45:35.699066084 (12h5m57.443122935s) [seg_pos={0 0}, upload=12475]: Done begin upload segment. Aug 7 21:45:35.699091982 (12h5m57.443148842s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.699635648 (12h5m57.443692518s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700048642 (12h5m57.444105509s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700358698 (12h5m57.444415561s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.700672646 (12h5m57.444729509s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701028296 (12h5m57.445085159s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701400580 (12h5m57.44545744s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.701701900 (12h5m57.445758763s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702301822 (12h5m57.446358688s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.702831318 (12h5m57.446888179s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703272004 (12h5m57.447328915s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.703720399 (12h5m57.447777259s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704065340 (12h5m57.44812221s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704350352 (12h5m57.448407212s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.704707337 (12h5m57.448764198s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:35.705111011 (12h5m57.449167878s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Uploading piece... Aug 7 21:45:36.332909886 (12h5m58.076966752s) [upload=12475]: Next returned nil segment Aug 7 21:45:36.332933437 (12h5m58.076990311s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:45:36.489772849 (12h5m58.233829715s) [noise=true , address=104.36.82.10:4078 , piece_id=BEMP3VUCCLIJAY2OU2TFDJ5SQDJFO4OGL3UEHQ7YPMJGBRZ4E6BQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.509052197 (12h5m58.253109052s) [noise=true , address=104.36.82.12:4151 , piece_id=UUTIELJOX7CUEFHHFWDD4XETR3EB7IQOCKUZXQALKKGXPBVWH4TA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.542228341 (12h5m58.286285191s) [noise=true , address=77.90.31.12:4590 , piece_id=BF63RJ2G676UQFZGWIAXFCZR2DC67F4AH5VZAB3MYIDDOMJKGISA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573060953 (12h5m58.317117805s) [noise=true , address=45.91.116.6:4974 , piece_id=5HPCLRLKSQWUBMWQXW5LEOVQVBIKV62QAYF4YKQGQREWT3ZTAI2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.573821057 (12h5m58.317877923s) [noise=true , address=45.91.116.5:4299 , piece_id=YK7OVMQRE2PYUS2LDZOZGBKYV42U6VDWI2D6CMI4D2CIEV2DBACQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.620961481 (12h5m58.365018331s) [noise=true , address=77.90.31.17:4771 , piece_id=LDTJCTYFHCPL2SHCYONX3SJWBX3GME4GGILSOCHZWG67HKEAHDYQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.631043610 (12h5m58.375100483s) [noise=true , address=45.91.116.14:4191 , piece_id=TYBU2DWWTVWWEEMTBD2PIQKVIU3YRHKXH7ACD7Y3VKZVDXPDRALA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.632954530 (12h5m58.377011377s) [noise=true , address=45.91.116.11:4119 , piece_id=K5FKXMYXUWHRKFZPDHFKZ425RAZLEQH254K65QOITKTC4APUYL7A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.644557641 (12h5m58.388614491s) [noise=true , address=104.36.82.7:4911 , piece_id=6HSMRZBG26Y2XZ7NQZ6JD7P2BXZTDFDC6A5JIIS7SM7T6FAL2PWQ, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.649884441 (12h5m58.393941291s) [noise=true , address=77.90.31.13:4626 , piece_id=KM7OLI4GIFFIK4UTVRX26KMGVRZMASK47B7B7FOYCS6DU5SMZEXA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.661770778 (12h5m58.405827645s) [noise=true , address=77.90.31.15:4697 , piece_id=XQ5CZFWWVUZ6MKSIYZVOUZG6YAA7UAVVISSKVAPJXXD7HPXJHK3A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.725730770 (12h5m58.46978762s) [noise=true , address=104.36.82.3:4767 , piece_id=D3DBXBQ3UNQCWF5DR3HV46XZSOMRQHR33SQ6VSA7BXY7OZEWXNAA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869176760 (12h5m58.613233628s) [noise=true , address=45.91.116.11:4118 , piece_id=QPX3R57IYGZFNNOCPHOSQCRRQH6KJMDYSBPGHLX34DNABYYUXZ6A, seg_pos={0 0}, upload=12475]: Done uploading piece. err: <nil> Aug 7 21:45:36.869206102 (12h5m58.61326296s) [seg_pos={0 0}, upload=12475]: Segment reached optimal threshold of 13 pieces. Aug 7 21:45:36.869561423 (12h5m58.613618286s) [noise=true , address=104.36.82.11:4115 , piece_id=U6HOQABV6JV5T4MFFT3EDQF6R77LXTPWZ33FJWUEYOWLWLPFLYWA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 21:45:36.869806552 (12h5m58.613863414s) [noise=true , address=104.36.82.5:4840 , piece_id=V3VLVQPAAHDACW2WGUPYAGVDU2NRGR4FLQD2SFKVP2XLQALKP3SA, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36532->104.36.82.5:4840: use of closed network connection Aug 7 21:45:36.869995683 (12h5m58.614052547s) [noise=true , address=104.36.82.13:4187 , piece_id=WCIN3NV6AFBYDUGO7TDPOMZG5YJ24ISXVB2WSLB46EZOPPGQKW2Q, seg_pos={0 0}, upload=12475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39184->104.36.82.13:4187: use of closed network connection Aug 7 21:45:36.870032790 (12h5m58.614089658s) [seg_pos={0 0}, upload=12475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:45:36.870148681 (12h5m58.614205541s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:45:36.870154041 (12h5m58.614210892s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:37.015978570 (12h5m58.76003542s) [upload=12475]: Done uploading segments... Aug 7 21:45:37.016019019 (12h5m58.760075869s) [upload=12475]: Upload finished. err: <nil> Aug 7 21:46:05.392535892 (12h6m27.136592742s) [upload=12486]: Starting upload Aug 7 21:46:05.392632482 (12h6m27.136689348s) [upload=12486]: Done starting upload Aug 7 21:46:05.392784466 (12h6m27.136841327s) [upload=12486]: Uploading segments... Aug 7 21:46:05.392963140 (12h6m27.13701999s) [upload=12486]: Got next segment. Inline: false Aug 7 21:46:05.393017249 (12h6m27.137074109s) [upload=12486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:05.393023723 (12h6m27.137080582s) [upload=12486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:46:35.116242044 (12h6m56.860298909s) [upload=12486]: Next segment err: context deadline exceeded Aug 7 21:46:35.116407347 (12h6m56.860464196s) [upload=12486]: Done uploading segments... Aug 7 21:46:35.116450315 (12h6m56.860507172s) [upload=12486]: Upload finished. err: context deadline exceeded Duration: 3.242418832s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................XX.X..........X.....X.........X..X........] duration: 3.837090057s FAIL (2025-08-07 22:02:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 2679]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:41.477492343 (12h18m3.221549204s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:57:41.672062399 (12h18m3.416119246s) [seg_pos={0 0}, upload=12679]: Begin upload segment... Aug 7 21:57:41.672109341 (12h18m3.416166188s) [seg_pos={0 0}, upload=12679]: Done begin upload segment. Aug 7 21:57:41.672137062 (12h18m3.416193926s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672169185 (12h18m3.416226053s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672462219 (12h18m3.416519083s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672724395 (12h18m3.416781259s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672734877 (12h18m3.416791747s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.672974216 (12h18m3.41703108s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673211496 (12h18m3.41726836s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673247825 (12h18m3.417304692s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673482119 (12h18m3.417538982s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.673729154 (12h18m3.417786013s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675300194 (12h18m3.419357058s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.675937494 (12h18m3.419994342s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676204488 (12h18m3.420261334s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676430906 (12h18m3.420487772s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676657651 (12h18m3.420714511s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.676895247 (12h18m3.420952107s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677161207 (12h18m3.421218069s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:41.677432270 (12h18m3.421489131s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Uploading piece... Aug 7 21:57:42.251495665 (12h18m3.995552516s) [upload=12679]: Next returned nil segment Aug 7 21:57:42.251513937 (12h18m3.995570787s) [upload=12679]: Waiting for error group managing segments... Aug 7 21:57:42.431114374 (12h18m4.175171221s) [noise=true , address=104.36.82.62:4418 , piece_id=6VFBRNWKTFF2P4UJAAE4XKOMWMAJKUPOIG3T5NGE6ZMT6Z3SKVQQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.434754441 (12h18m4.178811288s) [noise=true , address=104.36.82.37:4686 , piece_id=3DREXEFJJYUFH6BU756HUYO5JDO34FKE4VAEEO2MWVEHZNJW76GQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457422325 (12h18m4.201479172s) [noise=true , address=104.36.82.45:4928 , piece_id=BOD4M7OI4ZITAN45I2ERBGJYVS2H324RQ2TMMHYBLGR5BWAMI4OQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.457520666 (12h18m4.201577512s) [noise=true , address=104.36.82.2:4760 , piece_id=QKSPASOUZX52MCAMHC3OMH7C4BYOWJMJVF6F6J3QTGZJRKTCHUTA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.470582115 (12h18m4.214638961s) [noise=true , address=104.36.82.9:4014 , piece_id=CL7K3YS7RROMSXUTM2RGRWKCI3ZBADO6GWB754HRPAJPSM5SOB5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.474337979 (12h18m4.218394829s) [noise=true , address=104.36.82.59:4332 , piece_id=VR62DTY6ZTLEPHLUKOX53PBEQPNBPFVNS3REGJ3V46UHODL2L4FA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.546312273 (12h18m4.290369142s) [noise=true , address=45.91.116.11:4136 , piece_id=UPSCGAAGZ3UU7Z6OJGF4KCIDWSE2SZN5MR4VUX2CNXZVRUDGODFQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.554650633 (12h18m4.298707502s) [noise=true , address=77.90.31.10:4521 , piece_id=JNTCR4HL4LYRY45WGGNGRBW5H54UEWSLKPIGFFCHLE6YU5QHOQPQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.561724122 (12h18m4.305780975s) [noise=true , address=45.91.116.9:4277 , piece_id=V4WYZ4XSEZCTDDPKSO3VXZIGWZRZB5REMX2M4JWFI4PGTAHNJA7A, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.575632750 (12h18m4.319689621s) [noise=true , address=104.36.82.61:4390 , piece_id=HXEJGDOAH4YRF6J4RNCF33VMFF3E3KXOECITHRUHLN2R37T5YTWQ, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.600836061 (12h18m4.344892927s) [noise=true , address=45.91.116.8:4058 , piece_id=KGWNQ4LHMKJOYRMDI6TO5ZRZXNOBU233ZWRGHA6AMUJGF7I56YOA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.852181705 (12h18m4.596238565s) [noise=true , address=104.36.82.23:4456 , piece_id=J47AVK3CT4FLU2QQKQPX47EIONN4IGJKP6UD6BVYVQUD25OB2WFA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.923085668 (12h18m4.667142529s) [noise=true , address=79.127.205.234:4602 , piece_id=T2CTWB2PW3MWQ3TAG2FTVO5DTYORBJ2TSX4MHVMPWU3VJKGAUJAA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924189057 (12h18m4.668245921s) [noise=true , address=64.49.13.138:4486 , piece_id=XDYTG3LQ7GSGRYSZJOXS6PJKRCBAGYBYR7YAPT4AU6HT2Q55VN5Q, seg_pos={0 0}, upload=12679]: Done uploading piece. err: <nil> Aug 7 21:57:42.924227116 (12h18m4.668283979s) [seg_pos={0 0}, upload=12679]: Segment reached optimal threshold of 14 pieces. Aug 7 21:57:42.924427387 (12h18m4.668484248s) [noise=true , address=79.127.205.233:4633 , piece_id=NVSRBP4FMLQ6MV3FTMXA6AGSFGKDNCTUNKLJL4SLYZFPWWUBDXZA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924565879 (12h18m4.668622735s) [noise=true , address=104.36.82.58:4267 , piece_id=PQL4UPNJAGTWHZFN6HETPIEFZ63QBLV3D4FMP7Y5KKPFA464UBBA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ektWku5yW6MRm2Rph8kiAAuifJJWZUMopaA3gcwStUfY6Nax8): protocol: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57418->104.36.82.58:4267: use of closed network connection Aug 7 21:57:42.924659012 (12h18m4.668715873s) [noise=true , address=79.127.205.246:4254 , piece_id=LFFMQL4KJD27G65LNYBOSLEQALPUK6IHASQEWABCIXX4C25JT4WA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: expected piece hash; context canceled; EOF Aug 7 21:57:42.924686911 (12h18m4.668743763s) [noise=true , address=104.36.82.35:4592 , piece_id=PIEU4ZNYHIHWVEPBEO5X3AD2Z5DCBKD6JMI73COO3QSKUP7NLADA, seg_pos={0 0}, upload=12679]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fB7rqiDhMNQ4j9smkctbKdnGCXXdfFPWTgTxRrwQVZ9fKEFeZ): protocol: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54806->104.36.82.35:4592: use of closed network connection Aug 7 21:57:42.924713955 (12h18m4.668770815s) [seg_pos={0 0}, upload=12679]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:57:42.924812314 (12h18m4.668869174s) [upload=12679]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:57:42.924819863 (12h18m4.668876723s) [upload=12679]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:43.111275435 (12h18m4.855332295s) [upload=12679]: Done uploading segments... Aug 7 21:57:43.111294851 (12h18m4.855351702s) [upload=12679]: Upload finished. err: <nil> Duration: 3.837127175s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X......................................X...X.] duration: 970.845302ms FAIL (2025-08-07 22:02:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 970.872731ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........................] duration: 2.726078565s FAIL (2025-08-07 22:02:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.726111159s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........................] duration: 1.117909486s FAIL (2025-08-07 22:02:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:34:39.735850106 (11h55m1.479906957s) [upload=12318]: Uploading segments... Aug 7 21:34:39.735909766 (11h55m1.479966635s) [upload=12318]: Got next segment. Inline: true Aug 7 21:34:39.735915314 (11h55m1.479972174s) [upload=12318]: Waiting for error group managing segments... Aug 7 21:34:39.736018840 (11h55m1.480075701s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:39.736024699 (11h55m1.480081559s) [upload=12318]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:39.736028031 (11h55m1.480084892s) [upload=12318]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:09.464049958 (11h55m31.208106818s) [upload=12318]: Done uploading segments... Aug 7 21:35:09.464072716 (11h55m31.208129576s) [upload=12318]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:34.656807174 (11h55m56.400864101s) [upload=12328]: Starting upload Aug 7 21:35:34.656894285 (11h55m56.400951152s) [upload=12328]: Done starting upload Aug 7 21:35:34.656967676 (11h55m56.401024525s) [upload=12328]: Uploading segments... Aug 7 21:35:34.657016134 (11h55m56.401073003s) [upload=12328]: Got next segment. Inline: true Aug 7 21:35:34.657021131 (11h55m56.401077993s) [upload=12328]: Waiting for error group managing segments... Aug 7 21:35:34.657065881 (11h55m56.401122731s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:34.657071386 (11h55m56.401128236s) [upload=12328]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:34.657074624 (11h55m56.401131484s) [upload=12328]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:04.380124890 (11h56m26.124181742s) [upload=12328]: Done uploading segments... Aug 7 21:36:04.380153582 (11h56m26.124210448s) [upload=12328]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:04.798191174 (11h56m26.542248041s) [upload=12340]: Starting upload Aug 7 21:36:04.798286128 (11h56m26.542342995s) [upload=12340]: Done starting upload Aug 7 21:36:04.798372008 (11h56m26.542428871s) [upload=12340]: Uploading segments... Aug 7 21:36:04.798434433 (11h56m26.542491304s) [upload=12340]: Got next segment. Inline: true Aug 7 21:36:04.798441202 (11h56m26.542498066s) [upload=12340]: Waiting for error group managing segments... Aug 7 21:36:04.798496298 (11h56m26.542553165s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:04.798502276 (11h56m26.542559141s) [upload=12340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:04.798505608 (11h56m26.542562472s) [upload=12340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:34.510180222 (11h56m56.254237082s) [upload=12340]: Done uploading segments... Aug 7 21:36:34.510201902 (11h56m56.254258762s) [upload=12340]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:16.815303797 (11h57m38.559360649s) [upload=12354]: Starting upload Aug 7 21:37:16.815396631 (11h57m38.559453502s) [upload=12354]: Done starting upload Aug 7 21:37:16.815743613 (11h57m38.559800465s) [upload=12354]: Uploading segments... Aug 7 21:37:16.815841404 (11h57m38.559898275s) [upload=12354]: Got next segment. Inline: true Aug 7 21:37:16.815849471 (11h57m38.559906335s) [upload=12354]: Waiting for error group managing segments... Aug 7 21:37:16.815913502 (11h57m38.559970351s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:16.815919848 (11h57m38.559976709s) [upload=12354]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:16.815924222 (11h57m38.559981087s) [upload=12354]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:46.539729655 (11h58m8.283786508s) [upload=12354]: Done uploading segments... Aug 7 21:37:46.539785208 (11h58m8.283842069s) [upload=12354]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:27.592061643 (11h58m49.336118515s) [upload=12370]: Starting upload Aug 7 21:38:27.592133721 (11h58m49.336190591s) [upload=12370]: Done starting upload Aug 7 21:38:27.592194600 (11h58m49.336251462s) [upload=12370]: Uploading segments... Aug 7 21:38:27.592243475 (11h58m49.336300347s) [upload=12370]: Got next segment. Inline: true Aug 7 21:38:27.592249638 (11h58m49.336306506s) [upload=12370]: Waiting for error group managing segments... Aug 7 21:38:27.592301586 (11h58m49.336358453s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:27.592307915 (11h58m49.33636478s) [upload=12370]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:27.592314515 (11h58m49.336371384s) [upload=12370]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:57.295172349 (11h59m19.039229199s) [upload=12370]: Done uploading segments... Aug 7 21:38:57.295199050 (11h59m19.0392559s) [upload=12370]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:57.705852359 (11h59m19.449909219s) [upload=12378]: Starting upload Aug 7 21:38:57.705946659 (11h59m19.450003522s) [upload=12378]: Done starting upload Aug 7 21:38:57.706046021 (11h59m19.450102881s) [upload=12378]: Uploading segments... Aug 7 21:38:57.706107301 (11h59m19.450164171s) [upload=12378]: Got next segment. Inline: true Aug 7 21:38:57.706112988 (11h59m19.450169848s) [upload=12378]: Waiting for error group managing segments... Aug 7 21:38:57.706170165 (11h59m19.450227028s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:57.706175372 (11h59m19.450232233s) [upload=12378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:57.706178976 (11h59m19.450235838s) [upload=12378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:27.434609736 (11h59m49.178666597s) [upload=12378]: Done uploading segments... Aug 7 21:39:27.434644375 (11h59m49.178701235s) [upload=12378]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:56.317558286 (12h0m18.061615147s) [upload=12392]: Starting upload Aug 7 21:39:56.317614127 (12h0m18.061670988s) [upload=12392]: Done starting upload Aug 7 21:39:56.317683655 (12h0m18.061740509s) [upload=12392]: Uploading segments... Aug 7 21:39:56.317748361 (12h0m18.061805241s) [upload=12392]: Got next segment. Inline: true Aug 7 21:39:56.317754805 (12h0m18.061811669s) [upload=12392]: Waiting for error group managing segments... Aug 7 21:39:56.317831053 (12h0m18.061887914s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:56.317837732 (12h0m18.061894593s) [upload=12392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:56.317841055 (12h0m18.061897915s) [upload=12392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:26.032549829 (12h0m47.776606689s) [upload=12392]: Done uploading segments... Aug 7 21:40:26.032570897 (12h0m47.776627757s) [upload=12392]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:41.354808211 (12h1m3.098865072s) [upload=12402]: Starting upload Aug 7 21:40:41.354938701 (12h1m3.098995568s) [upload=12402]: Done starting upload Aug 7 21:40:41.355036214 (12h1m3.099093064s) [upload=12402]: Uploading segments... Aug 7 21:40:41.355109185 (12h1m3.099166055s) [upload=12402]: Got next segment. Inline: true Aug 7 21:40:41.355116835 (12h1m3.0991737s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:40:41.355183407 (12h1m3.09924027s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:41.355190433 (12h1m3.099247297s) [upload=12402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:41.355194357 (12h1m3.09925122s) [upload=12402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:11.078938147 (12h1m32.822994999s) [upload=12402]: Done uploading segments... Aug 7 21:41:11.078958711 (12h1m32.823015562s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.899938086 (12h1m56.643994947s) [upload=12417]: Starting upload Aug 7 21:41:34.900014273 (12h1m56.644071133s) [upload=12417]: Done starting upload Aug 7 21:41:34.900081071 (12h1m56.644137926s) [upload=12417]: Uploading segments... Aug 7 21:41:34.900153322 (12h1m56.644210172s) [upload=12417]: Got next segment. Inline: true Aug 7 21:41:34.900158578 (12h1m56.644215438s) [upload=12417]: Waiting for error group managing segments... Aug 7 21:41:34.900228776 (12h1m56.644285623s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.900237327 (12h1m56.644294194s) [upload=12417]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.900241831 (12h1m56.644298696s) [upload=12417]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:04.627262793 (12h2m26.371319653s) [upload=12417]: Done uploading segments... Aug 7 21:42:04.627301743 (12h2m26.371358604s) [upload=12417]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.477725352 (12h2m40.221782248s) [upload=12428]: Starting upload Aug 7 21:42:18.477815072 (12h2m40.221871935s) [upload=12428]: Done starting upload Aug 7 21:42:18.477903321 (12h2m40.221960189s) [upload=12428]: Uploading segments... Aug 7 21:42:18.478047763 (12h2m40.222104633s) [upload=12428]: Got next segment. Inline: true Aug 7 21:42:18.478056185 (12h2m40.222113051s) [upload=12428]: Waiting for error group managing segments... Aug 7 21:42:18.478131478 (12h2m40.222188328s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.478138171 (12h2m40.222195021s) [upload=12428]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.478144895 (12h2m40.222201761s) [upload=12428]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:48.201994950 (12h3m9.946051801s) [upload=12428]: Done uploading segments... Aug 7 21:42:48.202028187 (12h3m9.946085036s) [upload=12428]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.117936042s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)