Unlock CRYPTO_LOCK_UI on ui_open_session() failure

Both ui_open_session() implementations, open_console() in ui_openssl.c
in base, and the one in ui_openssl_win.c in portable, grab the lock of
type CRYPTO_LOCK_UI before doing anything else.

The only internal (and, as far as I can tell, the only existing) caller,
UI_process(), returns immediately on failure. The calling thread thus
keeps holding the lock and the next call to UI_process() will block
indefinitely. Fix this by using the common exit path, which calls
ui_close_session() aka close_console(), both implementations of which
release the lock.

Thanks to Kartik (@sage-mode-hunter) who proposed an alternative fix for
ui_openssl_win.c, which we would have to apply to ui_openssl.c as well.

Matches OpenSSL behavior since PR #2037
Closes https://github.com/libressl/portable/pull/1334

ok kenjiro

1 files changed, 5 insertions, 3 deletions

diff --git a/src/lib/libcrypto/ui/ui_lib.c b/src/lib/libcrypto/ui/ui_lib.c
index 3cfb3d371a..4562464cc7 100644
--- a/src/lib/libcrypto/ui/ui_lib.c
+++ b/src/lib/libcrypto/ui/ui_lib.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: ui_lib.c,v 1.54 2026/05/19 09:23:36 tb Exp $ */
+/* $OpenBSD: ui_lib.c,v 1.55 2026/07/25 07:33:43 tb Exp $ */
 /* Written by Richard Levitte (richard@levitte.org) for the OpenSSL
  * project 2001.
  */
@@ -461,8 +461,10 @@ UI_process(UI *ui)
 {
         int i, ok = 0;
 
-        if (ui->meth->ui_open_session && !ui->meth->ui_open_session(ui))
-                return -1;
+        if (ui->meth->ui_open_session && !ui->meth->ui_open_session(ui)) {
+                ok = -1;
+                goto err;
+        }
 
         if (ui->flags & UI_FLAG_PRINT_ERRORS)
                 ERR_print_errors_cb(print_error, ui);