Make use of X509_get_signature_info() in check_sig_level()

If an auth_level (i.e., security_level, but not quite, because Viktor) was
set on the X509_VERIFY_PARAM in the X509_STORE_CTX, the verifier would
reject RSA-PSS or EdDSA certificates for insufficient security bits due to
incorrect use of OBJ_find_sigid_algs() (this was also a bug in the initial
security level implementation in OpenSSL 1.1).

Using X509_get_signature_info() fixes this while preserving behavior for
all other algorithms.

Reported by Steffen Ullrich as one of multiple issues with RSA-PSS.

ok jsing

1 files changed, 3 insertions, 20 deletions

diff --git a/src/lib/libcrypto/x509/x509_vfy.c b/src/lib/libcrypto/x509/x509_vfy.c
index 4f597fa313..78ec8a4e81 100644
--- a/src/lib/libcrypto/x509/x509_vfy.c
+++ b/src/lib/libcrypto/x509/x509_vfy.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: x509_vfy.c,v 1.144 2024/08/04 08:15:36 tb Exp $ */
+/* $OpenBSD: x509_vfy.c,v 1.145 2024/08/28 07:37:50 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -2541,28 +2541,11 @@ check_key_level(X509_STORE_CTX *ctx, X509 *cert)
 static int
 check_sig_level(X509_STORE_CTX *ctx, X509 *cert)
 {
-        const EVP_MD *md;
-        int bits, nid, md_nid;
-
-        if ((nid = X509_get_signature_nid(cert)) == NID_undef)
-                return 0;
-
-        /*
-         * Look up signature algorithm digest.
-         */
-
-        if (!OBJ_find_sigid_algs(nid, &md_nid, NULL))
-                return 0;
-
-        if (md_nid == NID_undef)
-                return 0;
+        int bits;
 
-        if ((md = EVP_get_digestbynid(md_nid)) == NULL)
+        if (!X509_get_signature_info(cert, NULL, NULL, &bits, NULL))
                 return 0;
 
-        /* Assume 4 bits of collision resistance for each hash octet. */
-        bits = EVP_MD_size(md) * 4;
-
         return enough_bits_for_security_level(bits, ctx->param->security_level);
 }