X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

1 files changed, 1 insertions, 3 deletions

diff --git a/src/lib/libcrypto/x509/x509_purp.c b/src/lib/libcrypto/x509/x509_purp.c
index ffd986b4a1..ab5e7cb3c9 100644
--- a/src/lib/libcrypto/x509/x509_purp.c
+++ b/src/lib/libcrypto/x509/x509_purp.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: x509_purp.c,v 1.15 2022/04/21 04:48:12 tb Exp $ */
+/* $OpenBSD: x509_purp.c,v 1.16 2022/05/10 19:42:52 tb Exp $ */
 /* Written by Dr Stephen N Henson (steve@openssl.org) for the OpenSSL
  * project 2001.
  */
@@ -671,8 +671,6 @@ X509_check_ca(X509 *x)
                 CRYPTO_w_lock(CRYPTO_LOCK_X509);
                 x509v3_cache_extensions(x);
                 CRYPTO_w_unlock(CRYPTO_LOCK_X509);
-                if (x->ex_flags & EXFLAG_INVALID)
-                        return X509_V_ERR_UNSPECIFIED;
         }
 
         return check_ca(x);