Add sanity checks on p and q in old_dsa_priv_decode()

dsa_do_verify() has checks on dsa->p and dsa->q that ensure that p isn't
overly long and that q has one of the three allowed lengths specified in
FIPS 186-3, namely 160, 224, or 256.

Do these checks on deserialization of DSA keys without parameters. This
means that we will now reject keys we would previously deserialize. Such
keys are useless in that signatures generated by them would be rejected
by both LibreSSL and OpenSSL.

This avoids a timeout flagged in oss-fuzz #26899 due to a ridiculous
DSA key whose q has size 65KiB. The timeout comes from additional checks
on DSA keys added by miod in dsa_ameth.c r1.18, especially checking such
a humungous number for primality is expensive.

ok jsing

1 files changed, 15 insertions, 1 deletions

diff --git a/src/lib/libcrypto/dsa/dsa_ameth.c b/src/lib/libcrypto/dsa/dsa_ameth.c
index 4e8f4ac825..eb4d5d2dcd 100644
--- a/src/lib/libcrypto/dsa/dsa_ameth.c
+++ b/src/lib/libcrypto/dsa/dsa_ameth.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: dsa_ameth.c,v 1.32 2022/01/15 04:02:37 tb Exp $ */
+/* $OpenBSD: dsa_ameth.c,v 1.33 2022/02/24 08:31:11 tb Exp $ */
 /* Written by Dr Stephen N Henson (steve@openssl.org) for the OpenSSL
  * project 2006.
  */
@@ -480,12 +480,26 @@ old_dsa_priv_decode(EVP_PKEY *pkey, const unsigned char **pder, int derlen)
         DSA *dsa;
         BN_CTX *ctx = NULL;
         BIGNUM *j, *p1, *newp1;
+        int qbits;
 
         if (!(dsa = d2i_DSAPrivateKey(NULL, pder, derlen))) {
                 DSAerror(ERR_R_DSA_LIB);
                 return 0;
         }
 
+        DSA_print_fp(stdout, dsa, 0);
+
+        /* FIPS 186-3 allows only three different sizes for q. */
+        qbits = BN_num_bits(dsa->q);
+        if (qbits != 160 && qbits != 224 && qbits != 256) {
+                DSAerror(DSA_R_BAD_Q_VALUE);
+                goto err;
+        }
+        if (BN_num_bits(dsa->p) > OPENSSL_DSA_MAX_MODULUS_BITS) {
+                DSAerror(DSA_R_MODULUS_TOO_LARGE);
+                goto err;
+        }
+
         ctx = BN_CTX_new();
         if (ctx == NULL)
                 goto err;