Scapy special for DH_check()

The latest release of Scapy calls DH_check() on all the well-known
Diffie-Hellman parameters for RFCs 2409, 3526, and 7919. It does this
via pyca/cryptography at startup. Every single time. This is obviously
very expensive, due to our 64 MR rounds (which are complete overkill
now that we have BPSW). Instead of pondering the ideal number of rounds
for BPSW with FFDH, simply skip the check if the parameter matches a
well-known prime. These are known to be safe primes, so we can skip
those super-expensive and pointless checks without any risk.

This is only done for the public dh->p parameter. It could be further
optimized, but with the follow-up commit adding the RFC 7919 primes this
reduces the startup time to what it was before Scapy 2.7.0: < 1s.

Reverting from 64 MR rounds to BN_check_primes rounds, we would still
have ~8s startup time without this optimization, which isn't great for
an interactive tool.

Clearly, it's not entirely our fault, it's also Scapy and cryptography
that do something ... suboptimal, but I think we're better off if
DH_check() isn't a complete DoS vector. If you're using non-standard
parameters with FFDH, you deserve it.

We could consider adding a flag for non-well-known p and thus making
DH_check() indicate failure for candidate primes larger than, say, 4k.

https://github.com/pyca/cryptography/issues/14048

ok beck kenjiro

1 files changed, 60 insertions, 2 deletions

diff --git a/src/lib/libcrypto/dh/dh_check.c b/src/lib/libcrypto/dh/dh_check.c
index 1ba85bc824..143699b4f6 100644
--- a/src/lib/libcrypto/dh/dh_check.c
+++ b/src/lib/libcrypto/dh/dh_check.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: dh_check.c,v 1.31 2025/05/10 05:54:38 tb Exp $ */
+/* $OpenBSD: dh_check.c,v 1.32 2026/01/23 08:21:52 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -104,6 +104,58 @@ DH_check_params(const DH *dh, int *flags)
         return ok;
 }
 
+typedef BIGNUM *(*get_p_fn)(BIGNUM *);
+
+static const get_p_fn get_well_known_p[] = {
+        BN_get_rfc2409_prime_768,
+        BN_get_rfc2409_prime_1024,
+        BN_get_rfc3526_prime_1536,
+        BN_get_rfc3526_prime_2048,
+        BN_get_rfc3526_prime_3072,
+        BN_get_rfc3526_prime_4096,
+        BN_get_rfc3526_prime_6144,
+        BN_get_rfc3526_prime_8192,
+};
+
+#define N_WELL_KNOWN_P_FN (sizeof(get_well_known_p) / sizeof(get_well_known_p[0]))
+
+/*
+ * Scapy special: on startup it now calls DH_check() on all the well-known DH
+ * primes, which is a sensible thing to do. In any case, using BN_is_prime_ex()
+ * on a standardized domain parameter is dumb, so avoid it.
+ */
+static int
+dh_is_well_known_p(const BIGNUM *p, BN_CTX *ctx, int *is_well_known)
+{
+        BIGNUM *bn;
+        size_t i;
+        int ret = 0;
+
+        *is_well_known = 0;
+
+        BN_CTX_start(ctx);
+        if ((bn = BN_CTX_get(ctx)) == NULL)
+                goto err;
+
+        for (i = 0; i < N_WELL_KNOWN_P_FN; i++) {
+                get_p_fn get_p = get_well_known_p[i];
+
+                if (get_p(bn) == NULL)
+                        goto err;
+                if (BN_cmp(bn, p) == 0) {
+                        *is_well_known = 1;
+                        break;
+                }
+        }
+
+        ret = 1;
+
+ err:
+        BN_CTX_end(ctx);
+
+        return ret;
+}
+
 /*
  * Check that p is a safe prime and that g is a suitable generator.
  */
@@ -112,7 +164,7 @@ int
 DH_check(const DH *dh, int *flags)
 {
         BN_CTX *ctx = NULL;
-        int is_prime;
+        int is_prime, is_well_known;
         int ok = 0;
 
         *flags = 0;
@@ -150,6 +202,11 @@ DH_check(const DH *dh, int *flags)
                         *flags |= DH_CHECK_INVALID_Q_VALUE;
         }
 
+        if (!dh_is_well_known_p(dh->p, ctx, &is_well_known))
+                goto err;
+        if (is_well_known)
+                goto done;
+
         is_prime = BN_is_prime_ex(dh->p, DH_NUMBER_ITERATIONS_FOR_PRIME,
             ctx, NULL);
         if (is_prime < 0)
@@ -171,6 +228,7 @@ DH_check(const DH *dh, int *flags)
                         *flags |= DH_CHECK_P_NOT_SAFE_PRIME;
         }
 
+ done:
         ok = 1;
 
  err: