Make the bn_rand_interval() API a bit more ergonomic

Provide bn_rand_in_range() which is a slightly tweaked version of what was
previously called bn_rand_range().

The way bn_rand_range() is called in libcrypto, the lower bound is always
expressible as a word. In fact, most of the time it is 1, the DH code uses
a 2, the MR tests in BPSW use 3 and an exceptinally high number appears in
the Tonelli-Shanks implementation where we use 32. Converting these lower
bounds to BIGNUMs on the call site is annoying so let bn_rand_interval()
do that internally and route that through bn_rand_in_range(). This way we
can avoid using BN_sub_word().

Adjust the bn_isqrt() test to use bn_rand_in_range() since that's the
only caller that uses actual BIGNUMs as lower bounds.

ok jsing

1 files changed, 3 insertions, 7 deletions

diff --git a/src/lib/libcrypto/bn/bn_mod_sqrt.c b/src/lib/libcrypto/bn/bn_mod_sqrt.c
index bdd5b2cdba..280002cc48 100644
--- a/src/lib/libcrypto/bn/bn_mod_sqrt.c
+++ b/src/lib/libcrypto/bn/bn_mod_sqrt.c
@@ -1,4 +1,4 @@
-/*      $OpenBSD: bn_mod_sqrt.c,v 1.2 2023/07/08 12:21:58 beck Exp $ */
+/*      $OpenBSD: bn_mod_sqrt.c,v 1.3 2023/08/03 18:53:55 tb Exp $ */
 
 /*
  * Copyright (c) 2022 Theo Buehler <tb@openbsd.org>
@@ -237,7 +237,7 @@ static int
 bn_mod_sqrt_find_sylow_generator(BIGNUM *out_generator, const BIGNUM *p,
     const BIGNUM *q, BN_CTX *ctx)
 {
-        BIGNUM *n, *p_abs, *thirty_two;
+        BIGNUM *n, *p_abs;
         int i, is_non_residue;
         int ret = 0;
 
@@ -245,8 +245,6 @@ bn_mod_sqrt_find_sylow_generator(BIGNUM *out_generator, const BIGNUM *p,
 
         if ((n = BN_CTX_get(ctx)) == NULL)
                 goto err;
-        if ((thirty_two = BN_CTX_get(ctx)) == NULL)
-                goto err;
         if ((p_abs = BN_CTX_get(ctx)) == NULL)
                 goto err;
 
@@ -259,14 +257,12 @@ bn_mod_sqrt_find_sylow_generator(BIGNUM *out_generator, const BIGNUM *p,
                         goto found;
         }
 
-        if (!BN_set_word(thirty_two, 32))
-                goto err;
         if (!bn_copy(p_abs, p))
                 goto err;
         BN_set_negative(p_abs, 0);
 
         for (i = 0; i < 128; i++) {
-                if (!bn_rand_interval(n, thirty_two, p_abs))
+                if (!bn_rand_interval(n, 32, p_abs))
                         goto err;
                 if (!bn_mod_sqrt_n_is_non_residue(&is_non_residue, n, p, ctx))
                         goto err;