Annotate RSA-PSS SHA parameter encoding as wrong

A historic blunderfest in the ASN.1 module for RSA-PSS led to very
confusing text in various RFCs. davidben and my current reading of
this is that parameters for SHA-* should be encoded as an ASN.1 NULL
rather than omitted. The use of X509_ALGOR_set_evp_md() leads to them
being omitted, and is therefore counter to the specification (but
allowed. We should fix this. For now, leave a reminder.

See https://boringssl-review.googlesource.com/c/boringssl/+/67088
for a lot more details.

ok davidben

1 files changed, 9 insertions, 1 deletions

diff --git a/src/lib/libcrypto/rsa/rsa_ameth.c b/src/lib/libcrypto/rsa/rsa_ameth.c
index 0c351f49ef..c722188c43 100644
--- a/src/lib/libcrypto/rsa/rsa_ameth.c
+++ b/src/lib/libcrypto/rsa/rsa_ameth.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: rsa_ameth.c,v 1.57 2024/01/10 14:59:19 tb Exp $ */
+/* $OpenBSD: rsa_ameth.c,v 1.58 2024/03/17 07:10:00 tb Exp $ */
 /* Written by Dr Stephen N Henson (steve@openssl.org) for the OpenSSL
  * project 2006.
  */
@@ -605,6 +605,10 @@ rsa_md_to_algor(const EVP_MD *md, X509_ALGOR **out_alg)
 
         if ((alg = X509_ALGOR_new()) == NULL)
                 goto err;
+        /*
+         * XXX - This omits the parameters, whereas RFC 4055, section 2.1
+         * explicitly states that an explicit ASN.1 NULL is required.
+         */
         if (!X509_ALGOR_set_evp_md(alg, md))
                 goto err;
 
@@ -640,6 +644,10 @@ rsa_mgf1md_to_maskGenAlgorithm(const EVP_MD *mgf1md, X509_ALGOR **out_alg)
 
         if ((inner_alg = X509_ALGOR_new()) == NULL)
                 goto err;
+        /*
+         * XXX - This omits the parameters, whereas RFC 4055, section 2.1
+         * explicitly states that an explicit ASN.1 NULL is required.
+         */
         if (!X509_ALGOR_set_evp_md(inner_alg, mgf1md))
                 goto err;
         if ((astr = ASN1_item_pack(inner_alg, &X509_ALGOR_it, NULL)) == NULL)