Fix underlying pkey of RSA-PSS

There are currently very few differences between the rsa_asn1_meth for
plain RSA and the rsa_pss_asn1_meth for RSA-PSS apart from the obviously
necessary differnces for base_method, pkey_id, pem_str and info (and the
fact that RSA has support for legacy private key encoding). This had the
lucky side effect that it didn't really matter which ameth one ended up
using after OBJ_find_sigid_algs().

With the upcoming support for X509_get_signature_infO() for RSA-PSS, this
needs to change as for RSA-PSS we need to decode the PSS parameters for
extracting the "security level", whereas for RSA we can just use the hash
length. Thus, for RSA-PSS switch pkey_id from the incorrect rsaEncryption
to rassaPss.

ok jsing

PS: OBJ_find_sigid_algs() manual is no longer entirely correct, but this
has been the case since we added Ed25519 support to obj_xref.

1 files changed, 2 insertions, 2 deletions

diff --git a/src/lib/libcrypto/objects/obj_xref.c b/src/lib/libcrypto/objects/obj_xref.c
index 321c2f50b6..04611d68e3 100644
--- a/src/lib/libcrypto/objects/obj_xref.c
+++ b/src/lib/libcrypto/objects/obj_xref.c
@@ -1,4 +1,4 @@
-/*      $OpenBSD: obj_xref.c,v 1.14 2024/01/27 16:08:43 tb Exp $ */
+/*      $OpenBSD: obj_xref.c,v 1.15 2024/08/28 06:53:24 tb Exp $ */
 
 /*
  * Copyright (c) 2023 Theo Buehler <tb@openbsd.org>
@@ -178,7 +178,7 @@ static const struct {
         {
                 .sign_nid = NID_rsassaPss,
                 .hash_nid = NID_undef,
-                .pkey_nid = NID_rsaEncryption,
+                .pkey_nid = NID_rsassaPss,
         },
         {
                 .sign_nid = NID_id_tc26_signwithdigest_gost3410_2012_256,