Remove support for static buffers in HMAC/digests

HMAC() and the one-step digests used to support passing a NULL buffer and
would return the digest in a static buffer. This design is firmly from the
nineties, not thread safe and it saves callers a single line. The few ports
that used to rely this were fixed with patches sent to non-hostile (and
non-dead) upstreams. It's early enough in the release cycle that remaining
uses hidden from the compiler should be caught, at least the ones that
matter.

There won't be that many since BoringSSL removed this feature in 2017.
https://boringssl-review.googlesource.com/14528

Add non-null attributes to the headers and add a few missing bounded
attributes.

ok beck jsing

2 files changed, 3 insertions, 5 deletions

diff --git a/src/lib/libcrypto/md5/md5.c b/src/lib/libcrypto/md5/md5.c
index 35d1ac9144..744c66f005 100644
--- a/src/lib/libcrypto/md5/md5.c
+++ b/src/lib/libcrypto/md5/md5.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: md5.c,v 1.22 2024/03/28 08:00:08 jsing Exp $ */
+/* $OpenBSD: md5.c,v 1.23 2024/06/01 07:36:16 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -371,10 +371,7 @@ unsigned char *
 MD5(const unsigned char *d, size_t n, unsigned char *md)
 {
         MD5_CTX c;
-        static unsigned char m[MD5_DIGEST_LENGTH];
 
-        if (md == NULL)
-                md = m;
         if (!MD5_Init(&c))
                 return NULL;
         MD5_Update(&c, d, n);
diff --git a/src/lib/libcrypto/md5/md5.h b/src/lib/libcrypto/md5/md5.h
index d248c93a85..9191ff2131 100644
--- a/src/lib/libcrypto/md5/md5.h
+++ b/src/lib/libcrypto/md5/md5.h
@@ -1,4 +1,4 @@
-/* $OpenBSD: md5.h,v 1.21 2023/07/08 06:50:38 jsing Exp $ */
+/* $OpenBSD: md5.h,v 1.22 2024/06/01 07:36:16 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -98,6 +98,7 @@ int MD5_Update(MD5_CTX *c, const void *data, size_t len)
     __attribute__ ((__bounded__(__buffer__, 2, 3)));
 int MD5_Final(unsigned char *md, MD5_CTX *c);
 unsigned char *MD5(const unsigned char *d, size_t n, unsigned char *md)
+    __attribute__ ((__nonnull__(3)))
     __attribute__ ((__bounded__(__buffer__, 1, 2)));
 void MD5_Transform(MD5_CTX *c, const unsigned char *b);
 #ifdef  __cplusplus