Use natural sizes for S3I(s)->tmp's *_md arrays

It is a historical artifact that cert_verify_md[], finish_md[] and
peer_finish_md[] are twice as large as they need to be. This is
confusing, especially for finish_md[] and peer_finish_md[] which are
copied to to previous_client_finished[] and previous_server_finished[]
which are only half as large. It is easy to check that they will never
get more than EVP_MAX_MD_SIZE data written to them.

In 1998, EVP_MAX_MD_SIZE was 20 bytes long (for SHA-1). This got bumped to
16+20 for the SSLv3-specific md5+sha1.  Apparently under the impression
that EVP_MAX_MD_SIZE was still 20 bytes, someone else doubled finish_md[]'s
size to EVP_MAX_MD_SIZE*2 and added /* actually only needs to be 16+20 */.
A bit later finish_md[] was split up, and still a bit later the comment was
amended for TLSv1.  Shortly thereafter SHA-512 required a bump of
EVP_MAX_MD_SIZE to 64 by a third person and we have been carrying 192 bytes
of untouched memory in each of our SSLs ever since.

ok inoguchi jsing (jsing had the same diff)

1 files changed, 4 insertions, 6 deletions

diff --git a/src/lib/libssl/ssl_locl.h b/src/lib/libssl/ssl_locl.h
index 80a7d95be5..312c18f7ca 100644
--- a/src/lib/libssl/ssl_locl.h
+++ b/src/lib/libssl/ssl_locl.h
@@ -1,4 +1,4 @@
-/* $OpenBSD: ssl_locl.h,v 1.309 2020/12/14 15:20:31 tb Exp $ */
+/* $OpenBSD: ssl_locl.h,v 1.310 2020/12/15 16:04:49 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -892,13 +892,11 @@ typedef struct ssl3_state_internal_st {
         struct  {
                 int new_mac_secret_size;
 
-                /* actually only needs to be 16+20 */
-                unsigned char cert_verify_md[EVP_MAX_MD_SIZE*2];
+                unsigned char cert_verify_md[EVP_MAX_MD_SIZE];
 
-                /* actually only need to be 16+20 for SSLv3 and 12 for TLS */
-                unsigned char finish_md[EVP_MAX_MD_SIZE*2];
+                unsigned char finish_md[EVP_MAX_MD_SIZE];
                 size_t finish_md_len;
-                unsigned char peer_finish_md[EVP_MAX_MD_SIZE*2];
+                unsigned char peer_finish_md[EVP_MAX_MD_SIZE];
                 size_t peer_finish_md_len;
 
                 unsigned long message_size;