Clean up the TLS handshake digest handling - this refactors some of the

code for improved readability, however it also address two issues.

The first of these is a hard-to-hit double free that will occur if
EVP_DigestInit_ex() fails. To avoid this and to be more robust, ensure
that tls1_digest_cached_records() either completes successfully and sets
up all of the necessary digests, or it cleans up and frees everything
that was allocated.

The second issue is that EVP_DigestUpdate() can fail - detect and handle
this in tls1_finish_mac() and change the return type to an int so that a
failure can be propagated to the caller (the callers still need to be
fixed to handle this, in a later diff).

The double-free was reported by Matthew Dillon.

ok beck@ doug@ miod@

1 files changed, 2 insertions, 2 deletions

diff --git a/src/lib/libssl/ssl_locl.h b/src/lib/libssl/ssl_locl.h
index 1b768e3939..b79e9269ce 100644
--- a/src/lib/libssl/ssl_locl.h
+++ b/src/lib/libssl/ssl_locl.h
@@ -1,4 +1,4 @@
-/* $OpenBSD: ssl_locl.h,v 1.130 2016/10/19 16:38:40 jsing Exp $ */
+/* $OpenBSD: ssl_locl.h,v 1.131 2016/11/03 08:15:22 jsing Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -754,7 +754,7 @@ int ssl_init_wbio_buffer(SSL *s, int push);
 void ssl_free_wbio_buffer(SSL *s);
 
 int tls1_init_finished_mac(SSL *s);
-void tls1_finish_mac(SSL *s, const unsigned char *buf, int len);
+int tls1_finish_mac(SSL *s, const unsigned char *buf, int len);
 void tls1_free_digest_list(SSL *s);
 void tls1_cleanup_key_block(SSL *s);
 int tls1_digest_cached_records(SSL *s);