Add error handling for EVP_DigestInit_ex().

A few EVP_DigestInit_ex() calls were left alone since reporting an
error would change the public API.

Changed internal ssl3_cbc_digest_record() to return a value due to the above
change.  It will also now set md_out_size=0 on failure.

This is based on part of BoringSSL's commit to fix malloc crashes:
https://boringssl.googlesource.com/boringssl/+/69a01608f33ab6fe2c3485d94aef1fe9eacf5364

ok miod@

1 files changed, 4 insertions, 3 deletions

diff --git a/src/lib/libssl/d1_srvr.c b/src/lib/libssl/d1_srvr.c
index dee182f541..057d92109c 100644
--- a/src/lib/libssl/d1_srvr.c
+++ b/src/lib/libssl/d1_srvr.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: d1_srvr.c,v 1.45 2014/12/14 15:30:50 jsing Exp $ */
+/* $OpenBSD: d1_srvr.c,v 1.46 2014/12/15 00:46:53 doug Exp $ */
 /*
  * DTLS implementation written by Nagendra Modadugu
  * (nagendra@cs.stanford.edu) for the OpenSSL project 2005.
@@ -1213,8 +1213,9 @@ dtls1_send_server_key_exchange(SSL *s)
                                 q = md_buf;
                                 j = 0;
                                 for (num = 2; num > 0; num--) {
-                                        EVP_DigestInit_ex(&md_ctx, (num == 2)
-                                            ? s->ctx->md5 : s->ctx->sha1, NULL);
+                                        if (!EVP_DigestInit_ex(&md_ctx, (num == 2)
+                                            ? s->ctx->md5 : s->ctx->sha1, NULL))
+                                                goto err;
                                         EVP_DigestUpdate(&md_ctx,
                                             &(s->s3->client_random[0]),
                                             SSL3_RANDOM_SIZE);