Ignore warning alert returns from servername callback in TLSv1.3

If a servername callback returns SSL_TLSEXT_ERR_ALERT_WARNING, this
results in a fatal error in TLSv1.3 since alert levels are implicit
in the alert type and neither close_notify nor user_canceled make
sense in this context. OpenSSL chose to ignore this, so we need to
follow suit.

Found via a broken servername callback in p5-IO-Socket-SSL which
returns a Boolean instead of SSL_TLSEXT_ERR_*. This happened to
have worked before TLSv1.3 since warning alerts are often ignored.

This "fixes" sni.t and sni-verify.t in p5-IO-Socket-SSL.

ok beck jsing

1 files changed, 7 insertions, 3 deletions

diff --git a/src/lib/libssl/tls13_legacy.c b/src/lib/libssl/tls13_legacy.c
index beb8952402..0360f8159c 100644
--- a/src/lib/libssl/tls13_legacy.c
+++ b/src/lib/libssl/tls13_legacy.c
@@ -1,4 +1,4 @@
-/*      $OpenBSD: tls13_legacy.c,v 1.26 2021/07/01 17:53:39 jsing Exp $ */
+/*      $OpenBSD: tls13_legacy.c,v 1.27 2021/08/30 16:50:23 tb Exp $ */
 /*
  * Copyright (c) 2018, 2019 Joel Sing <jsing@openbsd.org>
  *
@@ -515,8 +515,12 @@ tls13_legacy_servername_process(struct tls13_ctx *ctx, uint8_t *alert)
         ret = ssl_ctx->internal->tlsext_servername_callback(s, &legacy_alert,
             ssl_ctx->internal->tlsext_servername_arg);
 
-        if (ret == SSL_TLSEXT_ERR_ALERT_FATAL ||
-            ret == SSL_TLSEXT_ERR_ALERT_WARNING) {
+        /*
+         * Ignore SSL_TLSEXT_ERR_ALERT_WARNING returns to match OpenSSL's
+         * behavior: the only warning alerts in TLSv1.3 are close_notify and
+         * user_canceled, neither of which should be returned by the callback.
+         */
+        if (ret == SSL_TLSEXT_ERR_ALERT_FATAL) {
                 if (legacy_alert >= 0 && legacy_alert <= 255)
                         *alert = legacy_alert;
                 return 0;