Bring back *_client_method() structs

The method unification broke an API promise of SSL_is_server(). According
to the documentation, calling SSL_is_server() on SSL objects constructed
from generic and server methods would result in 1 even before any call to
SSL_set_accept_state(). This means the information needs to be available
when SSL_new() is called, so must come from the method itself.

Prior to the method unification, s->server would be set to 0 or 1 in
SSL_new() depending on whether the accept method was undefined or not.
Instead, introduce a flag to the internal structs to distinguish client
methods from server and generic methods and copy that flag to s->server in
SSL_new().

This problem was reported to otto due to breakage of DoH in net/dnsdist.
The reason for this is that www/h2o relies on SSL_is_server() to decide
whether to call SSL_accept() or SSL_connect(). Thus, the h2o server would
end up responding to a ClientHello with another ClientHello, which results
in a handshake failure. The bandaid applied to www/h2o can be removed once
this fix has made it into snaps. No other breakage is known.

This commit brings back only about half of the duplication removed in the
method unification, so is preferable to a full revert.

ok jsing

1 files changed, 2 insertions, 1 deletions

diff --git a/src/lib/libssl/ssl_locl.h b/src/lib/libssl/ssl_locl.h
index 46a1ad4884..19d883e3b0 100644
--- a/src/lib/libssl/ssl_locl.h
+++ b/src/lib/libssl/ssl_locl.h
@@ -1,4 +1,4 @@
-/* $OpenBSD: ssl_locl.h,v 1.307 2020/11/11 18:14:12 jsing Exp $ */
+/* $OpenBSD: ssl_locl.h,v 1.308 2020/12/01 07:46:02 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -359,6 +359,7 @@ __BEGIN_HIDDEN_DECLS
 
 typedef struct ssl_method_internal_st {
         int dtls;
+        int server;
         int version;
 
         uint16_t min_version;