Move point-on-curve check to set_affine_coordinates

Bad API design makes it possible to set an EC_KEY public key to
a point not on the curve. As a consequence, it was possible to
have bogus ECDSA signatures validated. In practice, all software
uses either EC_POINT_oct2point*() to unmarshal public keys or
issues a call to EC_KEY_check_key() after setting it. This way,
a point on curve check is performed and the problem is mitigated.

In OpenSSL commit 1e2012b7ff4a5f12273446b281775faa5c8a1858, Emilia
Kasper moved the point-on-curve check from EC_POINT_oct2point to
EC_POINT_set_affine_coordinates_*, which results in more checking.
In addition to this commit, we also check in the currently unused
codepath of a user set callback for setting compressed coordinates,
just in case this will be used at some point in the future.

The documentation of EC_KEY_check_key() is very vague on what it
checks and when checks are needed.  It could certainly be improved
a lot. It's also strange that EC_KEY_set_key() performs no checks,
while EC_KEY_set_public_key_affine_coordinates() implicitly calls
EC_KEY_check_key().

It's a mess.

Issue found and reported by Guido Vranken who also tested an earlier
version of this fix.

ok jsing

1 files changed, 15 insertions, 3 deletions

diff --git a/src/lib/libcrypto/ec/ec_lib.c b/src/lib/libcrypto/ec/ec_lib.c
index df9061627e..3442c7a324 100644
--- a/src/lib/libcrypto/ec/ec_lib.c
+++ b/src/lib/libcrypto/ec/ec_lib.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: ec_lib.c,v 1.32 2019/09/29 10:09:09 tb Exp $ */
+/* $OpenBSD: ec_lib.c,v 1.33 2020/12/04 08:55:30 tb Exp $ */
 /*
  * Originally written by Bodo Moeller for the OpenSSL project.
  */
@@ -964,7 +964,13 @@ EC_POINT_set_affine_coordinates_GFp(const EC_GROUP *group, EC_POINT *point,
                 ECerror(EC_R_INCOMPATIBLE_OBJECTS);
                 return 0;
         }
-        return group->meth->point_set_affine_coordinates(group, point, x, y, ctx);
+        if (!group->meth->point_set_affine_coordinates(group, point, x, y, ctx))
+                return 0;
+        if (EC_POINT_is_on_curve(group, point, ctx) <= 0) {
+                ECerror(EC_R_POINT_IS_NOT_ON_CURVE);
+                return 0;
+        }
+        return 1;
 }
 
 #ifndef OPENSSL_NO_EC2M
@@ -980,7 +986,13 @@ EC_POINT_set_affine_coordinates_GF2m(const EC_GROUP *group, EC_POINT *point,
                 ECerror(EC_R_INCOMPATIBLE_OBJECTS);
                 return 0;
         }
-        return group->meth->point_set_affine_coordinates(group, point, x, y, ctx);
+        if (!group->meth->point_set_affine_coordinates(group, point, x, y, ctx))
+                return 0;
+        if (EC_POINT_is_on_curve(group, point, ctx) <= 0) {
+                ECerror(EC_R_POINT_IS_NOT_ON_CURVE);
+                return 0;
+        }
+        return 1;
 }
 #endif