Make i2d_ASN1_OBJECT() return -1 on error

This is what the (not quite appropriately) referenced ASN1_item_i2d()
page documents for errors, matches what the RETURN VALUE section has
been documenting for ages, matches BoringSSL, it's the usal behavior
for i2d_*. It's also what OpenSSL (of course incorrectly) documents.

discussed with jsing

1 files changed, 2 insertions, 2 deletions

diff --git a/src/lib/libcrypto/asn1/a_object.c b/src/lib/libcrypto/asn1/a_object.c
index 00b5e7af92..aae1b8bbd7 100644
--- a/src/lib/libcrypto/asn1/a_object.c
+++ b/src/lib/libcrypto/asn1/a_object.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: a_object.c,v 1.52 2024/05/29 16:04:50 tb Exp $ */
+/* $OpenBSD: a_object.c,v 1.53 2024/05/29 16:10:41 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -619,7 +619,7 @@ i2d_ASN1_OBJECT(const ASN1_OBJECT *a, unsigned char **pp)
         int objsize;
 
         if (a == NULL || a->data == NULL)
-                return 0;
+                return -1;
 
         objsize = ASN1_object_size(0, a->length, V_ASN1_OBJECT);