Ensure X509V3_EXT_print() only returns 0 and 1

In a rare mistake by schwarze, X509V3_EXT_print() is documented to return
0 and 1. This is also what most internal callers expect.

However, if either X509V3_EXT_DUMP_UNKNOWN or X509V3_EXT_PARSE_UNKNOWN is
set, the extension has an unknown NID or on failure to deserialize the
extension value, the return values of BIO_dump_indent() (which is number
of bytes written or -1 on error) and ASN1_parse_dump() (which is 0, 1, or
2 on EOC) are propagated.

Follow what OpenSSL did and translate to Boolean returns. Error indicators
are rather useless here since most errors are ignored anyway. Most callers
do if (!X509V3_EXT_print(...)) but they also pass a zero flag.

Reported by N. Dossche

ok kenjiro

1 files changed, 3 insertions, 3 deletions

diff --git a/src/lib/libcrypto/x509/x509_prn.c b/src/lib/libcrypto/x509/x509_prn.c
index 23c649a7b9..327bc33772 100644
--- a/src/lib/libcrypto/x509/x509_prn.c
+++ b/src/lib/libcrypto/x509/x509_prn.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: x509_prn.c,v 1.7 2025/06/02 12:18:22 jsg Exp $ */
+/* $OpenBSD: x509_prn.c,v 1.8 2026/05/16 06:27:05 tb Exp $ */
 /* Written by Dr Stephen N Henson (steve@openssl.org) for the OpenSSL
  * project 1999.
  */
@@ -207,10 +207,10 @@ unknown_ext_print(BIO *out, X509_EXTENSION *ext, unsigned long flag,
                 return 1;
         case X509V3_EXT_PARSE_UNKNOWN:
                 return ASN1_parse_dump(out,
-                    ext->value->data, ext->value->length, indent, -1);
+                    ext->value->data, ext->value->length, indent, -1) > 0;
         case X509V3_EXT_DUMP_UNKNOWN:
                 return BIO_dump_indent(out, (char *)ext->value->data,
-                    ext->value->length, indent);
+                    ext->value->length, indent) > 0;
         default:
                 return 1;
         }