Validating OpenSSL connections when using transparent negotiation.

Question

When using OpenSSL with non blocking sockets its convenient to use the transparent negotiation mode where the negotiation is initiated by calling SSL_write on a not-yet-securely-connected SSL context, and then handling the resulting error (usually SSL_WANT_READ) to read a handshake packet from the output BIO and send it over the transport.

However, without the explicit (blocking) call to SSL_do_handshake its not clear to me when to try to validate the certificate.

Do I just have to do a certificate check as soon as SSL_write returns success for the first time? Or is there a better signal?

Answer 1

Before you initiate the negotiation, you should use SSL_set_verify() to set the verification mode and optionally a verification callback. Handle any additional application-specific validation you need in the callback.

Then, if SSL_write() and/or SSL_read() return successfully, you know that the certificate has been validated.