Let’s Encrypt is a practical way of obtaining certificates and implementing TLS encryption across a wide range of applications. Looking at the number of issued certificates, it is the largest Certificate Authority (CA) in the world. It is also widely used in automation scenarios due to its convenient renewal mechanism.
The recent expiration of a root certificate in the Let’s Encrypt default certification chain (September 30th, 2021) causes serious issues with older OpenSSL versions. However, patches for OpenSSL versions present in Centos 6/Oracle Linux 6/CloudLinux 6 are already available and should be deployed as soon as possible for all affected systems.
The root certificate that expired in the Let’s Encrypt certification chain (DST Root CA X3) will remain in the chain until 2024. Recent versions of OpenSSL correctly ignore the expired certificate and validate using alternate certificates present in the chain, but older versions of OpenSSL will fail the verification. This causes serious issues, as TLS connections will fail when they should not. In an unfortunate twist of fate, the “certbot” utility itself, by default, will fail to update the chain and renew the Let’s Encrypt certificates (which would resolve the issue).
This problem affects multiple Linux distributions and versions thereof, but the versions covered by our Extended Lifecycle Support already have patches for OpenSSL that address the problem. The way this is done is by enabling the flag X509_V_FLAG_TRUSTED_FIRST in OpenSSL by default, which basically means that validation should try “good” certificates first when they are available rather than failing when it finds a “bad” (read: expired) certificate. Fixing the issue this way will also further prevent similar events from happening in the future when other certificates expire, either in Let’s Encrypt or any other certification chain.
If you would like to address the issue in an alternative way that does not fix the underlying issue in OpenSSL but which simply bypasses the current problem, you could run “certbot” as follows:
certbot renew –dry-run –preferred-chain “ISRG Root X1”
Or, in /etc/letsencrypt/renewal/, add
preferred_chain = ISRG Root X1
This will allow certbot to fetch certificates ignoring the certification chain that contains the expired root certificate. But, again, this does not solve the issue of OpenSSL failing in a situation where it should proceed, so future events where a root or intermediate certificate are expired (either Let’s Encrypt-related or on any other certification chain) will continue to cause issues.