Let's Encrypt has notified subscribers that on 28 January 2022 it will revoke certain certificates issued in the last 90 days, according to a staff member’s response on a Let’s Encrypt forum. Let’s Encrypt, a non-profit certificate authority run by Internet Security Research Group (ISRG), said that not everyone will be necessarily notified, and they are working to provide a way for subscribers to see if they are affected. This is another example of why PKI teams may be inadvertently placed in a situation where they need to quickly and automatically replace any individual Certificate Authority (CA), certificate or groups of certificates. Unfortunately, the vast majority of organizations don’t have the visibility or automation required to do this.
The revocation only affects certificates issued and validated with the TLS-ALPN-01 “challenge,” according to Let’s Encrypt, which it describes as a way for its servers to validate control of the domain names in that certificate. “When you get a certificate from Let’s Encrypt, our servers validate that you control the domain names in that certificate using ‘challenges,’ as defined by the ACME standard,” according to Let’s Encrypt.
The news was first reported by Bleeping Computer.
The Let’s Encrypt staffer goes on to say that “all successful issuance in last 90 days with the TLS-ALPN-01 challenge are affected and will be revoked. If you only use that challenge, you should force renew all of your certificates. If you only use that challenge for some domains but are having trouble determining which ones based on the account, it is safe to force renew all your certificates.”
This follows an incident in February 2020 when Let’s Encrypt revoked millions of certificates in response to a bug in its Certificate Authority Authorization (CAA) code.
Let’s Encrypt popularity explodes
Let’s Encrypt has become extremely popular with developers. That popularity makes an incident like this significant.
“Let’s Encrypt has boomed in popularity with developers over the last few years, as it gives developers a quick, free and easy way to issue TLS machine identities for all manner of critical web services—from websites to customer applications,” says Kevin Bocek, VP Security and Threat Intelligence at Venafi.
Bocek points to a recent crawler report from Venafi and security expert Scott Helme showing that Let’s Encrypt now has millions of active certificates in use, with 28% of the top 1 million sites making use of it.
“This means that when Let’s Encrypt suddenly has to revoke millions of certificates—as is the case right now—it can create major upheaval, putting critical services at risk of outage, with organizations having to quickly find and reissue potentially tens of thousands of machine identities within just two or three days,” Bocek says.
Doing this manually is almost impossible and highly prone to potentially costly errors, according to Bocek. Add to that the fact that businesses could have tens of thousands of machine identities that they aren’t even aware of.
“To protect against events such as these, which are becoming increasingly common, security teams should be automating machine identity management. By doing so, they can avoid manual rotation, replacement and revocation of all machines,” Bocek says.
Customers of Venafi’s machine identity management platform, regardless of what datacenter product or cloud product they are using, are protected by the agility Venafi offers, says Jing Xie, Ecosystem Manager of Business Development at Venafi.
“The Venafi Control Plane offers complete visibility and inventory of all Let’s Encrypt issued certificates. With a few clicks, it helps replace all affected certificates with newly issued and secure ones without service disruption,” Xie says.
This is not the first or last time that an incident will cause domain users to scramble to recover. Venafi has outlined the long history of CA errors that impacted the security of machine identities such as TLS certificates.