On February, 29 Let’s Encrypt announced a bug in its Certificate Authority Authorization (CAA) code. Basically, the bug would allow certificates to be issued for a certain period of time, even if a CAA record existed for that domain.
Ars Technica explained the bug as follows: “The bug LE discovered is that, rather than checking each domain name separately for valid CAA records authorizing that domain to be renewed by that server, Boulder would check a single one of the domains on that server n times (where n is the number of LE-serviced domains on that server). Let's Encrypt typically considers domain validation results good for 30 days from the time of validation—but CAA records specifically must be checked no more than eight hours prior to certificate issuance.”
CAA function compromised
Exactly what is the CAA functionality that has been compromised? As we noted in a previous blog, “Certification Authority Authorization allows domain owners to specify in their Domain Name Servers (DNS) which CAs are authorized to issue certificates for that domain. They can do this by creating a CAA record with an issuer domain name, an identifier which every CA includes in its certification practice statement (CPS) of how it issues and manages public key certificates. Domain owners can then add those records to their DNS or DNS Security (DNSSec).”
When domain owners can no longer control which certificates shouldn’t be issued for their domain, they may be exposed to fraudulent activity. Pratik Savla, senior security engineer at Venafi notes,
"Because CAA is mandatory, the impact of this bug is huge given the high number of Lets’ Encrypt issued certificates. This bug has uncovered a window of opportunity in which a certificate may be issued even if a CAA record for a domain prohibits it. This could potentially lead to wide-spread abuse so it’s understandable the stance that Lets Encrypt is taking here to contain this damage.”
Let’s Encrypt has identified around 3 million certificates that may suffer from this vulnerability. And because of the potential for abuse of these certificates they have announced that they will revoke those certificates on Wednesday, March 4. While they did reach out to notify impacted certificate owners, apparently, these domain owners were only given 24 hours to manually replace the certificates before they would be revoked by Let’s Encrypt.
While revoking impacted certificates is the best way to ensure that they will not be misused, this action may place an unreasonable burden on some certificate owners. Kevin Bocek, vice president of security strategy and threat intelligence at Venafi discusses this challenge:
Revoking over 3 million TLS certificates in the next 24 hours
“Unfortunately because of a bug, Let’s Encrypt is revoking over 3 million TLS certificates in the next 24 hours. These certificates serve as machine identities, authenticating and authorizing the flow of sensitive data between machines. As a result, millions of machines could be unavailable or be untrusted tomorrow, potentially causing harm to the companies that rely on them.
This is a great example of why security teams need to provide the agility to quickly and automatically replace any individual CA, certificate or groups of certificates. Unfortunately, the vast majority of organizations don’t have the visibility or automation required to do this quickly and painlessly.”
Ultimately, this is not the first or last time that a Certificate Authority (CA) error will cause domain users to scramble to recover. After the recent Symantec distrust event, we outlined the long history of CA errors that impacted the security of machine identities such as TLS certificates.
"it’s not the first time that Let’s Encrypt has found issues with its Boulder code base”
“Even though this is probably the first time that Let’s Encrypt has been forced to revoke such a huge number of certificates, it’s not the first time that Let’s Encrypt has found issues with its Boulder code base,” notes Pratik Savla. “Other issues with its code base have been discovered in the past and unfortunately that resulted in CAA rules being ignored and certificates mis-issued in the past. Hopefully, this incident will push CAs to review and tighten up their testing process covering CAA checks to prevent any kind of incorrect behavior.”
How quickly can you find and replace certificates that are invalidated by compromise or CA error?