Let me start by saying that Imperva themselves have released a detailed and pretty transparent statement regarding their breach, its causes and their remedial actions. In a nutshell, it appears that this is the case of an API key that was left on a forgotten server, stolen and used in an attack.
While there are obviously a few gaps in the narrative (and a LOT of spin, plus a bit of polish) it covers very well what happened and what steps Imperva took, to the extent I am not going to be offering anything new that isn't contained in that statement. Instead, I will look at the sequence of events, the lessons to be learned, and any actions that could have prevented this (many of which the company themselves state they have now taken).
So, from the top then
Amazon Web Services is the cloud offering from Amazon; AWS has a powerful system for controlling the security of components, which it calls Identity & Access Management. Like its main rival Azure, AWS supports a variety of two factor methods for users in its environment, and federation to external services (such as Active Directory, the Microsoft identity solution) so that customers using their own user authentication systems can have a single point of administration for user identities.
Unlike Azure however, IAM has TWO separate authentication systems. The first is for the web portal and supports 2FA. However in the modern world, many users of cloud services wish to automate routine actions, either by writing their own programs, or by running templated solutions such as Terraform—this is often called "Infrastructure as Code", "Configuration Orchestration" or a dozen other titles. But the principle remains the same—you run some software on your own machine and it sends commands to your cloud account on your behalf, allowing you to automate routine tasks, generate reports from your deployed systems and more.
Tale of 3 Clouds eBook: How Venafi Creates Digital Transformation
This second system doesn't use your usual IAM password, nor can it ask for your two factor codes. Instead, you go into a section of IAM management, and request an "Access Key." This is a pair of text strings (one is the key ID, and the other the key itself) which can be easily cut and pasted into an application or PowerShell script, and, when used with the AWS API, allow the program to do everything the user themselves could do within AWS.
Access Keys are core to the mishap at Imperva; they are obviously both powerful and extremely sensitive—you don't want to let them out of your control (and Access Keys are one of the main things attackers search for on Github because they are often accidentally uploaded there). AWS also recommends you replace them regularly, and to that end, AWS will list the age of the keys in the IAM section of their portal. Imperva generated and copied into a script such a set of Access Keys at some point during or prior to 2017 (we presume this, because the instance and snapshot date from September 2017) and remained on the instance until the date of the compromise in October 2018. Good key management practice should have disabled or replaced that key well within that time, but the key, the snapshot, and the instance were all "stale." In other words, they were used during a cloud migration process and then forgotten.
"Access Keys are core to the mishap at Imperva"
There are some perhaps obvious takeaways from this. Imperva were clearly unaware of a database snapshot, a running instance burning compute charges, and Access Keys that not only had not been rotated, but which were used in October 2018 (key last use date is another field available for key management). The Access Keys were also possibly used from a location other than the instance—we don't know of course. If the script used the security feature of the AWS API (to store the Access Keys encrypted) they may well have HAD to use them from the instance, for lack of a reasonable alternative.
This, in itself, looks bad. But there is more. The instance with these sensitive keys was not restricted to a private (VPN) link from Imperva, it was internet-facing; further, the source IPs for this internet access were not limited to Imperva's office IP addresses, so an attack could (and did!) connect to and compromise this instance, and subsequently, copy the database snapshot that the Access Keys on that instance had access to. As we don't know how the instance was compromised, nor exactly how the extraction was performed, there isn't much more we can do here.
So moving away from the technical details of the attack, we should look at WHAT was compromised here too.
What was compromised?
The first element is account credentials. Hashed and Salted passwords are often considered pretty strong, but we have no idea what hashing method was in use, and the attackers HAVE had over a year to work on them. With that kind of timeframe, at least some will have been cracked, and of course the sort of password most likely to be cracked is also the sort of password often re-used on other sites.
The second element is API Keys. We don't actually know which direction that runs, but essentially, we are talking Access Keys as detailed above. There are questions that remain unanswered. Are they for customers to access the WAF platform? For the WAF platform to access customer resources? Maybe to write logs? We don't know. But we do know they would need to be changed due to the compromise.
"...most importantly, the TLS keys"
And finally, and most importantly, the TLS keys. As a WAF, CDN, DDoS shield and so forth, the Imperva platform would need to "reverse proxy" the traffic—that is, inbound web connections would be decrypted there, checked (and if possible, satisfied from the cached files) and only clean, unique traffic forwarded on to the real web servers. While it is now the case that modern CAs will not issue certificates for multiple years, it is also true that the private keys from certificates renewed on a yearly basis (shorter for some; LetsEncrypt, for example, issues three-month certificates) tend to be re-used indefinitely. So while the certificates may have long since aged out, in many cases those private keys will still be in use, and of course that's the part that must be kept secret at all cost. The danger isn't so much decryption of intercepted traffic (modern TLS almost invariably uses a property called "Perfect Forward Secrecy" which means the private key isn't sufficient to read traffic on its own) as someone successfully pretending to be the genuine website, because they have both the certificate (which can be pulled from the site by design) and the private key (which by design can't be).
All affected customers are now aware of the issue (and presumably have replaced their keys, changed their passwords, and so forth as a precaution) but of course as with Imperva themselves, good key and password lifecycle management should mean customers who have taken out of service the credentials in the interval between snapshot and compromise, will be more secure than those who have done so a year after the fact.
Why Do You Need a Control Plane for Machine Identities?
Related posts