SSH keys are like plastic—we consume them constantly without really thinking about what happens to them. So, despite the incredible access that SSH keys grant, these valuable machine identities do not get the respect they deserve. In fact, systems administrators in your organization may be tempted to privilege convenience over security when they are using and storing their SSH keys. If they are easy for you to find, then chances are they are also easy for cybercriminals to find. And SSH keys are incredibly valuable to malicious actors, who can use them to pivot across your network until they find the data they’re looking for.
Do you know where all of your organization’s SSH keys are?
They can show up in the most unexpected places. Here’s a few places you can look to understand your relative exposure:
1. SSH Keys in AWS
We see regular company-wide efforts to ‘migrate to the cloud’. One of the first experiences often involves working with Amazon Web Services (AWS). Within the AWS compute platform, there are many services, one of the most popular being Elastic Compute Cloud (EC2). EC2 is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers. For example, EC2 can be used to create Linux, Windows, and many other types of Virtual Machines (VMs). In creating an AWS EC2 Virtual Machine Instance, a password is needed for the Administrator or Root account, which often starts as an SSH Key. Amazon generates an SSH Public/Private User Keypair and provides the user with the Private Key, retaining a copy of the Public Key. That Public SSH Key is used to generate the Windows Administrator password or used directly for logging into a UNIX/Linux Virtual Machine. Often times the private key is accidentally, insecurely and publicly stored in Amazon’s Simple Storage Service (S3). It’s not stealing when you’re giving it away!
2. In “Downloads” folders
While the use of a Private SSH Key is secure from a ‘data in motion’ perspective, it becomes a liability from a ‘data at rest’ one. I wonder how many Private SSH Keys are proliferating on users’ desktop environments, often sitting unprotected in Download or Desktop folders. These same SSH Keys can and are used for privileged access on all sorts of infrastructure and must be protected! Leaving them anywhere a cybercriminal might find them is risky behavior indeed.
3. In Continuous Integration (CI) & Continuous Delivery (CD) Pipelines & Release Trains
As developers contribute to remote code repositories (e.g. GitHub), data is often transferred using SSH keys. The developer generates the Public/Private User Keypair and then uploads the Public SSH Key to repository or administrative console. By doing so, a read-write link using SSH can be established with the remote repository instead of a read-only link which uses SSL/TLS/HTTPS. We have also seen some cases where SSH keys were discovered on GitHub and other repositories where it would be difficult to control their usage.
4. With Configuration Management Systems (e.g. Chef/Puppet/Ansible)
Often times in order to achieve scaled infrastructure management, commands need to be securely sent from an infrastructure management point- for example, a Chef Infrastructure Server, a Puppet Server, or an Ansible Control Node. Any of these management points need to be able to interact with the infrastructure being managed. There may be a few infrastructure management servers and hundreds or thousands of endpoints. In most instances, an SSH Public/Private User Keypair is generated on the management server. The private key is kept there and the public key installed on each endpoint. This allows the management server to control each endpoint to achieve distributed configuration management. Updating hundreds or thousands of servers becomes achievable through the power of SSH Keys. But if the proper security precautions aren’t followed, cybercriminals could also use those same SSH keys to update hundreds or thousands of servers. No one wants that.
5. In the Windows Registry
When using a Microsoft based workstation using Windows, the standard software used to connect to UNIX/Linux environments have historically been PuTTY. Like it’s command-line counterparts, PuTTY also supports SSH Keys, but unlike SSH Native tools, PuTTY SSH Host Keys are not stored on a filesystem but instead in the Windows Registry. When a user first connects to a UNIX/Linux server, they are presented with the host’s public key, often generated when the server was initialized. This host key is a server's unique ‘fingerprint’ that identifies it, similar to how an SSL/TLS certificate identifies a web site to a web browser. Since very few people maintain or even look at these fingerprints, the security on them is about as strong as a self-signed certificate.
As illustrated in the above examples, while it’s easy to understand how SSH keys proliferate ubiquitously across the infrastructure, it’s much harder to keep track of them. When InfoSec teams are instructed to manage the SSH footprint, they are often initially in a reactive mode and need to quickly understand the latent risk the organization is exposed to and take measures to implement effective controls.
Do you know where all of your organization’s SSH keys are? And do you know who’s using them?