Transport Layer Security (TLS) is an encryption and authentication protocol designed to secure Internet communications. TLS encrypts data sent over the Internet to ensure that cybercriminals will not be able to see private and sensitive information that is transmitted over the internet.
What is a TLS handshake?
A TLS handshake is the process that initiates secure communication using TLS between two machines, typically client and server. The TLS handshake involves multiple steps where the machines exchange information necessary to make further communication possible: acknowledge each other, verify the connection, set up cryptographic algorithms and establish session keys.
During the course of a TLS handshake, the client and server together will do the following:
- Specify which version of TLS (TLS 1.0, 1.2, 1.3, etc.) they will use
- Decide on which cipher suites (see below) they will use
- Authenticate the identity of the server via the server’s public key and the SSL certificate authority’s digital signature
- Generate session keys in order to use symmetric encryption after the handshake is complete
The TLS handshake may use slightly different steps, depending upon the kind of key exchange algorithm used (RSA or Diffie-Hellman) and the cipher suites supported by both sides.
How a TLS handshake works
In the TLS handshake, the client typically sends a request to establish a secure connection to the site’s server. The server then sends a public key (protocol) to your device and ensures to check that key against a pre-prepared list of protocols/certificates. The device then generates a key and uses the server’s key to encrypt it.
Here are the basic steps of a TLS 1.3 handshake:
- Client hello: The client kicks off the TLS handshake by sending a client hello message with the protocol version, the client random, and a list of cipher suites. The client hello also includes the parameters that will be used for calculating the premaster secret. What this means is that the client is assuming that it knows the server’s preferred key exchange method. This cuts down the overall length of the handshake.
- Server generates master secret: Because the server has received the client random and the client's parameters and cipher suites, the server can create the master secret.
- Server hello and "Finished": The server hello includes the server’s certificate, digital signature, server random, and chosen cipher suite. Because it already has the master secret, it also sends a "Finished" message.
- Final steps and client "Finished": Client verifies signature and certificate, generates master secret, and sends "Finished" message.
- Secure symmetric encryption achieved
TLS 1.3 handshake process
TLS vs. SSL handshakes
TLS has now replaced SSL as the security protocol for HTTP. As the protocols have evolved, certain differences have become apparent in the ways that SSL and TLS establish connections through the handshake process. In general, TLS (and particularly TLS 1.3) have worked to streamline the handshake process to increase the speed of connections. For example, it would take more time for the SSL handshake to make explicit connections via a port. Whereas TLS attempts to shortcut that process by facilitating implicit connections via protocol.
What does TLS Handshake Failed mean?
An error message that reads "TLS Handshake Failed" arises when there's an incompatibility during the protocol exchange phase, also known as the "handshake", between a client and a server using Transport Layer Security (TLS) for encryption. This discrepancy arises when the client and server are unable to find a common ground for the same TLS version, they both support. When this mutual understanding fails, the connection cannot be encrypted properly, triggering the "TLS Handshake Failed" error message. Identifying and addressing such protocol mismatches is a critical step towards ensuring secure, encrypted communication.
In the TLS handshake, the use of different asymmetric encryption algorithms can cause small changes in the steps. For example, Diffie-Hellman and RSA produce the same result in slightly different ways.
The Diffie-Hellman algorithm uses exponential calculations to arrive at the same premaster secret. Using these Diffie-Hellman parameters and the client and server randoms, it is possible for the client and server to calculate a shared, secret private key. The server and client each provide a parameter for the calculation, and when combined they result in a different calculation on each side, with results that are equal.
The challenge of expired certificates
When certificates are issued, they’re assigned an expiration date. If a certificate isn’t replaced before it expires, it can trigger a certificate-related outage of the system it supports. When that happens, the certificate is no longer available to establish a TLS connection through the handshake process.
The unplanned outage and the associated downtime will continue until a new certificate is issued and installed. Without the correct intelligence, such as knowing where each certificate is installed, which attributes it contains and who controls access to that system, certificate-related outages are notoriously difficult to diagnose.
Automating the entire machine identity life cycle—including the management of certificate requests, issuance, installation, renewals, and replacements—is important because it allows you to avoid error-prone, resource-intensive manual actions that may impact the success of your TLS connections. Automation can help you reduce the risk of vulnerable TLS handshakes as well as improving other vital security functions in your business.
Here are some of the benefits of automating your machine identity management:
- Avoid certificate-related outages by eliminating manual errors and automating the entire certificate life cycle to ensure machine identities are renewed before they expire. Information on certificate location and ownership quickly targets renewal requests with automated escalations as needed.
- Prevent breaches by automating the collection of risk intelligence required to quickly identify and respond to machine identity vulnerabilities, weaknesses, or security events. Automated policy-enforcement and life cycle management ensure unused or old keys and certificates are decommissioned.
- Accelerate incident response by automating the identification of impacted keys and certificates as well as the actions needed to remediate large groups of machine identities, so you can dramatically increase the speed of your response to large-scale security events.
- Streamline operations by automating routine administrative tasks to eliminate manual, error-prone processes and reduce the expertise and resources needed to manage the growing number of machine identities.
- Ensure compliance by automating policy enforcement to improve audit readiness, offering automated validation of TLS machine identity management, and generating scheduled or on-demand compliance reports.
Automating your management and security processes is the most effective way to build and maintain a successful TLS machine identity management program. The Venafi Control Plane for Machine Identities give you observability of machine identities across all environments so that you can verify that all your certificates have the proper attributes and use the most appropriate cipher suites for your business.