TLS/SSL: How Secure Connections Actually Work

TLS is one of those protocols you interact with hundreds of times a day and rarely think about. Every HTTPS connection, every API call over the internet, VPNs using 'SSL' — all of it runs on TLS. The handshake that happens before any of that data flows is the part most people skip over. This post goes through it in full, including the parts that don't usually get explained: why the keys are derived the way they are, what each handshake message actually contains, and how TLS 1.3 simplified the process without sacrificing security. The troubleshooting section covers the errors you'll encounter when certificates, versions, or configuration go wrong.

SSL and TLS

SSL (Secure Sockets Layer) and TLS (Transport Layer Security) are often used interchangeably in conversation, but they're not the same. TLS is the successor to SSL. SSL is deprecated and should not be used. When someone says "SSL certificate," they usually mean a TLS certificate. When a browser says a site is secured with SSL, it means TLS. The terminology has lagged behind the protocol.

SSL was retired for good reason. Multiple critical weaknesses were discovered over its lifetime, and modern systems should use TLS instead. A separate but famous TLS-era implementation bug is Heartbleed, a flaw in OpenSSL's implementation of the TLS/DTLS heartbeat extension. The heartbeat mechanism is a keepalive: one side sends a heartbeat message containing a payload and a declared payload length. The other side should respond with the same payload. The vulnerability: if the declared length in the request is larger than the actual payload, the responding server reads beyond the end of the request payload in memory and includes that extra memory in its response. An attacker could send a 1-byte payload while claiming a length of 64,000 bytes. The server would echo back the 1-byte payload plus up to 64KB of memory content, potentially including private keys, session tokens, user credentials, and other sensitive data sitting in the server's process memory. No authentication required. Repeatable indefinitely.

This is a memory disclosure vulnerability, not a decryption attack, but the damage can be just as serious. If private keys were exposed, attackers could impersonate services and may have been able to decrypt previously captured traffic when sessions did not use forward secrecy. Leaked session keys or credentials could also expose active sessions directly.

TLS versions have their own distinctions, covered at the end of this post.

Where TLS Sits in the Stack

TLS operates between Layer 4 (Transport) and Layer 7 (Application). It encrypts application data — the HTTP request, the API payload, whatever the application is sending — while leaving the TCP and IP headers unencrypted. Those headers are still needed for routing and delivery.

Packet structure:

[IP Header | TCP Header | TLS Encrypted Application Data]

The TCP handshake completes first. The TLS handshake runs on top of the established TCP connection. Only after both handshakes complete does application data start flowing.

As a VPN mechanism, SSL/TLS is most commonly used for remote-access or client-to-site VPNs. Site-to-site VPNs are usually IPSec, although some products can build TLS-based site-to-site tunnels. One advantage over IPSec client VPNs: SSL VPN connections often don't require a dedicated agent installed on the client. A browser-based portal can provide access without software installation, which is useful for BYOD and contractor scenarios.

Clients, Servers, and Certificate Authorities

Three parties are involved in TLS:

Client: The entity that initiates the TLS handshake. Typically a browser, a mobile app, or any software making a secure connection. In most deployments, the client doesn't need to authenticate itself — it's the server whose identity needs to be verified.

Server: The entity receiving the connection. Always needs to authenticate itself via a certificate. Without this, a client has no way to confirm it's talking to the right server and not an attacker intercepting the connection.

Certificate Authority (CA): A trusted third party that issues certificates. Both client and server implicitly trust the CA. The CA's role is to verify that whoever is requesting a certificate for example.com actually controls example.com, and then sign the certificate with the CA's private key.

How certificate trust works:

A certificate contains the server's public key and identifying information (domain name, organization, validity period). The CA generates a signature over the certificate contents using the CA's private key. This signature is attached to the certificate.

When a client receives a certificate, it already has the public keys of trusted CAs pre-installed (browsers and operating systems ship with a trust store containing hundreds of CA public keys). The client uses the appropriate CA's public key to verify the signature over the certificate. It also checks that the certificate contents have not been altered since the CA signed them. If verification succeeds and the certificate is otherwise valid, the server identity information in the certificate is trustworthy.

TLS 1.2 Handshake

The TLS 1.2 handshake uses RSA for key exchange in this walkthrough (other mechanisms like ECDHE are also possible in TLS 1.2; RSA is used here for clarity). The server starts with its certificate, public key, and private key already configured.

The handshake completes in two round trips after the TCP connection is established.

Round Trip 1

Client Hello (Client → Server)

The opening message. Contains:

• Version: The highest TLS/SSL version the client supports. This is the client's maximum capability, not necessarily what will be used.
• Random number: 32 bytes. The first 4 bytes encode a timestamp, which ensures that two Client Hello messages sent more than a few seconds apart aren't carrying identical random values. This random number is used later in key derivation.
• Session ID: All zeros in an initial connection. If the client is resuming a previous session, this field contains the session identifier from that prior session, allowing the server to skip the full handshake.
• Cipher suites: A list of cryptographic algorithm combinations the client supports, ordered by preference. Each entry specifies a key exchange algorithm, an authentication algorithm, a bulk encryption algorithm, and a hash function. The server will select one from this list.
• Extensions: Optional additional data. Common extensions include Server Name Indication (SNI), which tells the server which hostname the client is trying to reach before the certificate is sent — critical on servers hosting multiple domains.

Server Hello (Server → Client)

The server's response. Same fields as the Client Hello, with differences:

• Version: The TLS version selected by the server from the versions both sides support.
• Random number: The server's own 32-byte random value. Also used in key derivation. Between the client and server randoms, both sides now have two unique random values that will seed the key generation process.
• Session ID: Randomly generated by the server. Acts as a label for this session's keys. If the same session is resumed later, this ID allows the server to retrieve the associated keying material.
• Cipher suite: The server picks one entry from the client's list. This is the algorithm set both sides will use for the rest of the session.
• Extensions: The server provides any required responses to client extensions.

After the Client Hello and Server Hello exchange, both sides share: the selected TLS version, both random values, the session ID, and the agreed cipher suite.

Certificate (Server → Client)

The server sends its full certificate chain — its own certificate plus any intermediate CA certificates required to trace trust back to a root CA. The client receives the server's certificate and extracts its public key.

The client now validates the certificate:

1. Locates the appropriate CA public key in its trust store.
2. Uses that public key to verify the CA's signature over the certificate.
3. Confirms the certificate contents have not been altered since it was signed.
4. Checks the certificate's validity period, the domain name against the hostname being connected to, and that the certificate hasn't been revoked.

Server Hello Done (Server → Client)

An empty message indicating the server has finished its part of Round Trip 1 and is waiting for the client's response.

Round Trip 2

Client Key Exchange (Client → Server)

This is where the actual keying material is established and where the server proves it holds the private key corresponding to the certificate it sent.

The client generates a Pre-Master Secret: a 48-byte value where the first 2 bytes are the client version from the Client Hello and the remaining 46 bytes are randomly generated.

The client encrypts this Pre-Master Secret using the server's public key (extracted from the certificate) and sends it. Only the server — which holds the corresponding private key — can decrypt it. This is the proof: if the server can use the Pre-Master Secret to generate matching session keys (verified in the Finished messages), the server demonstrably holds the private key for that certificate.

Session Key Derivation

Both client and server now independently derive the same set of keys without transmitting them:

Step 1 — Master Secret:

Master Secret = PRF(Pre-Master Secret, "master secret", ClientRandom + ServerRandom)

The PRF (Pseudo-Random Function) takes the Pre-Master Secret, the literal string "master secret", and the concatenation of both random values, and produces a 48-byte Master Secret. Both sides compute this independently using the same inputs and arrive at the same value.

Step 2 — Session Keys:

Key Material = PRF(Master Secret, "key expansion", ServerRandom + ClientRandom)

Note the random values are in reverse order here. The PRF generates enough key material to produce at minimum four session keys:

• Client encryption key: The symmetric key the client uses to encrypt traffic it sends.
• Client HMAC key: The key the client uses to generate integrity check values for outgoing traffic.
• Server encryption key: The symmetric key the server uses to encrypt traffic it sends.
• Server HMAC key: The key the server uses to generate integrity check values for its outgoing traffic.

Creating separate keys for each direction means two independent encrypted tunnels — one from client to server, one from server to client. The server uses the client's keys to decrypt client-to-server traffic, and the client uses the server's keys to decrypt server-to-client traffic.

How HMAC works:

The sender combines the message with the shared HMAC key using a cryptographic hash function (SHA-256, for example). The resulting hash is the HMAC, sent alongside the message. The receiver recomputes the HMAC using the same key and function on the received message. If the computed HMAC matches the received one, the message is authentic and was not modified in transit. An attacker who can intercept and modify traffic but doesn't have the HMAC key cannot produce a valid HMAC for a tampered message.

Change Cipher Spec (Client → Server)

Not technically a handshake message — a separate record type. Tells the server that the client has calculated all the keys and is switching to the cipher suite agreed upon in the hellos. Everything after this point from the client is encrypted.

Finished (Client → Server)

The first encrypted message. This is the verification that the entire handshake went correctly and that both sides derived the same keys.

The client:

1. Takes a hash of every handshake message exchanged so far (all except Change Cipher Spec).
2. Combines that hash with the Master Secret and the literal string "client finished" and runs it through the PRF, producing the verification data.
3. Encrypts the verification data using the client session keys and sends it.

The server decrypts the Finished message using the client session keys it derived independently. It then computes the same hash of the same handshake messages and runs the same PRF. If the verification data matches, the server confirms: the client has the correct keys, the handshake wasn't tampered with, and both sides computed identical keying material.

Change Cipher Spec (Server → Client)

The server signals it's also ready to switch to the negotiated cipher spec.

Finished (Server → Client)

Same process as the client's Finished, but encrypted with the server's session keys. The handshake hash this time also includes the client's Finished message. The client decrypts it and runs the same verification. If it checks out, the handshake is complete.

Encrypted application data begins flowing.

TLS 1.2 Round Trips:

Round Trip 1:
  Client → Server: ClientHello
  Server → Client: ServerHello, Certificate, ServerHelloDone

Round Trip 2:
  Client → Server: ClientKeyExchange, ChangeCipherSpec, Finished
  Server → Client: ChangeCipherSpec, Finished

→ Encrypted communication begins

TLS 1.3 Handshake

TLS 1.3 redesigns the handshake for speed and removes a significant amount of legacy complexity. It uses ephemeral Diffie-Hellman-style key exchange, usually ECDHE in normal browser traffic. Static RSA key exchange is removed. Handshake messages after the Server Hello are encrypted. The whole thing completes in one round trip.

Round Trip 1

Client Hello (Client → Server)

Same general structure as TLS 1.2, with several important differences:

• Version: In TLS 1.3, the legacy version field is set to TLS 1.2 for compatibility. Actual version negotiation happens through the supported_versions extension.
• Random number: 32 bytes, but the timestamp encoding from TLS 1.2 is removed.
• Cipher suites: Restricted to AEAD-based ciphers only — AES-GCM and ChaCha20-Poly1305. CBC-based cipher suites, which have been subject to padding oracle attacks, are entirely removed.
• Key Share extension: The client includes its ECDHE public key. This allows the server to compute the shared secret immediately, without waiting for a separate key exchange round trip. This is what makes the one-round-trip handshake possible.
• Pre-Shared Key (optional): If the client is resuming a previous session, it includes an identifier for the prior session's keying material.
• Extensions: SNI, ALPN, and others as needed.

Server Hello (Server → Client)

• Selects the cipher suite and TLS version.
• Includes its own ECDHE public key in the Key Share extension.

At this point — after just Client Hello and Server Hello — both sides have the information needed to compute the shared ECDHE secret. Key derivation happens immediately. Everything the server sends after this is encrypted.

Encrypted Extensions (Server → Client)

A new message type in TLS 1.3, absent from TLS 1.2. Sent encrypted immediately after key derivation. Contains additional server parameters and extensions that would have traveled in the clear in TLS 1.2 (ALPN negotiation results, session resumption info). Moving these into an encrypted message hides some negotiation metadata from passive observers. Standard SNI is still visible unless Encrypted Client Hello is used.

Certificate (Server → Client)

Same certificate chain as TLS 1.2, but now sent encrypted. In TLS 1.2, the certificate traveled in the clear. In TLS 1.3, an observer watching the connection cannot see which certificate the server presented.

Certificate Verify (Server → Client)

A new message in TLS 1.3 with no equivalent in TLS 1.2's basic RSA flow. The server signs a hash of all previous handshake messages using its private key and sends the signature.

This proves two things simultaneously: the server possesses the private key matching the certificate it presented, and the entire handshake up to this point hasn't been tampered with. In TLS 1.2, server proof of private key possession was implicit — if the server could derive matching session keys, it must have decrypted the pre-master secret. TLS 1.3 makes this proof explicit.

Finished (Server → Client)

The server sends a Finished message encrypted with the derived session key. Same verification role as in TLS 1.2 — proves the server has the correct keys.

The client now verifies the server's Certificate, the Certificate Verify signature, and the Finished message. If all checks pass, the client sends its own Finished message.

Finished (Client → Server)

Encrypted with the session key. Once the server verifies this, both sides can exchange application data.

TLS 1.3 Round Trip:

Round Trip 1:
  Client → Server: ClientHello (includes KeyShare)
  Server → Client: ServerHello (includes KeyShare), EncryptedExtensions,
                   Certificate, CertificateVerify, Finished

→ Client → Server: Finished
→ Encrypted communication begins

No second round trip. The server's portion of Round Trip 1 carries everything needed to complete authentication and key establishment.

0-RTT (Zero Round Trip Time): When a client is reconnecting to a server it's talked to before, TLS 1.3 supports sending application data in the very first message alongside the Client Hello, before the handshake completes. This pre-shared key-based mechanism eliminates even the single handshake round trip for resumed sessions. 0-RTT has specific replay attack considerations and is appropriate for idempotent requests.

TLS 1.2 vs TLS 1.3: Key Differences

1. Handshake round trips: TLS 1.2 requires two round trips after TCP. TLS 1.3 requires one.

2. Key exchange: TLS 1.2 supports multiple key exchange mechanisms, including static RSA and forward-secret DHE/ECDHE cipher suites. TLS 1.3 removes static RSA and static DH key exchange. Normal certificate-based TLS 1.3 handshakes use forward-secret key exchange, though PSK-only resumption is a special case where forward secrecy depends on the selected PSK mode.

3. Change Cipher Spec messages: TLS 1.2 has explicit Change Cipher Spec messages signaling the transition to encryption. TLS 1.3 encrypts immediately after the Server Hello, no transition message needed.

4. Key derivation: TLS 1.2 uses a multi-step derivation process (pre-master secret → master secret → session keys via PRF). TLS 1.3 replaces this with a single unified key schedule using HKDF (HMAC-based Key Derivation Function), simpler and less prone to misconfiguration.

5. Encrypted handshake: In TLS 1.2, the certificate and most negotiation metadata travel in plaintext. In TLS 1.3, everything after Server Hello is encrypted, including the server's certificate.

6. Cipher suites: TLS 1.2 supports a wide range of cipher suites including older, vulnerable ones (RC4, export ciphers, CBC-mode ciphers susceptible to BEAST and POODLE). TLS 1.3 permits only AEAD ciphers (AES-GCM, AES-CCM, ChaCha20-Poly1305), removing the entire category of padding oracle vulnerabilities.

TLS Version Comparison

TLS 1.1 is deprecated and should not be used. Forward secrecy in TLS 1.1 and TLS 1.2 depends on the negotiated cipher suite; DHE/ECDHE suites can provide it, while static RSA does not. TLS 1.2 remains acceptable when properly configured with modern cipher suites and forward-secret key exchange. TLS 1.3 removes static RSA and legacy cipher-suite complexity, permits only AEAD cipher suites, and encrypts most of the handshake after Server Hello.

TLS 1.0 and SSL in all versions should be disabled on any modern system. TLS 1.2 remains widely deployed and is acceptable when properly configured (PFS cipher suites enforced, CBC disabled where possible). TLS 1.3 is the current standard.

Troubleshooting TLS/SSL

Certificate Not Trusted

The browser shows "Your connection is not private" or an equivalent warning. The certificate exists but isn't accepted.

Common causes:

Self-signed certificate: The certificate was signed by the server itself rather than a trusted CA. No browser trust store contains the signing key, so the certificate can't be validated. Acceptable in internal development environments with manual trust configuration. Not acceptable for anything user-facing.

CA not recognized: The CA that issued the certificate isn't in the browser's trust store. Symantec's CA certificates were distrusted and removed from major browsers due to certificate issuance violations — any certificate signed by Symantec's now-distrusted roots produces this error regardless of expiry or domain match. Check whether the issuing CA is in current browser trust stores.

Missing intermediate certificate: The server's certificate chain is incomplete. The server should send its certificate plus all intermediate CA certificates between it and the root. If an intermediate is missing, the browser can't trace the chain to a trusted root.

Wrong TLS Version

The client and server can't agree on a mutually supported version. Some servers are configured to require TLS 1.2 as a minimum. Clients running outdated browsers or operating systems that don't support TLS 1.2 receive a handshake failure.

The error "a fatal error occurred while creating a TLS client credential" or similar indicates a version negotiation failure. Update the browser and operating system to support current TLS versions. On the server side, ensure TLS 1.2 and 1.3 are enabled and older versions (TLS 1.0, SSL) are disabled.

Expired Certificate

A certificate's validity window has passed. Browsers reject expired certificates because the CA's signature only vouches for the certificate contents at the time of issuance — an expired certificate provides no current assurance.

Web administrators need a certificate renewal process that accounts for all domains and subdomains in use. Expired subdomains are a common oversight that breaks services for users who don't understand the error.

A less obvious cause: the client's system clock is wrong. If the client's clock shows a date outside the certificate's validity window, the browser reports the certificate as expired even when it isn't. Resetting the clock to the correct time resolves it. This is fundamentally an NTP problem — a system with a misconfigured or unreachable time source drifts and eventually causes certificate validation failures.

Common Name Mismatch

The hostname in the browser's address bar doesn't match what's listed on the certificate.

This happens when:

• A user types a subdomain the certificate doesn't cover (e.g., typing www.example.com when the certificate only covers example.com).
• The certificate was issued for the wrong domain or contains a spelling error.
• The server is hosting multiple websites and presents the certificate for the wrong domain (see SNI below).
• The certificate doesn't include all legitimate variations of the domain.

Certificates include a SAN (Subject Alternative Name) extension listing every domain name the certificate is valid for. Modern certificates should list all variations — example.com, www.example.com, api.example.com — in the SAN. The old CN (Common Name) field alone is no longer sufficient for domain validation in most browsers.

Server Not Using SNI

SNI (Server Name Indication) is a TLS extension that addresses a specific problem: a single server hosting multiple domains, each with its own certificate. When a client initiates a TLS connection, it connects to an IP address. That IP might serve dozens of different domains. Without SNI, the server doesn't know which certificate to present before the handshake begins — the hostname is part of the HTTP request, which comes after TLS is established.

SNI solves this by having the client include the requested hostname in the Client Hello message, before any certificate is sent. The server reads the hostname from the SNI extension and presents the appropriate certificate.

If the server doesn't support SNI — or if the client doesn't include it — the server falls back to a default certificate, which probably won't match the hostname the client requested. The result is a common name mismatch error.

Encrypted SNI (ESNI), later standardized as Encrypted Client Hello (ECH), goes further: it encrypts the SNI field so that observers on the network can't see which hostname the client is connecting to. SNI in standard TLS is visible to network intermediaries even though the connection itself is encrypted.

For web administrators: ensure your hosting platform and web server support SNI. For troubleshooting: if a common name mismatch occurs and the certificate chain and domain name are both correct, SNI support (or lack of it) on either the client or server is the likely cause.