My last article explored how Security Service Edge (SSE) enforces Zero Trust principles using a network-centric approach. A critical aspect of this approach is the inspection of encrypted internet traffic that requires decryption for effective security. Secure Web Gateways (SWGs) play a key role here, analyzing and filtering web traffic to protect users and ensure secure browsing. However, as encryption protocols advance, traditional web proxies and SWGs face increasing limitations. This article examines these challenges and explores alternative strategies enterprises can leverage to maintain robust security in an increasingly encrypted digital landscape.

Core Capabilities of SWGs

Web Filtering: The First Line of Defense

Web filtering is a fundamental feature of Secure Web Gateway (SWG) solutions. It allows enterprises to monitor and control web traffic, enforce internet use policies, and block access to dangerous or inappropriate sites. By categorizing websites, SWGs help restrict access to high-risk websites, such as gambling, social media, and file-sharing platforms. SWGs rely on specific metadata from web flows to make accurate categorization and precise filtering decisions.

TLS Inspection: Unpacking Encrypted Traffic

Web filtering relies on web traffic metadata, and deeper security functions like threat detection and data leak prevention (DLP) need visibility into encrypted traffic. Transport Layer Security (TLS) inspection is vital here, as over 90% of web traffic is now encrypted via HTTPS. Decrypting and analyzing this traffic enable organizations to spot threats and prevent data breaches. Without this deeper look, potential dangers can easily slip through. To comply with legal and privacy regulations, organizations use selective inspection methods. This approach allows them to exempt sensitive traffic, such as healthcare or financial services data, from decryption. By doing so, they ensure that privacy standards are maintained.

The Technical Foundation of TLS Inspection

To understand how traditional web proxies and SWGs handle encrypted traffic, let’s start with TLS 1.2, a widely used version of TLS. Although most of metadata in TLS 1.2 handshake messages is encrypted, certain fields, like Server Name Indication (SNI) and Application Layer Protocol Negotiation (ALPN), remain in plaintext. These visible fields allow SWGs to apply filtering and selective inspection. In transparent proxy deployments, SWGs use specific fields, such as the Common Name (CN) from server certificates and the SNI from Client Hello (CH) messages, to categorize web traffic and enforce enterprise security policies. Then the poxy issue a certificate to the client browser that mimics the server’s certificate, acting as a Man-in-the-Middle (MITM) to inspect traffic. In explicit proxy deployments (e.g., using PAC files), SWGs can read the hostname directly from the HTTP CONNECT request.

New Protocol Challenges for SWGs in Traffic Inspection

New transport and encryption protocols, such as TLS 1.3, QUIC, and TLS Encrypted Client Hello (TLS ECH) introduce challenges for SWGs. While these protocols enhance user privacy, they also reduce network visibility, making it harder for organizations to monitor encrypted traffic and enforce security policies. Let’s explore this in more detail:

TLS 1.3: Enhanced Privacy, Reduced Visibility

TLS 1.3 increases privacy by encrypting the server certificate, which SWGs rely on to identify the destination service and categorize web traffic. As a result, SWGs are left with only the destination IP address and the SNI in the CH message to determine the destination service. To address this, SWGs must use a “TLS probe” to retrieve the server certificate for accurate categorization. However, SNI is not entirely reliable (see Proof of concept here), making it challenging for SWGs to filter traffic accurately.

TLS ECH: A New Frontier for Privacy

As of November 2024, the TLS ECH extension remains in draft status, with the latest version being draft-ietf-tls-esni-22, published in September 2024. ECH takes privacy a step further. It encrypts the CH message, including the SNI. It encapsulates the entire legacy CH handshake message (Inner CH) within a new CH wrapper (Outer CH) message. This makes it much harder for SWGs to filter traffic. As ECH becomes more common, SWGs will face even bigger challenges. These evolving protocols reduce SWG visibility, making it more difficult to monitor traffic and enforce security policies. Organizations must adapt their strategies to effectively manage these changes and maintain robust security.

QUIC: Enhanced Performance, Challenging Inspection

QUIC was developed to enhance speed and security, but it poses significant challenges for SWGs in traffic inspection. Designed to prevent MITM interference, QUIC obscures or encrypts metadata like flow control, packet numbers, and connection IDs, making stateful inspection nearly impossible. Unlike HTTPS, where records span multiple packets, QUIC encrypts each packet individually, requiring multiple cryptographic operations to decrypt traffic. Additionally, since QUIC is stream-based, stream reassembly can only occur after packet decryption. This results in considerable computational overhead, making the decryption process far more complex for SWGs than traditional TLS inspection.

Enterprise Alternatives to Address Encrypted Traffic Challenges

Blocking or Disabling New Protocols

While the industry consensus leans toward blocking or disabling these new protocols like TLS ECH, QUIC, or DNS over HTTPS (DoH), this approach is often impractical. These protocols are designed to mimic regular traffic, making them difficult to detect and block effectively. Currently, the only reliable method to block them is by disabling them directly in the browser. Furthermore, as they become widely adopted, blocking them could disrupt essential enterprise internet services, leading to significant issues for businesses.

Decrypting New Protocols

Some SSE vendors claim their solutions can still decrypt and inspect new protocols, including the latest TLS extensions and QUIC. While this is technically possible, it demands significant computing power to decrypt and reassemble QUIC streams. Additionally, intercepting DNS over HTTPS to modify DNS records for TLS ECH decryption adds operational overhead, raising concerns about the long-term sustainability of these methods. The Internet Engineering Task Force (IETF), in RFC 7258, has emphasized limiting pervasive monitoring through deliberate protocol design. Given this stance, organizations may need to weigh the costs and benefits of such decryption approaches. Rather than relying on resource-heavy decryption, they might adopt a new security model that respects privacy needs while ensuring sustainable and effective protection.

Web Traffic Fingerprinting: Enhancing Security Without Decryption

Traffic fingerprinting is an innovative approach that enables enterprises to detect malware within encrypted traffic without the need for decryption. By employing machine learning models, this method identifies unique "fingerprints" indicative of malicious activity within encrypted streams. Tools like Cisco’s Encrypted Visibility Engine (EVE) utilize this technique, offering visibility into encrypted traffic without compromising performance. However, challenges persist regarding false positive rates and the detection of zero-day threats, highlighting the need for continuous improvement of these tools.

Enterprise Browsers

Enterprise browsers bring security closer to the user by embedding protection features directly at the browser level. Instead of positioning SWGs as an MITM for traffic inspection, these browsers can inspect traffic internally before rendering webpages. They can also integrate with SSE solutions to leverage additional capabilities like sandboxing or DLP engines.

The Future of Network Security: Balancing Privacy and Security

As encryption protocols evolve, inspecting network traffic is becoming more difficult and, in some cases, impossible. This shift aligns with the IETF’s guidance in RFC 7258 which views pervasive monitoring as a threat and promotes privacy-focused protocol designs. For organizations adopting Zero Trust strategies, adapting to these changes is essential. New protocols like QUIC and TLS ECH make it harder for traditional security tools to see inside network traffic. This reduction in visibility highlights the need to rethink how enterprise assets are protected. SSE was initially designed to support a network focused Zero Trust approach. However, Zero Trust must extend beyond the network layer to achieve comprehensive protection. It should also cover the application layer and endpoints, minimizing the need for in-transit inspection.

Key Takeaways

In today’s highly encrypted landscape, traditional network security must evolve. This article explored essential strategies to help enterprises address these challenges, emphasizing a shift toward sustainable, effective protection without the need to block or break encryption protocols for security inspection, a practice which is now considered as intrusive by the IETF.

• Traffic Fingerprinting: Using machine learning models to detect malware in encrypted traffic without requiring full decryption.
• Enterprise Browsers: Embedding security controls directly in the browser to inspect traffic before displaying content while enforcing enterprise DLP policies.
• Application-centric Zero Trust for any internal application: Enforcing Zero Trust principles at the application level through context-aware access controls.

By adopting these strategies, organizations can balance privacy with robust security, ensuring protection of critical assets as encryption protocols evolve.