Rising privacy concerns and the prevalence of cybercrimes have necessitated the use of encryption technologies as means to protect data that is being transported over millions of public and private IP networks. Encryption uses various algorithms and keys to convert plain text data into an unreadable format, ensuring data integrity and confidentiality. However, networking vendors find themselves in a unique predicament due to encryption-induced visibility loss. Over the years, encryption has evolved to become increasingly complex and virtually indecipherable. New encryption protocols such as TLS 1.3 and ESNI, for example, leave very little readable information for networking tools which are dependent on real-time traffic visibility.
The encryption dilemma
DPI (deep packet inspection), a staple for real-time network analytics, is very much dependent on packet readability. Unfortunately, as more and more IP networks and applications start using advanced encryption protocols, the resulting visibility gaps raise questions about the efficacy and the relevance of DPI in delivering accurate traffic analysis at the packet and flow level. To address such concerns and to analyze the evolution of DPI against encrypted traffic, we initiated a research project that culminated in our recently published report ‘Deep packet inspection and encrypted traffic visibility for IP networks’. The research was based on an industry survey of 32 leading networking vendors who shared their views on the encryption and visibility dilemma, and the role of DPI in delivering encrypted traffic visibility.
Network disruption
The survey finds loss of traffic visibility from new encryption protocols to be a major concern for the majority (85.3%) of networking vendors. These vendors span all major solution segments including security/revenue assurance, analytics, network performance management, traffic management and policy control. With encryption, it becomes impossible to identify the underlying applications and services. This compromises a large number of traffic processing tasks that are entirely dependent on packet identification. Traffic prioritization, for example, requires identification of packets by ‘important’ and ‘less important’ applications. Similarly, a lack of data at the individual application level leaves performance monitoring with only macro indicators. Advanced network features such as AI-based network slicing and self-healing become impossible to implement when network administrators are kept in the dark about the applications and services navigating the network.
Why not just decrypt?
The report also assessed a number of workarounds adopted by networking vendors to regain the visibility lost to encryption. The most common method among these is decryption where SSL/TLS inspection tools decrypt, read and re-encrypt data packets. One obvious drawback of decryption is the latency that is added by decryption and re-encryption. More serious issues relate to security, privacy and compliance, especially with increasing regulatory pressures regarding user data protection in most parts of the world.
All these point to an obvious fact: The more encrypted traffic becomes, the higher the need for a viable, efficient and workable solution for analyzing encrypted traffic. Based on the views of networking vendors participating in the survey, the report outlines a number of criteria for such a solution, including easy integration and deployment, consistent performance and accuracy, and future readiness.
The alternatives
The report establishes two key points – networking vendors need visibility into encrypted traffic and they need techniques that are non-intrusive, highly practical and secure. The report assesses two such techniques – one based on statistical and behavioral/heuristic analysis, and one based on machine learning (ML) / deep learning (DL) – and finds strong preferences for both. The former analyzes packet movements, for example arrival intervals between packets, packet direction and flow entropy. The latter adopts advanced AI-based algorithms to identify the underlying applications and services.
Next-gen DPI: Back with a vengeance
As preferences are shifting to these newer and more advanced techniques, the survey looked at how DPI can continue playing its role in delivering real-time traffic intelligence for today’s networks. To do so, the survey first establishes the current adoption of DPI. It finds FOUR out of FIVE networking vendors using or planning to use DPI. Admittedly, DPI tools have evolved over time to go beyond packet payload information analysis to incorporate more novel techniques capable of reading encrypted traffic. This helped DPI retain a large part of its filtering capabilities in the wake of various encryption and obfuscation methodologies.
These gradual improvements, unfortunately, proved inadequate as new and more stringent encryption protocols emerged, pushing DPI to adopt a more revolutionary approach to managing encrypted traffic. We at ipoque responded to this challenge and successfully developed our own encrypted traffic intelligence (ETI). ETI combines ML and DL algorithms with high-dimensional data analysis. These algorithms include k-nearest neighbors (k-NN), decision tree learning models, convolutional neural networks (CNN), recurrent neural networks (RNN) and long short-term memory (LSTM) networks.ETI merges this with statistical and behavioral/heuristic analysis and advanced caching technologies such as service and DNS caching. By incorporating ETI into our DPI product line R&S®PACE 2 and the VPP-based engine R&S®vPACE, ipoque brings to market next-gen DPI that is capable of delivering highly accurate, real-time classification of protocols, applications and services for encrypted traffic in any type of networking environment.
Market readiness for next-gen DPI
Interestingly, the research finds a whopping 92.6% of network vendors to have already invested in encrypted traffic intelligence or planning to incorporate it into their DPI solutions, proving the strong demand for next-gen DPI with encrypted traffic intelligence.
While some vendors may already have such capabilities in-house, it is worth noting that advanced techniques such as ML/DL rely on complex algorithms and high computational power, something most network vendors are unlikely to develop in-house due to time and resource constraints. In such scenarios, networking vendors can take advantage of our OEM DPI software R&S®PACE 2 and R&S®vPACE to embed these capabilities across their solutions. Equipped with ETI, the DPI technology by ipoque supports any form of application-aware traffic processing, enabling networking vendors to configure application-based authentication policies, implement dynamic traffic rules and produce granular traffic reporting for encrypted traffic. From identifying ransomware to pinning down fraudulent log-in attempts into critical applications, ETI also helps security vendors detect traffic patterns and other indicators of compromise commonly associated with encrypted malware and malicious domains.
The benefits of ETI do not end there. Our new report ‘Deep packet inspection and encrypted traffic visibility for IP networks’ presents many other aspects of encryption including its long-term impact on networks that ETI can help to mitigate. The report also assesses different encryption protocols and layers of information visibility that are most impacted, while taking a deep dive into DPI itself in terms of procurement preferences and deployment models.
As a final takeaway, the report shows that with next-gen DPI, vendors and networking administrators need not be held back by evolving encryption algorithms, advanced obfuscation and anonymization techniques, or be held up by more ‘stricter-than-ever’ compliance mandates. With ETI, DPI will continue to be an indispensable tool for today’s networks.
For anyone wishing to learn more, the report ‘Deep packet inspection and encrypted traffic visibility for IP networks’ is now available for download here.
