In my last blog post, I described some of the overall design goals of Immunet Protect. In the next couple of posts, I wanted to talk about some pieces of our protection stack in some detail. It will not be feasible for me to describe all aspects of the technology (nor would it be advisable for me to talk about all the tricks we have up our sleeves!).
Our major protection technologies include, but are not limited to, traditional fingerprint-based protection, Ethos generic detections, and Spero machine learning based detections. In this post I will talk about signatures and leave the other topics for the future.
Immunet's traditional fingerprint based detections are ostensibly similar to those seen in other AntiVirus engines. They are geared towards specific threat instances, and essentially form an initial line of defense. While signatures alone fail to provide comprehensive protection against the vast spectrum of today’s threats, they still are suitable for handling specific threat classes (such as widespread malware). Furthermore, signatures are a well understood technology and tend to be less false positive prone than more generic approaches.
There are two aspects of Immunet’s signature-based technology that are worth noting. First, we have invested considerable effort into signature automation. Consequently, our signatures are not hand built by analysts and as such the time it takes for us to make a signature available to our end customers is short. In contrast, incumbent AntiVirus vendors have not streamlined this process and incur latencies that can be measured in the tens of hours to tens of days from the time they first learn about a threat to the time a customer is protected against it. Second, we use a cloud-based delivery model, which allows us to make protection more readily available to our customers.
These two facets are especially germane under today’s threat landscape where the vast majority of threats are fly-by-night and will likely be encountered on a small number of machines (if even encountered at all) for a brief period of time. Because we have a cloud-based solution and a vast signature database, we have unique visibility into threat lifetimes. Based on our actual in-field data, approximately three-quarters of threats that we observe in the field (and have signatures for) are seen just once – meaning the first time and the last time we see them are one and the same. Of the remaining, approximately 10% have a lifetime of less than 10 hours – meaning that the last time one of our users encountered the threat was approximately 10 hours after the first user encountered it.
Therefore, the process of translating “back-office” knowledge of a threat into a customer facing detection mechanism needs to be as swift as possible. Furthermore, it turns out that only a small fraction of the threats that we have visibility into in our back office are ever seen in the wild on actual customer machines. Among threats seen in the wild there appears to be a power law distribution where a small fraction appear on a large number of machines and a large fraction appear on a small number of machines. At the same time, it is not always possible to determine up front which of these threats will be runaway successes and which will more-or-less be one-hit wonders.
Consequently, the standard “push” model in which AntiVirus signature update packages are sent to each user is impractical since the likelihood that a given user will actually encounter a given threat is very small. Instead, we employ a pull model via the cloud. As I alluded to earlier, beyond being computationally efficient, this model also allows us to gather real-world data on how our technologies perform in the field. This type of view into actual field operations is unparalleled among existing AntiVirus offerings. While existing vendors have some data collection systems, these systems are often disparate to the point where the underlying protocols, programming languages, databases, schemas, data formats, operating systems, operating procedures, and operational personnel are different. To make matters worse, personnel who might have initially developed some of these systems may be long gone. Even still, these systems were not all developed with the purpose of creating a tight feedback loop between in-field data and actual protection mechanisms. A data driven approach must be a fundamental design principle from the onset – it cannot simply be thrown on after the fact. Companies wishing to employ a model like this (who are not already doing so) will need to overhaul technical systems, deal with large-scale integration issues, and will likely need to significantly reorganize their operations.
Because we have taken a clean slate approach at Immunet, we are not encumbered by these types of legacy concerns. Even though the industry as a whole recognizes the need to be data driven, we have been in a far better position to execute on that vision. We replace the dozens of systems other vendors use with literally just a handful. Furthermore, we established data models, schemas, and protocols that allow for seamless correlation between these systems.
Another benefit of our pull-based cloud model is that on the back-end we can (economically) store signatures for all the threats we encounter (regardless of whether we think that threat will become popular). While traditional AntiVirus vendors could theoretically ship signatures for every threat they know about, doing so with the traditional push model is too cost prohibitive. For example, if one were to try shipping signatures for all known threats, it would require (at least) on the order of a half a gigabyte of hard-drive space (something very few users are comfortable with). In this regard, a cloud model is not only more economical and more efficient, but it leads to improved protection and overall product quality.
For all their benefits, though, it is clear that we shouldn’t put all of our eggs into the traditional signature basket. Far too many threats have short lifetimes, and using traditional 1-1 signatures to catch each one not only fails to scale well, it also fails to adequately protect our users. If a single threat can infect a system, it can wreak havoc and cause that system to be permanently vulnerable. To address that concern, we’ve built technologies like Ethos and Spero, among others. I’ll be describing those in my next blog post, so stay tuned!