Part Two: Blacklisting – Finite Utility
Malicious actors are ruled by the laws of economics just like everyone else; they have finite resources. If they want to attack many targets, the chances are good that they will reuse their tools and techniques. If their malware binaries are detected or activity patterns are determined, it makes sense to record these things. If we detect these binaries or activity patterns again, we can raise an alarm and protect ourselves.
The Mandiant APT1 report is an excellent documentation of the playbook used over and over again. The tools used by the adversaries have been discovered and are signatured and blacklisted. The repeated activity and behavior of the adversary are also noted with the intention of being able to protect ourselves if we see the same activity in the future.
Because of the repeated nature of malicious activity, blacklisting is a good idea. Most security tools offered by vendors are based on some kind of blacklisting.
Change in Malicious Behavior
But what happens when these malicious organizations change their tools and change their activity? For a high-value target where a malevolent actor uses a completely unique method of attack, blacklisting fails.
A lot of malware tools used in attacks are polymorphic in nature, which means that every infection is unique and requires a new signature for future detection. Defense responds by looking for repeated patterns of specific portions of the binary file, and signatures are applied to portions of malware, not just the file as a whole.
Where there’s a Will
But even this theory has limitations. The malware author can create uniquely patterned subroutines within their software. This might be a lot of work, but remember, if a malicious actor is going after a target of high enough value, the effort to create a unique payload is worth it. There is an entire underground world that is dedicated to making malicious payloads undetectable.
Whether or not a defensive tool can catch previously detected malicious activity is never a good estimate of its strength. Malware detection rates are not very useful. If you have something of value, there is likely someone who wants to steal it and has the ability to do so — especially if you only rely on blacklisting defense.
A Custom Computer Analogy
Let’s perform a mind experiment. Imagine you have just built a computer and developed software for it. It works great for all the inputs you give it, but as soon as you send it out for quality control it is found to crash when specific values are entered.
You collect the list of values that crash your computer and you fix the bugs. Quality control continues to iterate, over and over, until you find a multitude of problems. You even begin to notice patterns of values that crash your computer, and begin to create fixes for entire categories of bugs. You think that if you just collect enough values that crash your computer, you will eventually fix most or all of your bugs.
After growing your bug list to huge lengths, you begin to wonder whether or not there is going to be a way to formulate a way to derive the entire bug list and, thereby, solve your problem.
Enter Alan Turing.
The Turing Findings
In 1936, Alan Turing wrote a paper that was his way to derive the answer to a similar problem presented above. What he demonstrated was that “there is no algorithm which can be applied to any arbitrary program and input to decide whether the program stops when run with that input.” (See this exploration of the “halting problem” for additional detail.)
As Sophos points out, this has huge implications for how we have traditionally performed detection of malicious activity. AV, IDS/IPS, SIEMs, Big Data are all great ideas and you should probably invest in some blend of blacklisting tools, but keep in mind that they are fundamentally limited.
Defensive systems that claim to not rely on blacklisting have appeared. At Entrust, we applaud this progressive thinking, especially the focus on social-engineering as an attack vector. But these vendors have had to engage in merger activity with experts in threat intelligence to round out their offering.
Dedication to Defense
I have had the pleasure of being trained by and alongside many people in the threat intelligence business. These skilled people dedicate their lives to defense and are capable of responding to attacks with discipline and systematic process. They are simply amazing. They are also exceedingly rare for the scale of attacks that are occurring today — and likely for tomorrow.
Economics affect the defenders as well. Blacklisting is relatively less costly, especially in a potential future world of information-sharing, but it is limited as an effective defense against attacks that are repeated.
If a malicious actor wishes to breach a target of a high enough value, the attack can be unique enough to make it difficult to detect. Threat intelligence experts can help mitigate against this, but they are finite.
That’s the balance where we find ourselves today. But I continue to believe that malicious actors do not have a fundamental advantage and that we must use the same kind of diagonal thinking they use in order to put the odds back into our favor.
Stay tuned for Part Three: Tuning the Signal – Identity Context
 “APT1: Exposing One of China’s Cyber Espionage Units,” Mandiant, February 18, 2103.