The news is full of stories about the talent shortage in IT, especially in IT security. This shortage has created pressure on organizations to grow IT operations and to do that securely, all while having too few staff. Many are turning to threat intelligence to give their security analysts the tools they need to evaluate threats quickly and effectively. Essentially offering “Intelligence as a Service,” these tools enable organizations to benefit from the research of others.
In addition to acquiring new tools, teams have been forced to become more efficient. Improving processes, streamlining operations, and flattening decision-making is the proverbial low-hanging fruit of efficiency. It’s not always easy, but it does work and costs very little apart from the time invested.
Organizations often come to realize that further gains are possible by automating part or all of a process. If a decision is so basic that a machine can make it, why waste an expensive human on it? Automation works really well for some things. Simple up/down monitoring, temperature, CPU or RAM monitoring are all good candidates for automation. Security processes, on the other hand, have been harder to automate.
A good example of this difficulty was the way Intrusion Prevention Systems (IPS) supplanted Intrusion Detection Systems (IDS). As the name implies, IDSes were only supposed to detect intrusions. Responding to them was up to the humans.
When IPSes were introduced, they offered the ability to see attacks in real time and to respond automatically, blocking anything that threatened.
There was one problem though. Nobody trusted IPSes enough to let them actually block anything. False positives were common enough that management would not allow automatic blocking to be enabled. For months, or even years, IPS systems stayed in “passive” or “observe” modes.
This concern was understandable. They were not confident enough in the technology to risk blocking legitimate traffic. Gradually, with time and improvements in IPS technology, most came to believe that malicious traffic, regardless of the source, was a threat that deserved to be blocked automatically. Today, most organizations use an actively blocking IPS in some form as a key part of their defense strategy.
Threat intelligence feeds are at a similar crossroads now. Just as an IDS notification alerts you to an attack on your own network, indicators of compromise are identifiable pieces of information captured from a compromise attempt that has happened either on your network or somewhere else in the world. These indicators are collected and distributed, either by open source or commercial entities. Organizations can subscribe to one or more feeds and build databases of indicators.
Indicators can take many forms, the most common being IP addresses and domain names, but they can also include URLs, MD5 or SHA hash values, or even snippets of code.
A database full of these bits of information can be very useful. For example, allowing users to check an IP address seen in the firewall logs can help you determine if it has been involved in any malicious behavior. Users could also compare hashes in the logs of their anti-malware solutions to those in the database. However, when you consider that tens of millions of indicators are created each day, the chance of manually checking even a tiny fraction of indicators against our logs is vanishingly small.
This scenario is a perfect example of one of those processes that can be automated. Since we want to know every time our users try to communicate with a known bad IP address or domain, and since it is an enormous time sink to do it by hand, we should get a machine to do it. By integrating threat intelligence feeds with a log management or SIEM solution, we can create rules that compare each indicator against some relevant piece of information in a log entry. It can then notify us of every match, whether in real time or by adding it to a list for our review.
Which is great and all, but at that point, we are right back where we were in the early days of IDS. We get told about things automatically but can only respond manually. This situation does not scale, and therefore is of limited benefit if we don’t have the staff to review the results.
Instead of simply notifying us of a match, we can integrate our enforcement points with a log manager. Then, when our endpoint system produces a hash value of a known malware variant, we can automatically notify the endpoint system via its own API to block all further communication between that host and the offending IP address or domain. By integrating the other enforcement points as well, we can take what we learned from the endpoint and communicate it to the rest of the environment. We can then block that IP address not only at the host level, but also at the firewall, IDS, WAF, web gateway, or anywhere else.
Which brings us back to the crossroads I mentioned. We can set up all these nifty automated responses, but what if management won’t allow us to block automatically? The threat intelligence space is relatively new, and the quality of feeds can be wildly variable. It can be difficult to convince business stakeholders that we won’t block legitimate traffic with our automated system.
We can help ease some of the fear by framing the decision in terms of risk. There is always a small chance that a business partner or customer could end up on a threat feed list, but we should weigh the risk of blocking legitimate traffic against the risk of interacting with a known bad indicator. Do we really want to keep communicating with our supplier if they are compromised and could potentially compromise our systems as well? Which represents the greater threat?
The second thing we can do to mitigate the risk of blocking legitimate traffic is to set a very high bar for blocking. There are many ways to do this, such as mandating that simple connections or attempts to or from a known indicator are not enough to block automatically, but that a connection attempt accompanied by an alert from a second technology is. Another way is to use threat feeds or threat intelligence management platforms that enrich their feeds with additional data. This data can take the form of confidence or reputation scores, and can help, along with information on categories and severity to make your automated evaluation much more granular. This level of specificity reduces even further the chances that legitimate traffic will be blocked. These curated feeds and management platforms are often not free, but they can greatly improve the quality and manageability of threat intelligence feeds.
Threat intelligence can be a useful addition to your analyst tool kit. Analysts get the advantage of sophisticated security research, allowing them to understand the context of attacks in a way not previously possible except for in the largest organizations. I encourage you to examine your own use of threat intelligence and see if it holds any opportunities for your organization to improve or automate your own processes. If you’re lucky, perhaps you will be able to squeeze some extra value from the security dollars you have already spent.
