Many companies are looking to find a source of threat intelligence that can give them better visibility into the risks unique to their technology stack. While some may not be using threat intelligence, others may not be getting the value they could. Choosing and integrating threat intelligence sources into your cybersecurity monitoring is challenging, but you do need to keep some considerations in mind during the process.
SELECTING THREAT INTELLIGENCE SOURCES
Choosing a source for threat intelligence is critical. You need to select an appropriate source for your needs and your team’s capabilities.
One of the most important considerations should be the methods supported for collecting and ingesting those feeds when choosing sources. Some are much more manual than others. A few offer integration agents that provide indicators as downloads of text or database files, while others support STIIX or TAXII for collection.
Others simply post lists to a web page and allow users to scrape the page via scripts. Some feed and platform vendors offer a flexible, well-documented API.
Whatever the collection and integration methods offered, make sure you can support these methods in your current technology stack before you purchase anything.
OPEN SOURCE THREAT INTELLIGENCE
Open source threat intelligence comes from data and information available to the general public, whether on the surface, dark, or deep web.
You can start by cultivating a series of web alerts based on things like:
- Devices
- Vendors
- Breaches
You can also use open source tools that help you combine that information with common vulnerabilities and exposures (CVEs) to address severity.
Many open source feeds get their indicators from the same sources and report on the same indicators, creating large areas of overlap and duplication of data, which must be managed.
PAID THREAT INTELLIGENCE FEEDS
Open source gives you information, but sometimes it can be too much or too vague. This is where paid threat intelligence feeds are helpful.
A paid feed is often:
- Current: open source information may only be available after you investigate the root cause, which means you might be compromised by the time you get the information
- Reviewed: paid feeds usually have expert analysts reviewing the data to help provide reliable information
- Relevant: paid feeds have their own areas of focus to reduce the amount of data you need to review so you can take meaningful action
Although the quality of paid feeds is often high, they can have a much narrower focus, requiring multiple feeds to get the breadth desired. Be aware–the cost of subscribing to multiple feeds can add up quickly. The challenge of managing integrations from multiple vendors must also be considered.
THREAT INTELLIGENCE PLATFORMS
Threat intelligence platforms collect, aggregate, and curate data from multiple sources and formats. Typically, these platforms include original research as well open source intelligence. This curation, enrichment, and deduplication reduce the noise that comes with free feeds while giving you the breadth and depth of multiple paid feeds. These platforms are often packaged with a well-developed API (Application Program Interface) or another tool that simplifies the integration of their feeds.
Threat intelligence platforms also provide a knowledge base that analysts can use to research and gain contextual information on indicators spotted in your environment. Though they can be expensive, these platform providers can offer a valuable stepping stone for organizations just getting started with threat intelligence.
INTEGRATING THREAT FEEDS
Once you have selected your feed sources, it’s time to integrate this data, usually with a SIEM or centralized log management systems. This is often the first step for users since it requires only one integration and can be used with logs from many different security technologies.
There are other types of integrations available as well. Direct integrations are available for firewalls, proxies, DNS, EDR, web gateways, and IPS, depending on vendor support. Additional custom integrations are possible for any technology that supports making API calls.
When integrating with a SIEM or log management solution, the basic premise is to take a list of indicators from the threat intelligence feed then check those indicators against the log messages coming from the various security technologies in your environment.
The three things you need to make this work are:
- A way to ingest the indicators themselves
- The logs that contain the information against which you wish to compare
- The SIEM resources (rules, reports, lookup tables) needed to make the comparisons.
The method you use to ingest indicators will vary according to your selected source of intelligence.
- Open source feeds: may require building a system to collect and process indicators into a list you can use for comparison, including scripts to fetch the feed data from a web page or extract indicators from a local database after collection.
- Paid feed or platform: may provide their tools to collect and ingest the indicators they offer.
It’s also important to ensure that the vendor you selected has a means of updating and/or expiring indicators and that your system can support these updates. Without it, your indicator list will grow forever. Your indicators change over time, which means they will become “stale” and inaccurate without updating. Outdated or inaccurate information is worse than no information at all.
Once you have the indicators ingested, you need to examine the logs from your security devices to determine if they capture any information you can use to compare against indicators. The most common types of indicators in threat feeds are:
- IP addresses
- domain names
- URLs
- malware hashes
Technologies that usually log this type of information include:
- Firewalls,
- Intrusion Detection Systems (IDS)
- Domain Name Servers (DNS)
- Proxies
- Endpoint Detection and Response (EDR)
- Web gateways,
- Mail servers.
To compare your logs against the indicators, they must contain at least one of those types of information. Don’t give up if your device does not include the required information by default. Many security vendors provide more detailed levels of logging that may include relevant information. Some allow you to customize the logs to build your log messages to have just what you need.
COMPARING INDICATORS
In most cases, you must parse the information you’re comparing. That means that it can’t be part of a big blob of text in a log message. The information must be isolated and put into its field. Parsing methods vary by vendor. Some vendors even parse messages automatically. Consult your SIEM vendor’s documentation for details.
After collecting indicators and parsing events, you must set up a mechanism for comparing and alerting analysts when a match exists.
Some SIEM vendors include that mechanism out of the box. Others require you to build it yourself within their system.
Generally, the mechanism is a lookup table of some sort and one or more rules to compare.
EXAMPLE
The rule might tell the system to compare the source and destination IP of every event passing through the firewall and check for matches against our indicator list. Domain names may also be useful when compared against an organization’s DNS or email message tracking logs. EDR and mail gateway systems may capture malware hashes used for comparison.
Enriching Indicators
You may opt to enrich indicators as they come in by adding information that will be useful to analysts right into the log events themselves. You could subscribe to a threat intelligence knowledge base that provides context and speeds investigations.
Using Alerts
Notifying analysts of a match is the final step in the integration. You can create something real-time like email alerts for each match. Depending on your sources and the volume of events you produce, this may result in a flood of notifications.
For most companies, starting with a daily report detailing the matches makes more sense. This will give you a sense of how many matches you are getting and provide an easy way for analysts to follow up on the most interesting.
INVESTIGATING MATCHES
Matches do not necessarily represent attacks against your network. Investigation and vetting of these matches is required. Part of creating a strong cybersecurity posture is threat hunting, looking for IoCs that might indicate that your systems have been compromised.
If the list of matches is too big, try focusing on the connections to and from critical resources or look at the highest severity indicators. Some feed vendors will allow you to set thresholds in their system so that low threat or low confidence indicators are never included in the first place.
AUTOMATING RESPONSES
You may eventually choose to automate responses to certain events based on the criteria you set. By automating responses, you’re able to use your threat intelligence more effectively. The longer it takes you to analyze the activity, the more damage threat actors can do.
CONCLUSION
Integrating threat intelligence feeds into your environment helps you create an actionable review that can help mitigate risk. Once you have a handle on the basics, you can begin to iterate.
Much like threat intelligence gathering itself, you must continually review and improve your systems and analysis over time. Whatever enhancements you build into your process, the ongoing maintenance keeps your defenses up-to-date, which is essential to protecting your business.
