The rapid shift to working remotely brought included many new IP addresses in the company log data. With the workforce scattered across the city, state, and sometimes country or farther, IT Teams needed to review and understand the new and evolving traffic patterns. Graylog lets you extract and visualize Geolocation information from IP addresses in your logs. It’s no surprise that the steps to configure the Geolocation resolution and create a map with the extracted geo-information was a popular post in 2020.
Gathering logs that contain IP addresses are quite common across your infrastructure. Your firewalls, web servers, wireless infrastructure, and endpoints can contain IP addresses outside your organization. Having additional data on those logs that give you the Geolocation of the IP address helps your investigations and understanding of your traffic patterns.
For example, if you can see logs on a World Map, you can quickly identify communications with countries you haven’t communicated with previously.
Now you can easily see traffic to Brazil, something your organization might not normally do.
Let’s go through all the steps you will need to do on your Graylog box to get your system to enrich your log data.
CONFIGURATION
The first step is to download the database of Geolocation information. (In this step-by-step guide, we will use the MaxMind GeoLite2 Database in the binary format (.mmdb). You can download these files here.)
Once you have downloaded the files, the next step is to store the geolocation database on all servers running Graylog and then set the file permissions to allow Graylog to read the file. In this example, the file’s location is here: at /etc/graylog/server/GeoLite2-City.mmdb
Next, we need to configure the Lookup Table to read the database when an input is added. We will first create the data adapter under System -> Lookup Tables and Select “Data Adapters.” On this page, select “Create data adapter,” and fill in one like below:
Next, we need to create a Cache with the “Create cache” button like below:
In the last step in the Lookup Table, we need to create the table itself, using the created data adapter and cache from the previous two steps:
Now that we have created the lookup table and it’s ready for use, we need to create a pipeline rule to utilize it and add the metadata to each message with an IP address.
Go to (System -> Pipelines) and under the “Manage rules” we will need to create a new rule. Give it a description so you can remember it, and in the Rule Source put:
rule "GeoIP lookup: src_ip"
when
has_field("src_ip")
then
let geo = lookup("geoip", to_string($message.src_ip));
set_field("src_ip_geo_location", geo["coordinates"]);
set_field("src_ip_geo_country", geo["country"].iso_code);
set_field("src_ip_geo_city", geo["city"].names.en);
end
The when condition is only allowing the rule to process when the log has the field src_ip. NOTE: Please change this to the field containing IP addresses in Your logs. Once it finds those logs, it runs the lookup against our Lookup Table “GeoIP” with the data in the src_ip field, then adds in the Location, Country, and City.
Notice, this rule only applies to the source IP address. If the destination address should be looked up as well, add additional lines to this rule, or create a second rule for logs with destination IP addresses.
After we’ve created the rules, we will add them to a stage in the pipeline processing the logs with an IP address in them, and the enrichment will start working.
And once new logs come through the pipeline, you will see the enriched log entries.
Finally, if you run a search aggregate on “src_ip_geo_location” and change the table type to be a “World Map” you will get:
ADDITIONAL THOUGHTS
In the screenshot below, you can see there are many fields in the lookup table. You could pull additional data out and put it into the logs if you would like to help enrich your logs further.
