Webinar: What's New in Graylog 6.0? | Watch On-Demand >> ​

Risk Based Alerts Using Lookup Tables

Proactive Event Notifications

Proactive event notification is one of the most valuable components of centralized log management and SIEM. It allows us to identify problems, misconfigurations, and potential security risks at an early stage. One of the ways we can improve event notification within Graylog is through the use of Lookup Tables.

In this post we will demonstrate how we can create a lookup table which we will use in an Event Definition to identify when high risk users, those who recently handed in their resignation, are making changes to Active Directory. We will do this by maintaining a MongoDB lookup table of employees who are currently on their notice period, provided to us by the HR department, and then use this list in an Event Definition which watches for object changes in Active Directory.


Lookup Table Configuration

Navigate to System > Lookup Tables. Lookup tables in Graylog require 3 components: a data adapter, a cache, and the lookup table configuration.

We’ll start by creating a data adapter to store our list of employees on their notice period. I’m going to use the MongoDB data adapter option however there are many ways in which we can tackle this problem, including performing an Active Directory lookup for a relevant attribute.


Create The Data Adapter


After creating the adapter, we can begin adding entries. Entries to the table can be added and removed in 3 ways:


  • Via the Graylog UI
  • Pipeline Rules
  • REST API calls


The REST API method may be a good way to maintain this list automatically, perhaps a query could be made to an HR database and the results of the query used to update the MongoDB Data Adapter.


In my entry table I have a single key, “users”, and a list of the users who are currently working in their notice period; “s_jdawson”, “s_kbromham”.


Create The Cache for The Data Adapter


After the entries have been added, we need to create a cache that can be used in the data adapter configuration. Caches are used to improve the lookup performance and to avoid overloading databases and APIs. In my case I created an In-Memory Cache with the default values as my environment is quite small.

Finally, we can configure the Lookup Table itself. This is simply a case of giving the configuration a name and linking our data adapter and cache:


Once our Lookup Table is created, we’re able to perform Test Lookups to validate that it is returning expected results.


Create The Event Definition


We now want to use our new Lookup Table as part of an Event Definition. Navigate to Alerts and select Event Definitions. After setting a Title and Description, we can configure the Event conditions.

We’re going to use a Filter and Aggregation event type and our search query will consist of the following:

gim_event_subcategory:”iam.object modify” AND source_user_name:$username$

In my environment I’m ingesting my logs via Graylog Illuminate content parsers, which conform the messages to the Graylog Information Model (https://schema.graylog.org/en/stable/). This means the fields in my messages are normalized across different log sources. Two of these fields are the gim_event_subcategory and the source_user_name.

My search query is first looking for “iam.object modify” events and combining that search with a parameterized lookup of the source_user_name.


After adding the search query into the text field, we need to declare the parameter:

This is the configuration where we link our Lookup Table with this Event Definition:


Once we’ve saved the Query Declaration, if we have results within the search timeframe, we should be able to validate that the Event Definition is working properly:



It’s then a case of configuring the remaining Event Definition components such as Fields and Notifications before saving our configuration. Lookup tables are a powerful part of Graylog, beyond Event Definitions, they can also be used as part of Decorators, Pipeline Rules, and Extractors.

Get the Monthly Tech Blog Roundup

Subscribe to the latest in log management, security, and all things Graylog blog delivered to your inbox once a month.