Webinar: What's New in Graylog 6.0? | Watch On-Demand >> ​

Lookup Tables and Integration with MISP

The other day, there was a great post by Ion Storm about integrating Graylog logs with intelligence from your MISP threat sharing portal. In this video, we will use this post as a basis for how to create a lookup tables and integrate them with MISP. 

Getting Started 

To get started, fire up MISP and go to events ->automation.  This is where all the information that we need lives. Your API key is found right at the top. That’s*your* API key. Below, you will see your URL. Scroll all the way down and you’ll even see the headers that we need. These headers are the important pieces.  Return to your Graylog portal, where we will create a lookup table. I’ve already made it, but I’m still going to show you everything that happens.


Looking at the buttons on the top right, we’re going tos tart with the furthest to the right. We are going to create a new data adapter called an HTTP JSON path adapter. From here, give it a name. We’ll use a short name to refer to it later.  Now, what we need are those details from that MISP portal. Enter the URL from the MISP portal as a lookup URL.  This information is also in Ionstorm’s tweet. Great work, Ionstorm.


The domain URL for my lab is called stumbling in and now we have included the MISP attributes, the REST search value, and the ${key}. ${key} isthe value we’re going to send to in our MISP lookup. If we want to play around and see what the results look like, we can test the output using curl. This process is manually curling the URL and seeing the resulting information. The output is perfect JSON, so I can pipe it to JQ. Reading the blue text response attributes that were received as a numbered list.  The square brackets within the results show some of the details from valid search output, such as category. Again, IonStorm said, “Hey, that’s a great single value.  Return the category!” I agree. Perfect. Further down the screen, we need to add those HTTP headers. The headers we need, are the name “accept” and the value “application/JSON”, exactly as it appears on the MISP portal. Once we have the three headers in there, we put the URL.  I suggest you double check for typos or better yet, make sure you’ve copied and pasted everything. Next, create a cache. A lookup table cache is how long we’re going to hang on to that data.  You can tune this as you see fit, and as you need to, but if you just want to get up and running, you can simply give it a name and description, then hit save.


The next step is creating the actual lookup table. The part that matters, is the name. After we give the lookup table a name, we can add more details that can be used while searching, or in dashboards. That name block that’s highlighted, that’s what we’re going to use in our pipeline rule. Next, we select the data adapter we made, and the cache that we made. Here is a pipeline rule and how it looks. On my particular So for firewalls there is afield called domain.  I’m going to shove all these domains into the MISP threat feed. In my lab, I’ve set up the COVID themed malware indicators in MISP.


If you remember right, we set a single value to be returned as the MISP category. Verifying malware detections is my favorite part of creating rules. Let’s go click on some COVID themed malware!  Oh, cool, some Russian casino is going to solve all my money problems. Now that I’ve clicked some links, some domains that should now be logged and then returned as threats from MISP. Of course, the next thing to do is set up some dashboards and alerts, then make it all nice and pretty, and ultimately make it so you are well aware whenever there is traffic to a suspicious domain according to your threat feed.  When you want to see some really, really cool stuff, like parameterized dashboards, automated alerts, or really kick off some serious magic, give our experts a call and find out how to do it. Thanks for taking the time to give me a couple of minutes. Once again, shout outs to@Ionstorm and the MISP project. I’m Abe, thanks for watching.


“Adam “Abe” Abernethy spent most of the last two decades in government security roles, delivering association conference talks and committing useful but terrible code on Github.  He is currently the lead Training Engineer helping organizations mature their security posture with Graylog.