Detecting Network Anomalies With Graylog

A step by step walkthrough showing you how to detect network anomalies with Graylog Security
Detecting Network Anomalies With Graylog Security

A Graylog Security Use Case

We’re going to do things a little bit different in this demo today and instead of showing you all sorts of different features of Graylog and letting you guess at how they might work in a real-world scenario, we’re going to turn this on its head and actually work a real-world scenario utilizing tools that many of you are using today within your environment including Graylog Security and Illuminate.

In front of you you might be thinking, hey this doesn’t look like the Graylog console and you’d be 100% correct this is PagerDuty. Here we are specifically looking at an incident that has been created within PagerDuty for a high severity Cisco ASA anomaly that was detected via the anomaly detection engine within Graylog Security.

As I scroll down here you can see we have a link right within the ticket itself that will actually take me into the messages that generated this anomaly. We are going to hop over to this other tab here, which is just a click through of that link, and show you what this looks like.

Investigate the Data

We have a search for the last one hour associated with a high confidence high grade anomaly for the specific detector in question which is the Cisco ASA unusual data transfer anomaly detector.  As we come into the search message window, I’ll open this up and we can start to see details around this anomaly such as when did the data that associated with anomaly start and stop, how confident are we that this anomaly is actually anomalous, and most importantly, the Source_IP address of the data involved in this unusual data transfer.

From here I want to see what’s actually going on on this machine so I’m going to click this little arrow here and insert this into a view or into what some of you all may know as a parameterized dashboard. I’ll grab this Illuminate core device investigation drill down, which is part of our core Illuminate content that comes with Graylog Security. 

We want to insert that IP address as you would probably imagine and go ahead and click submit here immediately we get to the summary page. It starts to give us a little bit of information  around what is actually going on within this system specifically. The first thing I notice is that we have very little activity for many many hours until all of a sudden, around 3:30 UTC, we suddenly see this massive spike and activity across this machine.

In this tab here let’s get a look at where this traffic is actually heading and as soon as we click on this screen it becomes immediately obvious where the vast majority of this data is being filtered out to this address. 

Use Scratchpad in Graylog for Notes

If you’re like me you like to utilize the Scratch Pad feature within Graylog which gives me an ability to just take some quick  otes to use as we continue to Pivot through the product. I’m going to write down this address and I’m also going to just take note of the IP address that actually triggered the anomaly of

I’m going to save this for a little bit later and go ahead and pivot out of this screen into my search tree so if you’re thinking from that security mindset the first thing I want to know is:  “What the heck is that 162 IP address?”  I’m going to set my time here for a day just to make sure we get all of the data that’s possibly occurring within this. 

I’m going to go ahead and grab this IP address here, do a search for the destination IP address there. I want to get specific into our Cisco ASA logs because we are is that the specifically what triggered the anomaly and I want to focus I want to really hyper focus in on that for this moment of my investigation.

What Happened?

I’m going to  run our search and you can see that we have quite a few messages. It has already been identified by our illuminate enrichment process as potentially an address associated with: ( and register to Dropbox Inc).

I’m going to scroll here and continue to see that this Geolocation is showing Taiwan but we are  based in the United States. We don’t do a lot of data  transfers to Taiwan specifically, I’m going to continue to scroll down to where we find our destination IP address which is right here. 

I’m going to use another new feature associated with graphic security which is these right-click investigation tools. I want to go ahead and look this up against VirusTotal. We can see that it comes back clean so we know that this is in fact a Dropbox IP address which led the question if this is this a shadow I.T issue or is this potentially a malicious actor utilizing Dropbox to cover their tracks. In some ways Dropbox can be used under the radar for this large data exfiltration. So to uncover that, I’m going to pivot away from specifically the destination and dive into our source. 

Investigating the Workstation

We’re going to come back to our 192 address here and we’re going to select Source_IP, the IP address, and get rid of our  Cisco events for now.  We’ve understood what we can from the Cisco side of things at this moment and we’re going to refocus on Windows. 

So I’ll go ahead and run our Windows events. I’ll grab our Windows logs and go ahead and run this search here. Immediately due to the color coding and the message summaries that are included with Graylog security, we’re starting to get a picture of what’s going on here. Okay, we can see that this device is called “Shells Workstation” and that we have a pretty large number of login failures associated with this username. The “Shells” workstation is followed by a success and that’s about all that we have on this box during this time associated with this IP address.

A Brute Force Alert?

So now really quickly we were able to go from an anomaly and walk backwards into where that data was flowing, how much data it was so far and potentially an entry point into the attack. Ok, now I’m going to stop here, with this brute force, and show you another place where you would have where we would have also caught something associated with this which would have been around the Brute Force and looking at our events you can see here that yet another Illuminate event, this potential Brute Force attack greater than 10 failed login attempts in a short period of time was also triggered. 

Incident Sent to PagerDuty

We had this set up to create an event and create an incident within PagerDuty. We would have seen that created there as well. Lastly, to look at this from yet a third angle, I want to come to our new security overview page. 

The Security Overview page is where we can really dive into what’s going on with the health of my organization from a security perspective. As I come in here, we can start to look at the different things going on at the user level and maybe things that are unique to different users. 

Anomaly Detected

We could also look into our network activity and we can see that there’s a very obvious spike that happens right around that 3:30 mark that we saw our data exfiltration explode. We can also identify the IP address that that exfiltration is coming from and lastly, we can come to the anomaly page and actually get pointed out to us right in the middle in red, here is one high grade anomaly. This means that it is highly anomalous. We can pivot into this to see that it is in fact our Cisco ASA unusual data transfer anomaly. 

In Summary

Now I know we breezed through this really quickly in a short period of time but, there’s a lot of directions we can continue to go with this. We could continue to go down the route of  investigating our CIS monologues and understanding really what happened on that box at a process level. 

Between that Brute Force success, to the data exfiltration how did that attack actually happen? How were they able to execute that? It’s possible that as we investigate along that path, we would have found the tools hashes and additional IP addresses that would have been caught by our threat intelligence as well. 

This is just one quick example of things that Graylog and Graylog Security are designed to detect and respond to and show you a little bit of a day in the life of what it looks like to actually run this tool.