The Graylog blog

The Data Explosion and its Effect on Security

Data is exploding. The shift to digital business is driving a massive expansion in the volume of data that organizations produce, use, and store. It is also accelerating the velocity of data—that is, the data is changing more rapidly than ever before. Which in many ways is great—more data can bring more insight into customers, markets, and opportunities.  But more data can also be a problem. Because that very data can become the target for attackers who are intent on stealing your customer information, ripping off your IP, or simply holding your business to ransom.

The traditional approaches to managing the risks around all this data have been to monitor it, especially by looking at the log files that all those data-processing systems create. Keeping an eye on the logs, the logic went, meant that you could more easily and quickly spot an attacker before they got chance to do any real harm. Of course, it’s never been as easy as that; the
difficulties in collecting and monitoring all those log files has meant that businesses have often had limited success in keeping the log files under careful observation.

And now it’s about to get much, much worse. Rapid technology advancements and a growing number of cloud-based systems are continuously introducing new complexities and a whole new level of scale in managing log data. And it’s not going to slow down any time soon. Per Gartner, by 2020, more than 20 billion connected things will be in use worldwide ( With all of those connected things comes even more massive amounts of data your organization must be able to manage, analyze, and use. Throw in global networks with locations around the world and the vast number of possibilities for infrastructures with on-prem and cloud-based systems, and you have a scalability issue that presents a challenge to businesses with even the most sophisticated IT approaches.


Image via


The ability to protect sensitive data is hugely important to your organization’s success and now widely becoming a legal mandate. But, of course, the challenge remains: before you can find evidence of threats in your data, you must first get a handle on collecting and managing your data in a comprehensive and easy-to-analyze way. So how do you consume and manage all of this data and use it to take action to safeguard the security of your business?

The answer is to go back to the foundations of good security. Log files still sit at the very heart of security management best practices. The data contained in log files can show you who’s doing what, and when and where they’re doing it. But before you can gain that kind of insight into what’s happening in your IT environment, you must get all that event data captured in your log files into one spot where you can make sense of it. And even if you have all your data in one place, you’re not improving security if the amount of data is overwhelming to the point that you can’t sift through it.

A proactive log management solution is critical to the investigation of and response to security incidents. A log management solution should provide a scalable, centralized process that collects, stores, analyzes, and archives of the data from your systems.


It’s time to reevaluate your approach to log management. While many other security technologies will promise silver bullets that can solve your security challenges, the simple fact is that time after time, post-breach analysis has shown that a thorough understanding of the data in log files would have spotted attackers before they could do real damage.

So put log files back where they belong—at the heart of your security planning. Start thinking about ways to more easily manage the massive volume of potentially usable data that’s captured in those log files. Then bring the logs together, simply and easily, into a central repository where you can extract the maximum insight from them, quickly enough to stop attackers dead in their tracks.

The technology of log management has evolved spectacularly in the last five years, and new solutions exist that offer incredible flexibility and ease of use to help drive simpler use of the data your business already collects—the vital insight already captured in your logs.


While no single security technology can solve your security, compliance, and governance challenges, the central role of log files as a security and risk management tool is every bit as important today as it was a decade ago. In many ways, with attackers working hard to exploit every possible gap in security controls to steal your data, the power of log files may be even more important.

Take a look at how your organization handles log management, and seek out opportunities to extract even more value from the data already there – you might be very surprised by what you find.

Get the Monthly Tech Blog Roundup

Subscribe to the latest in log management, security, and all things Graylog Blog delivered to your inbox once a month.