Welcome to This Week in Log Management!
Every week we get many great questions through support, the community, social media, and our weekly demo. On Fridays, I like to share the most common questions and answers, tips, insights, a closer look at Graylog, interviews, etc.
If you have any questions for me, drop them on Twitter, and I’ll do my best to fold them into upcoming Friday posts. Our handle is @graylog2.
HOW DOES ARCHIVING WORK IN GRAYLOG?
Archiving is one of the most popular features in Graylog Enterprise, and for a good reason. A primary function is for log retention and storage. Depending on how much storage you can keep in the Graylog cluster, you may want to only keep a certain amount of logs available to you in the system. However, this does not mean that you want to throw away your logs that you don’t need.
One major log management use case for Archiving is Compliance. Some industries and certain types of data processed require that you keep logs in an archive for specific periods of time to meet operational compliance requirements.
Another example of why Archiving is important to log management is you may find you need to perform a deep dive into a security incident. This might be a known incident, but you may be required to go back in time to investigate logs in the Archives. One way that you might do your archive investigations, is to create a separate Graylog cluster and restore your archive there. This allows your investigation to take place outside of the main cluster so that you don’t impact that cluster’s day-to-day operations.
When archives are created, they are set up in such a way that they allow you to have archives by specific stream. You may find that certain log types log much more data, and you may require to have different archival needs and timeframes based on these different groups of logs. Once you have created configurations to Archive, these logs can be compressed in different formats, encrypted, and then completely moved off your Graylog cluster into other backup storage utilized in your organization.
For more information:
A great step-by-step video on How to Configure Archiving will help you through this process.
Index Rotation and Configuration
A feature that is directly related to your Archiving is Index Rotation and configuration. It is important to understand that your configuration setup in each of the indices you have built has a relationship to your Archival. Each index is assigned or connected to a stream. Depending on how many logs you are ingesting in Graylog, as mentioned when Archiving, it is important to keep Log Retention and Rotation as part of the equation when building out your logging needs.
For more information:
Here is an article on Log Indexing and Rotation For Optimized Archival in Graylog.
Thanks for joining us, and Happy Logging With Graylog!
