You have Gigabytes or Terabytes of logs coming in on a daily basis, but now what do you do with them? Should I keep 10 days, 30 days, or 1 year? How do I rotate around my logs and configure them in Graylog? Let’s talk about the best practices around log retention and how to configure them in Graylog.
Log rotation can be done for various reasons ranging from meeting a compliance goal, keeping the size of the index down for faster searches, or to get rid of data after a set amount of time. Graylog enables you to rotate the indexes based on a few methods. Message count will rotate the index after a number of messages have been written into the index. Index size rotates the index after the size defined has been reached and Index time rotates the index after the specified time. All have their uses, with the most common being index time. With index time, you can meet the three months online, but setting the index to rotate every week, and keeping 12 of them online, or every day and setting the max indices to 90.
INDEX ROTATION
To get into your index rotation strategy you need to go to System -> Indices and select Edit next to the index you would like to modify. In this example below, we have a 1-month rotation and are keeping 12 indexes for a full year of data.
After you set your rotation strategy, you will also need to select your retention configuration. Let’s have a quick overview of each choice and why you might want to pick that option.
● Delete: When you delete the indices you are having the minimal resource consumption by Elasticsearch and removes the index from disk thus saving disk space. This would be a good setting for operational data, which after a couple of weeks has no value (System Metrics, Flow Data, etc.)
● Close: Closing an index, blocks Elasticsearch from writing more data into it, but keeps it online and maintains the index’s metadata so you can still search it.
● Do Nothing: No resource-saving on Elasticsearch, and will keep the index open and on disk until manual removal.
● Archive: Discussed Later
ARCHIVING
If you would like to archive your data you can use the Enterprise version of Graylog to set up a backend storage location, allowing older indexes to be moved and compressed for long term storage. After you have a mount location on your server and configured that as the storage location of your archives, you can then set up the “Archive Index” retention strategy and give the number for when it will start moving them to the archive. In this example, the 21st index will be moved to the archive location, and the index closed. Many will choose to delete the index after archiving, as the archive feature allows for restoration if needed.
CONCLUSION
Understanding your log retention and rotation strategy is essential in any deployment of Graylog. With correct rotation strategies, your logs will be collected and maintained as expected, and allow for a fast and useful log aggregation tool.
