NOTE: Graylog has made many updates to the application since this release. We encourage you to update to the latest version and take advantage of the large number of new features and functionality.
The official blog post for Graylog v2.3.0 is here! We are excited to reach this milestone and appreciated your testing and feedback along the way. In v2.3.0, we introduced lookup tables and support for Elasticsearch 5 which includes the AWS Elasticsearch service. We’ve also fixed bugs and have noted the full list of changes below.
DOWNLOAD LINKS
Download Graylog v2.3.0:
- DEB or RPM packages are available in our repositories. Check our documentation for details
- Docker image
- OVA / Appliance
- Tarball (manual installation)
Let’s see what’s new!
ELASTICSEARCH 5 SUPPORT
In v2.3.0, Graylog is switching from the Elasticsearch node client (using the version-specific binary protocol) to a lightweight HTTP client. This enables Graylog to use all major versions of Elasticsearch which are currently available, including 5.x. In addition, we are supporting the AWS Elasticsearch service so you no longer need to run your own ES server.
Depending on your environment and due to the overhead of generating JSON and using HTTP, you might notice a slight drop in indexing performance. We have optimized the last tiny bit out of it and were able to keep the overhead as small as possible. Switching to the HTTP APIs was a careful consideration of future Elasticsearch support and deployment flexibility.
If you want to know more about the differences and implications of the different ways to access an Elasticsearch cluster, you can read more about it in Elasticsearch’s documentation.
For detailed advice on how to configure the Elasticsearch part of a new Graylog installation or migrate from an existing configuration, you can view our upgrade notes.
If you would like to read up on the technical details, you can inspect the relevant changes.
Lookup Tables
Enriching messages with data from external sources, such as CSV files, Geo IP data, or results from remote HTTP sources is a common pattern in today’s log management. Graylog now supports a core framework for lookup tables, making it easy to work with external data sources.
Lookup Tables are fully pluggable and always consist of a combination of a data adapter, which is providing the data and a cache. Out of the box Graylog ships with three data adapters:
- a local CSV file adapter for static mappings
- a HTTP JSON + JSONPath adapter to retrieve results from HTTP APIs
- a GeoIP2/GeoLite2 adapter for MaxMind databases
There are various ways to interact with lookup tables in Graylog, allowing to build powerful workflows.
After a lookup table is created, it is accessible from:
- Decorators
- Extractors
- Converters
- Processing pipeline rules (both for ingestion and decorator usage)
Data adapters in Graylog can return both single values as well as complex data structures. For example, the GeoIP data adapter returns the latitude/longitude pair as its simple value, but the entire GeoIP database entry for the multi value case. This offers you the flexibility in pipeline rules to enrich as much or as little data as required.
The same is true for the other data sources like HTTP. This means you could decorate search results with data from your customer database, including information such as contact addresses, SLA levels, or host names.
Since you can choose the caching strategy for each lookup table individually, you can tailor them to the specific requirements, such as heavy caching for incoming messages to maintain a high performance ingestion or very short caching on search result decoration.
CHANGES
The full Graylog 2.3.0 changelog is available here.
WE LOVE YOUR FEEDBACK
We want to hear what you think about Graylog v2.3.0! There are a variety of ways to provide feedback, all of which can be found on our community page: Report bugs and other issues in our GitHub graylog-server repo. Help with documentation in our GitHub documentation repo.
