NOTE: Graylog has made many updates to the application since this release. We encourage you to update to the latest version and take advantage of the large number of new features and functionality.
We could not be more excited to announce that Graylog v2.0 is now GA!
In v2.0, you will find significant architectural improvements as well as tons of new features, such as live tail, message processing pipeline, map widget, collector sidecar, and more. We built this series based on feedback from countless conversations with users in person and online via our various community channels. A huge thank you to everyone in the Graylog community for helping with this huge release milestone.
Along with v2.0 GA, we are also unveiling our first commercial product, Graylog Enterprise. Built on top of the open source platform, Graylog Enterprise offers additional features that enable users to deploy Graylog at enterprise scale and apply Graylog to specialized use cases beyond log management fundamentals. The first feature of Graylog Enterprise is Archiving, with more features coming.
We’ve already seen the v1.0 series of Graylog displacing expensive, incumbent legacy solutions all over the world, and we’re confident that the v2.0 series will enable even more use cases and bring Graylog into even more data centers out there.
Now let’s dive in and check out what’s new:
THE WEB INTERFACE IS NO LONGER A SEPARATE PROCESS
We are now serving the web interface directly from graylog-server, and we have removed the extra graylog-web-interface process. The entire web interface code base has been rewritten with modern react.js technology, and many bugs or shortcomings were removed during that process.
With this unification of Graylog’s web interface and server processes, plugins now have the ability to include user interface components. The Graylog web interface becomes extensible by plugins, opening up endless possibilities for customization. You now have the power to make use of everything that Graylog has to offer today, while extending it with both back-end and front-end features at the same time. Many of the new Graylog features are being written as plugins, so you can be sure that the workflow of writing plugins will be rock solid. The documentation for writing your own plugins is available here.
WE NOW SUPPORT ELASTICSEARCH V2.X
Go ahead and upgrade to Elasticsearch v2.x to take full advantage of all the new goodness. Previous versions of Elasticsearch are no longer supported, and it is important to know that Elasticsearch does not support a downgrade. Test Graylog v2.0 before going to production, as there will be no easy rollback.
LIVE TAIL
The much requested tail -f feature enables you to see what is being written to Graylog in real time. You can now configure your latest search results to reload automatically in 1 second to 5 minute increments, or any other time increment you want. Follow your latest log activities, like errors and exceptions after a deployment of your application, without having to hit the reload button all the time.
MESSAGE PROCESSING PIPELINE
Graylog has always provided several ways of processing log messages, like extractors and drools, but sometimes it’s been a bit confusing to use or simply not flexible enough. We now provide you with a more powerful solution that suits all of your processing needs. The new Message Processing Pipeline allows greater flexibility in routing, blacklisting, modifying, and enriching messages as they flow through Graylog.
The Message Processing Pipeline allows you to write custom rules and combine these rules in different processing pipelines that can transform incoming messages in almost any way you want. It will also be possible to write plugins extending the set of functions you can use in rules, meaning that you will be able to extend the system to suit your specific needs. Please take into account that this is the first version of the processor and we are still working heavily on it, so things may break or not work exactly as expected.Documentation can be found here.
MAP WIDGET
This release adds the long awaited map widget and GeoIP resolver plugin to Graylog. Now you can enrich your messages with GeoIP data and visualize the results on an interactive map.
ARCHIVING
Graylog currently enables you to configure a retention period and automatically deletes older messages – this is to help you control the costs of storage in Elasticsearch. But we know it’s not ideal deciding between keeping less messages in Graylog or paying more for hardware. Additionally, many of our customers are required to store data for long periods of time due to compliance requirements like PCI or HIPAA.
With the new Archiving functionality, Graylog now enables you to archive log messages until you need to re-import them into Graylog for analysis. You can instruct Graylog to automatically archive log messages to compressed flat files on the local filesystem before retention cleaning kicks in and messages are deleted from Elasticsearch. Archiving also works through a REST call or the web interface if you don’t want to wait until retention cleaning to happen. We think flat files are great in this scenario, because they are vendor agnostic so you will always be able to access your data.
You can then do whatever you want with the archived files: move them to cheap storage, write them on tape, or even print them out if you need to! If you need to search through archived data in the future, you can move any selection of archived messages back into the Graylog archive folder, and the Graylog web interface will enable you to temporarily import the archive so you can analyze the messages again in Graylog.
Archiving will be the first feature in Graylog’s new commercial product, Graylog Enterprise, meaning that it’s considered an add-on to the open source core that users will have to pay to use. Read more about the why and how in this blog post.
Get a free trial and learn more about Graylog Enterprise here.
Collector sidecar
We want to make your experience sending messages to Graylog as easy and flexible as possible. That is why we are introducing the new Collector Sidecar. The Sidecar runs next to your favorite log collector (like fluentd or nxlog) and configures it for you. Enjoy central configuration from the Graylog web interface, as well as no more tinkering with configuration files.
Those of you sending logs to Graylog through NXLog can start using the Collector Sidecar to manage its configuration from within the Graylog Web Interface. This first version only supports NXLog with a file input and a Windows EventLog input, but we will extend the Collector Sidecar to support multiple input types and log collector backends in the future (e.g. rsyslog).
The Graylog Collector Sidecar runs on Linux, Windows and Mac OS X. You can download a DEB package and a Windows installer from the repository release page for now. We have also listened to user feedback and wrote it in Go instead of Java. This means you’ll download a native binary instead and do not have to install any runtime dependencies like a JVM.
Streams filter
Some of you manage tens or hundreds of streams, and sometimes it’s difficult to find the stream you’re looking for among all the options. We have added a new search filter on the Streams page that lets you find streams by title or description.
Search surrounding messages
The new Show Surrounding Messages feature, one of our most requested features, allows you to investigate the events surrounding a given log message that was logged in the same context. This helps provide more context around a search result. What happened before and after this event? What else was happening on this host at the same time? Think of this like a grep -C [X] to find surrounding lines of a known line.
Query range limit
This adds a configuration option to limit the time range for searches. If you manage a team, some of your users on Graylog systems might be unintentionally overloading your Elasticsearch clusters by executing resource-intensive searches over an extended timeframe. If you set the query range limit to 1 month, for example, no one will be able to search for logs that are older than one month.
Go to System/Configurations in the navigation to configure your query range limit.
Configurable query ranges
The relative time range options for search in Graylog have always been fixed, and the maximum fixed time range was up to the last 30 days. But we had users who wanted to execute searches over longer time frames or other customized time frames. With the introduction of configurable query ranges, you can now change or delete the default values and also add new ones.
Go to System/Configurations in the navigation to configure your default time range options.
BUGFIXES AND IMPROVEMENTS
We’ve been tracking squashed bugs and various improvements in each release blog for v2.0 alpha, beta, and RC. Here is a list of bugfixes since v2.0-RC.1:
SERVER
DOWNLOAD IT NOW
Graylog v2.0 GA can be downloaded from here.
Our virtual appliance in OVA format has also been updated for this release.
The operating system packages for v2.0 GA are available in our repositories. See our documentation for details.
Docker images are available on Docker Hub.
UPGRADING FROM GRAYLOG 1.X TO 2.0.X
Please refer to the instructions in our UPGRADING document.
WE NEED YOUR FEEDBACK
We’re super excited about releasing v2.0 GA, and we value your feedback. There are a variety of ways to provide feedback, all of which can be found on our community resources page:
Go forth, try out this release, and let us know what you think!
