We are excited to announce a new release of Graylog, version 4.3. This release is also the launching point for the general availability of Graylog Security and the renaming of Graylog Enterprise to Graylog Operations. The core platform shared by Security and Operations will be referred to as the Graylog Platform.
DOWNLOAD LINKS
TARBALLS (MANUAL INSTALLATION):
- Graylog Server
- Graylog Operations Server
- Graylog Operations Plugin
- Graylog Integrations Plugin
- Graylog Operations Integrations Plugin
GRAYLOG FORWARDER
DOWNLOAD LINKS
TARBALL (MANUAL INSTALLATION):
OS PACKAGES
DOCKER IMAGE
IMPORTANT NOTE
- We caution you not to install or upgrade Elasticsearch to 7.11 and later! It is not supported. If you do so, it will break your instance!
- We now support OpenSearch v1.1, 1.2, or 1.3. Graylog Security customers must use OpenSearch v1.2 or 1.3.
CORE
Pre-flight Checks for Elasticsearch/OpenSearch and MongoDB Versions
When Graylog starts up, it now performs connectivity and version checks for MongoDB and Elasticsearch/OpenSearch. This will particularly allow users who initially install Graylog or who upgrade to a new Graylog release to verify their instances are compatible with these services.
By default, Graylog will wait indefinitely until it can reach MongoDB and Elasticsearch/OpenSearch. Once the connections are established, it will assert the versions for compatibility. If those fail, Graylog aborts the startup without running any further actions, e.g. migrations, periodicals, etc.
This change introduces two configuration flags:
- skip_preflight_checks which is false by default
- mongodb_version_probe_attempts which is 0 (indefinitely) by default.
The preflight check will also assert that the message_journal_dir is writable and has enough space to contain message_journal_max_size for the on-disk journal. Additionally, it will also apply a check to prevent the journal from using a network filesystem.
PROCESSING PIPELINE IMPROVEMENTS
We added several enhancements to Graylog’s processing pipelines:
- Visual Indicator of Where Pipeline Rules are applied – This UI update shows pipelines using a rule on the Rules list pages, increasing visibility.
- Stop processing further stages with drop_message – Improves performance and metrics accuracy with drop_message.
- JSON parsing: add shallow parsing up to a certain path depth – Adds a new optional parameter called depth. This parameter specifies the maximum parse depth and ignores any nodes that are more deeply nested.
- Add Support for parsing JSON Arrays in Processing Pipelines which introduces a new function: flatten_json.
Configurable Time Zones for Event Notifications
This update allows you to choose a custom time zone for recipients when configuring email notifications.
Manually Stopped Inputs Will No Longer Restart Automatically
4.3 allows for users to maintain the desired state of an input in the DB and avoid restarting inputs where the desired state is STOPPED.
Users can choose the legacy behavior by setting a flag in the graylog.conf: auto_restart_inputs = true.
Early Warnings for License Violations (Operations)
We now provide early license violation warnings by showing a clear notification banner in the UI. In addition, you can choose to receive license violation warnings via email and distribute to a configurable list of subscribers.
Additionally, the traffic graph in System / Overview now shows the license limit as a line so that it may be easily compared with the amount of data being used.
SEARCH
Search Query Validation
In the Graylog UI, you can now validate the search query before a user submits the search. Warnings and errors will appear in real-time as you enter characters in the search bar. In addition, a box appears with a note identifying the respective warning or error.
Search Query Field Name Suggestions
The search query auto-completion offers suggestions in different scenarios. For example, if you write a field name in the search bar like http_method, related suggestions appear, e.g. GET, POST, etc.
Multi-line Search Query
Search query strings can be lengthy. Now it’s possible to define multi-line search queries. The update search bar breaks into a second line when reaching a maximum character limit on the first. Also, you can use a keyboard combination of SHIFT and ENTER to enable line breaks.
Search API Versioning
To facilitate implementing breaking changes, we created versioning for our search API. A different version can be used by changing the `Accept/Content-Type` header of a request. Note that there is currently no need for users to change their API scripts.
Reporting Improvements (Operations)
- Time zone option – It is now possible in the UI to define a localized time zone for a report, increasing the relevance of the data to the receiver.
- Edit links for entities in reports – A link is now available for each dashboard, page, and widget that redirects you to the related edit page.
- Changes to `report_render_engine_port` setting – Prior to this 4.3 this report value was set to 9515, by default. Starting with 4.3, the engine will bind to any random free port. If you prefer port 9515, you’re required to set it explicitly.
- Hourly interval – You can now schedule Graylog to send reports every hour.
INTEGRATIONS (OPERATIONS)
Geolocation Processor Updates
The Geolocation processor is now even faster with a couple of updates. First, you now have the ability to set the processor to use both MaxMind and IPInfo databases for greater flexibility in your environment. Second, you can enable it for only fields in our Graylog Schema and Graylog Illuminate. This maintains enriched geolocation data usability across all content created by Graylog in the future.
HTTP Gelf Input Bulk Receiving Option
A new Enable Bulk Receiving configuration option has been added to the HTTP Gelf input that supports bulk receiving of messages separated by new lines. When the option is enabled, the input will automatically separate these multiple Gelf messages, which are then newline delimited (n, rn).
User Lookup Table Overrides
Through Illuminate, Graylog distributes content and in some cases assigns values to items. For example, an item like a risk score can be attributed to a user identified as “administrator”. By utilizing lookup table overrides in 4.3, you can now make the changes you want directly via the Web UI, and it does not affect the distributed content. The lookup tables first check the user-defined data then fall back if not found in distributed lookup values. This allows you to change the risk score or add new values to the tables without having to update them for a new release.
Azure Store Full Message Field
A new Store Full Message field option has been added to the Azure Logs input, which stores the entire message payload received from Azure Logs. This allows you to manually parse the content of all Azure log message types through your processing pipelines.
GRAYLOG SECURITY
Anomaly Detection
Graylog Anomaly Detection is now a tool you can utilize in your Graylog Security product. Its primary purpose is to help you detect outliers in a dataset and get notified whenever something deviates from its usual behavior or normal levels within your log data. As a result, you and your team become empowered to navigate cyber threats proactively, identify unusual (anomalous) activities, and take steps toward mitigating anomalies within your organization.
Anomaly Detection is a powerful Artificial Intelligence / Machine Learning (AI/ML) security tool that is supported by the OpenSearch data service. Organizations can define a security baseline built on accumulating log data while hunting for anomalies in real time. This anomalous data is funneled into streamlined Graylog Security dashboards with various customizable widgets, offering users interactive and actionable analytics for event management.
CHANGELOGS
Detailed changelogs are available for Graylog and Graylog Operations.
Let us know what you’d like to have included in our GitHub issue tracker.
