Log Collection & Fleet Management

Efficient log collection and fleet management are essential for managing your logs, retention size, and storage constraints. Our platform integrates common log shippers like NXLog, Winlogbeat, Filebeat, and Metricbeat for seamless log gathering. Graylog ensures flexibility in handling diverse log formats by supporting various input types, including Syslog, CEF, GELF, BEATS, JSON, IPFIX, Netflow, and Plain text. Additionally, many custom integrations are included for many different sources.

Log Collection and Fleet Management

How It Works:

Log Collection and Fleet Management screenshot

Each log shipper is configured to collect specific logs from their respective sources. For example:

NXLog: For its versatility in collecting logs from various sources and its ability to transform and relay logs in different formats from multiple operating systems.

Winlogbeat: Specialized in forwarding Windows event logs with minimal impact on system performance, keeping your Windows infrastructure transparent and secure.

Filebeat: Ideal for monitoring log files, collecting log events, and forwarding them to our platform for analysis. Its lightweight nature ensures efficiency and reliability for shipping logs from multiple operating systems and devices.

Metricbeat: An application used to ship metrics from operating systems from the services running on those servers.


Input Types

To accommodate the diverse nature of log data, our platform supports a wide array of input types, ensuring maximum flexibility and compatibility:

  • Syslog: The standard for message logging, widely used in network devices and Unix-based systems.
  • CEF (Common Event Format): An open log management standard, ideal for integrating security information from different devices into a single format.
  • GELF (Graylog Extended Log Format): Specifically designed for Graylog, this format is ideal for structured logging and avoiding the shortcomings of Syslog.
  • BEATS: Lightweight data shippers, part of the Elastic Stack, designed for forwarding logs and metrics to Elasticsearch or Logstash.
  • JSON: A highly interoperable format that facilitates structured logging, making logs more readable and easier to analyze.
  • IPFIX & Netflow: Protocols designed for monitoring network traffic flow, essential for network performance and security monitoring.
  • Plain Text: Ensures compatibility with the most straightforward log formats, offering unparalleled flexibility.

These inputs allow for the ingestion of logs from a myriad of sources, ensuring that no critical data is left behind.

Graylog Sidecar

Graylog Sidecar is a configuration management utility that manages and configures log collectors, or sidecar collectors, from a central server. It serves as an intermediary between the server and collectors like Filebeat, Winlogbeat, or NXLog, facilitating centralized configuration management without manual adjustments on each host. Benefits include:

Centralized Configuration Management: Graylog Sidecar simplifies the management of log collector configurations across multiple hosts from a single Graylog interface. This eliminates the need to manually access each server to configure or update log collectors, saving time and reducing the potential for configuration errors.

Scalability: As infrastructure grows, managing configurations for log collectors on each new server can become increasingly complex. Graylog Sidecar allows for scalable log collection management, making it easier to maintain a growing infrastructure.

Consistency and Compliance: Ensuring consistent configuration across all log collectors is crucial for compliance and effective log management. Graylog Sidecar helps maintain uniform configurations, ensuring that all log data is collected and processed in a consistent manner.

Flexibility: Graylog Sidecar supports various log collectors, providing the flexibility to choose the most suitable collector for specific log sources or environments. This allows for optimized log collection tailored to the needs of different systems.

Automated Updates: Administrators can push configuration changes and updates through the Graylog interface, and Graylog Sidecar will automatically apply these changes to the connected log collectors. This automation helps in maintaining up-to-date configurations without manual intervention.

Enhanced Monitoring and Control: With Graylog Sidecar, administrators have better visibility into the status and health of log collectors across their environment. This improved monitoring capability allows for quicker responses to issues and more effective control over log collection processes.

In summary, Graylog Sidecar is used to enhance the efficiency, consistency, and scalability of log collection management within an infrastructure, streamlining the process of collecting, managing, and analyzing log data across multiple servers and devices.

Learn More About Log Collection & Fleet Management

Our platform integrates inputs for NXLog, and Beats data shippers for logging Windows and Linux logs. This enables seamless logging of data into Graylog. Graylog Sidecar can manage your log levels and configuration centrally for these log shippers making it easier to manage.

The system supports various input types, including Syslog, CEF, GELF, BEATS, JSON, IPFIX, Netflow, and Plain text. There are many custom integrations that are included directly with Enterprise and Illuminate built-in content.

Incoming logs are processed through Pipelines and rules, where they can undergo a transformation for enrichment, and parsing based on defined criteria into a schema. Pipelines can be used to lookup threat intel as well as enhance the logs with pertinent asset data as well as GeoIP information.  The Graylog information model schema is applied for security analytics and threat with curated alerts as well as Sigma rules and anomaly detection.

Graylog’s Sidecar Fleet Management simplifies the management of log collection agents across distributed environments, providing centralized control over configuration, deployment, and updates. The Sidecar feature gives you one central location to configure profiles for the different logging levels you may require for your different assets for Workstations, Servers, and equipment.

Pipelines and streams facilitate efficient log processing. Pipelines transform and enrich logs, while streams enable organized data management and granular control over log routing. Streams separate your log types based on criteria so you can have different retention periods for different types of data.  This gives you the flexibility to keep logs longer for compliance and trends over time.

By leveraging Graylog’s technologies, our platform ensures efficient log processing, enabling organizations to gain valuable insights and enhance operational efficiency.