When you’re adding or subtracting fractions, you need to make sure that they have a common denominator, a number that allows you to compare values. In the same way, your IT environment needs a common “language” for your event log data. Your environment consists of various devices running different operating systems, software, and firmware. However, to understand what’s happening in your environment and respond to service requests, you need a common denominator to help you aggregate, correlate, and analyze log data.
Syslog is the common logging protocol that can help you gain the observability necessary for responding to service requests as quickly as possible.
What Is Syslog?
System logging protocol (Syslog) is a standard logging protocol that simplifies log message formats and standardizes them across diverse device types. The open-ended protocol acts as a common denominator across:
- Operating systems: Linux, Unix, macOS
- Firmware: routers, switches, firewalls, Internet of Things (IoT)
- Software: databases, media, browsers
Syslog consists of three layers:
- Content: information in the event message
- Application: generates, routes, interprets, and stores messages
- Transport: sends the message
It’s important to remember that Syslog is a protocol, meaning that it extracts a log’s elements then gives you a way to standardize how the data is put back together.
How does Syslog work?
The Internet Engineering Task Force (IETF) formally documented the protocol in its 2009 RFC 5424. At a very high level, Syslog requires:
- Originator: generates the syslog content that will be in the message
- Collector: ingests the syslog content for further analysis
- Relay: sends forwarded messages from originators and other relays to the collector or other relays
- Transport sender: hands off messages to a defined transport protocol
- Transport receiver: accepts the messages from the transport protocol
- Workstation is the originator, generating the machine data that ends up in the message.
- Using the User Datagram Protocol (UDP) as the relay, the machine data travels to the collector, a Syslog server.
- The Syslog server ingests the data and breaks it into the chosen pieces, parsing it.
- Using a Transport Layer Security (TLS) client-server, the syslog messages pass between the sender and receiver encrypted in transit.
Syslog gives you a way to format and parse some device data, but it’s not compatible with all devices. The protocol has a clear set of rules about how a log should look, but many devices can’t comply with these requirements. To overcome this, you can create a custom message input and parser for each format that differs from Syslog, but this can become time consuming.
Why is Syslog important
Syslog enables you to standardize the message format across diverse software, operating systems, and firmware. By breaking the machine data into its pieces and then putting it all back together in the same order, Syslog enables you to aggregate, correlate, and analyze data from across the environment.
Every Syslog message has the same format:
- Header
- Structured data
- Message
Header
The header can include any of the following information:
- Priority Value (PRIVAL): the Facility and Severity
- VERSION: Syslog protocol version
- HOSTNAME: machine and domain name that originally sent the message
- APP-NAME: Device or application originating the message
- PROCID: Process name or process ID associated with syslog system
- MSGID: Type of message
- TIMESTAMP: hour and minute
Structured Data
The structured data expresses information in a defined, easily parseable, interpretable format and can include:
- SD-ELEMENT: name and parameter value pair
- SD-ID: uniquely identify type and purpose of the SD-ELEMENT
- SD-PARAM: name (PARAM-NAME) and value (PARAM-VALUE)
Message
This is a free-form message providing information about the event that should be in Unicode but may use other encoding if the Syslog application can’t encode the message in Unicode.
Jan 09 19:00:01 localhost graylog-router: *Jan 10 13:58:40.411: %SYS-5-CONFIG_I: Configured from console by console
Jan 09 19:00:01 localhost graylog-router: *Jan 10 13:58:46.411: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.1.239 stopped – CLI initiated
Jan 09 19:00:01 localhost graylog-router: *Jan 10 13:58:46.411: %SYSMGR-STANDBY-3-SHUTDOWN_START: The System Manager has started the shutdown procedure.
Jan 09 19:00:01 localhost graylog-router: *Jan 10 13:58:46.411: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.1.239 started – CLI initiated
Jan 09 19:00:01 localhost 0.0.0.0: *Jan 10 13:59:12.491: %SYS-5-CONFIG_I: Configured from console by console
What Is a Syslog Server?
The Syslog server collects and stores all the messages in a central location, making them usable by giving you a way to view and filter them. Whether it’s a physical server, virtual machine, or software, the Syslog server should at least consist of:
- Syslog listener: ingests and gathers Syslog data
- Database: store Syslog data for reference
If you want a more robust Syslog server, you can look for something that enables you to:
- Login from anywhere using the internet
- Configure alerts
- Set up responses, like forwarding messages or logging to a file
- Scheduling reports
Syslog vs. Windows Event Logs
Although you use Syslog and Windows Event Logs in a similar way, it’s important to understand their differences.
Windows Event Logs provide a subset of Syslog information focused on specific data that you can use to troubleshoot system, application, device, or security issues. However, Microsoft uses a proprietary format that isn’t the same as the Syslog format. Further, Windows Event Logs only usually provide the following severity levels:
- Critical: Demand immediate attention, usually at a system-wide level
- Error: Problems that don’t require immediate action
- Warning: Problems that could lead to critical errors later
- Informational: Noncritical information
- Verbose: Progress or success messages
Meanwhile, Syslog has eight severity values:
- 0 Emergency: system is unusable
- 1 Alert: action must be taken immediately
- 2 Critical: critical conditions
- 3 Error: error conditions
- 4 Warning: warning conditions
- 5 Notice: normal but significant condition
- 6 Informational: informational messages
- 7 Debug: debug-level messages
To correlate your Windows Event Logs with Syslog formatted events, you can use a third-party utility that creates a Syslog formatted event and forwards them to the Syslog server.
Graylog Operations: All the log data, one unified format
Purpose-built for modern log analytics, Graylog Operations removes complexity from your daily activities, giving you the real-time log view you need that ensures continued availability and streamlines investigations. Using Graylog Cloud Platform, you can maximize the value of your resources, getting the flexibility needed to add new data sources, processing rules, search templates, alerts, visualizations, reports, and unlimited users.
To get an overview of Graylog, including a 20-minute in-depth demo and 10-minute Q&A session, contact us today.
