Overview

• As a company's business expands, the number of servers that need to be managed increases. From the server administrator's perspective, it is necessary to monitor each server's situation frequently, as servers typically store various kinds of log messages in a local environment from multiple applications. If these log messages can be transmitted to a central log repository with minimal overhead in real time for centralized monitoring, management would be much easier. This article outlines the method of collecting logs to a remote Graylog server in Syslog UDP format using RSYSLOG.

Log Message Transmission Flow

• Each server where logs originate can configure through the RSYSLOG service which local logs to send to which server. In the example below, logs will be transmitted to the Graylog server using the Syslog format over the UDP protocol.

• The Graylog server can receive and store various message formats, including Syslog. In the example below, a Syslog UDP input will be created to listen on port 1541.

• Once logs are collected at the central Graylog server, it is possible to query, filter, and visualize them almost in real time.

Creating Graylog Syslog UDP Input

• To receive Syslog UDP log messages in Graylog, the following Syslog UDP Input must be created.

Graylog Web Interface
→ System → Inputs
→ Select Input: [Syslog UDP] → [Launch new input]
# Launch new Syslog UDP input
→ Check [Global]
→ Title: SYSLOG_UDP_INPUT
→ Port: 1514
→ Check [Store full messages?]
→ [Launch Input]

RSYSLOG Installation

• Using RSYSLOG, logs can be sent to other servers, or can be received from other servers. Below is the installation procedure.

# Install RSYSLOG on RHEL, CentOS, Amazon Linux
$ sudo yum install rsyslog systemd -y
$ sudo systemctl start rsyslog
$ sudo systemctl enable rsyslog

RSYSLOG Configuration

• Logs created in the local environment of the operating system can be transmitted in real time to the remote Graylog server using Syslog UDP. Below are the settings and restart the service after configuration.

$ sudo nano /etc/rsyslog.conf
$PreserveFQDN on
$template GraylogFormat,"<%PRI%>%TIMESTAMP% %HOSTNAME% %programname% %syslogtag%%msg%"
*.* @{graylog-host}:{graylog-syslog-udp-port};GraylogFormat
& stop

$ sudo systemctl restart rsyslog

Verification of Transmitted Log Messages

• The log messages sent to Graylog are as follows. Logs matching a keyword can be found using message:{keyword} or by searching for logs containing a specific keyword with message:/.*{keyword}.*/.

#timestamp
2024-04-19 12:00:00.000

#source
ip-100-101-102-103.ap-northeast-2.compute.internal

#message
ip-100-101-102-103.ap-northeast-2.compute.internal systemd systemd:Starting System Logging Service...

#full_message
<30>Apr 19 12:00:00 ip-100-101-102-103.ap-northeast-2.compute.internal systemd systemd:Starting System Logging Service...

References

• RSYSLOG Properties