Remote Logging with Rsyslog
RSYSLOG is the rocket-fast system for log processing. After syslog, now rsyslog comes pre-built with the Linux systems, meant for local and remote logging. In any system, you will want to (a) log the system and application logs on the local machine, and/or (b) log the system and application logs to a remote machine.
Below given are 2 cases, useful for forwarding OS logs and application logs:
Forwarding only OS logs: #
Add the below given line at the bottom of the /etc/rsyslog.conf file, and later restart the rsyslog service -
*.info;authpriv.*;cron.*;mail.* @remote_ip:514
By default, rsyslog uses port number 514 for its activities. If the logs need to be forwarded through UDP, mention a single ‘@’ before the remote_ip, and for TCP, mention ‘@@’ before the remote_ip.
*.info – all logs with info severity
authpriv.* – all logs related to authorization and privileges
cron.* – all logs related to cron – scheduled jobs
mail.* – all logs related to mail and mail servers
Forwarding OS and Application logs: #
# Add the following module - it is the module for forwarding logs from a file.
# Add this along with the other $ModLoad tags at the top of the file
$ModLoad imfile
# Add 'local7.none' to the below line as shown below.
# This will stop the logging of local7 messages in /var/log/messages, as we need to forward our application logs through local7 service
*.info;mail.none;local7.none;authpriv.none;cron.none /var/log/messages
# Comment the local7 for boot logs, to stop logging the application logs to /var/log/boot.log which we are forwarding through local7 service
#local7.* /var/log/boot.log
# Add the below lines to forward the logs from their respective files. First 3 lines are variable, the other 2 are static.
# $InputFileName takes the path to log file (absolute path of the file)
# $InputFileTag will attach the mentioned tag (here: tag_jio.com) to the original log
# $InputFileStateFile is the State file where the logs are stored before forwarding (for eg. useful in case of network failure)
$InputFileName /path/to/log/file
$InputFileTag tag_website.com:
$InputFileStateFile buffer_file_name
$InputFileFacility local7
$InputRunFileMonitor
# Add this line at the bottom of the file, for forwarding
# local7.* (all logs of local7 - application),
# *.info (all logs with info level),
# authpriv.* (all logs of authorization-privilege) and
# cron.* (all logs of cron)
# - to the receiver IP and Syslog port 514.
# Add '@' for sending logs through UDP, '@@' for TCP.
local7.*;*.info;authpriv.*;cron.* @receiver_IP:514
(Above given configuration is for Red Hat based systems only. It may differ in Debian based systems.)
Common Troubleshooting Steps: #
- Check network connectivity between the sender and receiver – Firewall port opening (Port: 514 – TCP/UDP), Ping, Traceroute
- Check if logs are present at the mentioned log file path
- Check the ‘space’ and ‘semicolon’ in the rsyslog configuration file
- Change the $InputFileStateFile’s value to something else (eg. change buffer_file_name buffer_file_name to buffer_file_name_1)
- Restart the rsyslog service