SyslogNG
From EGEE-see WIki
Contents |
Introduction
A Logging facility is a mandatory part for every computer system. In typical Unix and Unix-like systems the application that provides support for system and kernel logging is syslog. Although syslog is ubiquitous and robust, it lacks several features that are needed from most modern Unix systems. [http://www.balabit.com/products/syslog_ng/ Syslog-ng] (syslog new generation) is an attempt to provide a better logging infrastructure for Unix systems. This document describes the setup of the syslog-ng facility in the HG-01-GRNET site.
Installation
To compile syslog-ng for your system you need to download the latest version of syslog-ng from here. You will also need the Support library for syslog-ng (libol) which can be found here. An example for a typical syslog-ng installation follows:
# cd libol-0.3.16 # ./configure && make # cd ../syslog-ng-1.6.8 # ./configure --with-libol=../libol-0.3.16 && make && make install
Configuration
General Setup
There are two hosts in the HG-01-GRNET site that are using syslog-ng: a primary log host (loghost) and a backup log host (backuphost).
The site's logging information is stored centrally on these two hosts. All other hosts (worker nodes, storage elements, user interface etc) forward their logs to loghost using the classic syslog facility. The logs that are gathered at loghost (both local and remote) are stored locally. For security reasons these logs are also forwarded to backuphost.
Configuration File
A stripped-down version of the configuration file used for the aforementioned setup is presented bellow:
Options
This section of the configuration file is used to used to set the global options of syslog-ng. A full list of the possible options can be found in the syslog-ng manual.
#### global options
options {
create_dirs(yes); # if a dir does not exist create it
owner(root); # owner of created files
group(root); # group of created files
perm(0600); # permissions of created files
dir_perm(0700); # permissions of created dirs
};
Sources
Sources are an abstraction of a collection of various methods that are used to gather log messages. These methods are called "source drivers" in syslog-ng terminology. A list of supported source drivers can be found here. In our configuration we define only one source, which is named s_all.
#### sources
source s_all {
udp(); # remote logs arriving at 514/udp
unix-stream("/dev/log"); # local system logs
file("/proc/kmsg"); # local kernel logs
internal(); # internal syslog-ng logs
};
Filters
Filters are expressions that can be used to route log messages from various sources to destinations. A list of the supported filter functions can be found here.
Host Filters
Using host filters we can filter information based on the host that they were created. Note that it's possible to use regular expressions in the various filters. In our setup we define 8 different categories of hosts.
#### host filters
filter f_host_loghost { host("loghost"); }; # loghost
filter f_host_wn { host("wn[0-9][0-9]"); }; # WNs (wn01,...)
filter f_host_se { host("se[0-9][0-9]"); }; # SEs(se01,...)
filter f_host_ui { host("ui01"); }; # UI
filter f_host_ce { host("ce01"); }; # CE
filter f_host_mon { host("mon"); }; # Mon
filter f_host_lfc { host("lfc"); }; # LFC
filter f_host_rb { host("rb"); }; # RB
Program Filters
Using Program Filters we can filter information based on the program which generated the log message. We define two filters, so we can keep all the generated output of various grid related programs in one place.
#### program filters
# grid programs
filter f_prog_grid {
program("GRAM") or
program("gridinfo") or
program("gridftpd");
};
# edg programs
filter f_prog_edg { program("edg-"); };
Destinations
Destinations are consumers of log messages. As with sources, there are various "destination drivers" supported by the syslog-ng application.
File Destinations
File destinations allow to output log messages in simple files. The destination filename may include macros which get expanded when the message is written. In our configuration we use the $FACILITY macro to sort log messages based on the facility they were tagged with.
destination d_loghost_facility {
file("/var/log/ng/loghost/$FACILITY.log");
};
destination d_ui_facility {
file("/var/log/ng/grid/ui/$FACILITY.log");
};
## here are placed similar entries
## for WNs, SEs, CE, MON, LFC, RB
## (...)
destination d_gridprogs {
file("/var/log/ng/grid/gridprogs.log");
};
destination d_edgprogs{
file("/var/log/ng/grid/edgprogs.log");
};
destination d_fallback{
file("/var/log/ng/fallback.log");
};
Network Destinations
Network destinations allow us to forward messages to remote hosts using UDP or TCP. In our setup we define the backuphost as a destination.
destination d_remote_backuphost {
udp("backuphost");
};
Log Paths
Log paths are basically triples of sources, filters and destinations. Any message coming from any of the listed sources, matching the all the filters are sent to all listed destinations. In our setup we first forward the messages to backup host and then we store it to the appropriate file.
# forward logs to backuphost
# This must be done first because of the final flags
# in all other log actions.
log {
source(s_all);
destination(d_remote_backuphost);
};
# log actions for grid programs for all hosts.
# The final flag guaranties that these logs will
# not be stored twice.
log{
source(s_all);
filter(f_prog_edg);
destination(d_edgprogs);
flags(final);
};
log{
source(s_all);
filter(f_prog_grid);
destination(d_gridprogs);
flags(final);
};
# loghost logs
log{
source(s_all);
filter(f_host_loghost);
destination(d_loghost);
flags(final);
};
# Grid hosts logs
log{
source(s_all);
filter(f_host_ui);
destination(d_ui_facility);
flags(final);
};
## here are placed similar entries
## for WNs, SEs, CE, MON, LFC, RB
## (...)
# this will identify any incoming logs that were
# ignored.
log{
source(s_net);
destination(d_fallback);
flags(fallback);
};
Misc
Init Scripts
If your distribution does not provide a binary package for syslog-ng then you also need to configure the proper init scripts. In most of the cases you can use the syslog init script that comes with your distribution with some minor modifications.
Log File Rotation
After setting up syslog-ng you must also configure the log rotation of the various log files. One of the most common utility that is used for this purpose is logrotate. If this program is installed on your system you can add a syslog-ng file at the /etc/logrotate.d/ directory that will handle the rotation of your logs. An example file based on the previous setup is:
/var/log/ng/wn/*.log
/var/log/ng/se/*.log
/var/log/ng/grid/ui/*.log
/var/log/ng/grid/ce/*.log
/var/log/ng/grid/mon/*.log
/var/log/ng/grid/lfc/*.log
/var/log/ng/grid/rb/*log
/var/log/ng/grid/gridprogs.log
/var/log/ng/grid/edgprogs.log
/var/log/ng/loghost/*.log
{
weekly
missingok
compress
create 600 root root
rotate 8
sharedscripts
postrotate
if [ -f /var/run/syslog-ng.pid ]; then \
kill -HUP `cat /var/run/syslog-ng.pid`; \
fi;
endscript
}
