You are here: Foswiki>Logging Web>SyslogInstall (06 Jun 2007, CharlesWaldman)Edit Attach

Syslog-ng Install

Syslog-ng Install

Overview

The following describes a system of two syslog-ng servers: one providing a source of logging data, the second providing a central logging host which can receive logging information from a number of syslog-ng hosts. Here is what was done to setup an initial two-host system:

Install syslog-ng on a central logging host (uct3-edge6.uchicago.edu); congfigure with the standard template for a central logging host.
Install syslog-ng on an ATLAS DQ2 site services host; configure with a socket for a specific DQ2 logfile, i.e. the subscriptions log, and forward to the central logging host.
Create a bootscripts for these hosts.
Launch services on each host by executing the boot scripts.
Validate by writing a test message into the grid/dq2 service host (into its socket), look for it on the central logging host.

Installing Syslog-ng software

Instructions for an independent syslog-ng instance on any host. Here we focus on installations which do not override or replace the systems native syslog program.

Obtain sources from http://www.balabit.com/downloads/syslog-ng/2.0/src/, put them in /opt/src/.
Then:

cd /opt/src/
tar xvzf eventlog-0.2.5.tar.gz
tar xvzf syslog-ng-2.0.2.tar.gz
cd eventlog-0.2.5
./configure --prefix=/opt/eventlog
make
make install
#
cd ../syslog-ng-2.0.2
export EVTLOG_CFLAGS="-I/opt/eventlog/include/eventlog"
export EVTLOG_LIBS="-L/opt/eventlog/lib/ -levtlog"
./configure --prefix=/opt/syslog-ng
make
make install

Configuration of the source DQ2 service host

On MWT2_UC, this is currently uct2-grid1.mwt2.org. After installing the syslog-ng software, you need a configuration file and a boot script.

This configuration file will go into /opt/syslog-ng/etc/syslog-ng.conf:


options {
   #time_sleep(50);  # polling interval, in ms (helps reduce CPU)
   time_sleep(1000);  # polling interval, in ms (make this once per second)
# Note - time_sleep(1000) does not seem to work, should this be 100 ?
   use_fqdn(yes);  # use fully qualified domain names
   ts_format(iso);  # use ISO8601 timestamps
   #
   # for normal load
   flush_lines (10); # number of lines to buffer before writing to disk
   log_fifo_size(100); 
   #
   # for heavy load
   #flush_lines (1000); # number of line to buffer before writing to disk
   #log_fifo_size(1000); 
   #
   stats_freq(3600);  # number of seconds between syslog-ng internal stats events; these are useful
                               # for ensuring syslog-ng is not getting overloaded
};
#
# /tmp/dq2logs.socket is the destination of 'tail -F dq2logfile | logger -t dq2 -u /tmp/dq2logs.socket' (See boot script below)
#
source dq2log_src { 
       unix-stream("/tmp/dq2logs.socket");
 };
#
# syslog-ng internal logs; useful for testing syslog-ng config
source syslog_ng {
       internal();
};
#
# define the Forwarding Destination
destination dq2log_dst { 
       tcp("uct3-edge6.uchicago.edu" port(5142)); 
};
destination syslog_ng_dst {
  file ("/tmp/syslog-ng.log" perm(0644) );
};
#
# forward sources to destination
log { 
   source(dq2log_src); destination(dq2log_dst); flags(flow-control);
};
# for syslog-ng debugging
log {
  source(syslog_ng); destination(syslog_ng_dst);
};

Boot script for the DQ2 service host

Capture and forward dq2 subscription logfile (subscriptions.log).


#! /bin/sh
#
# Source function library
. /etc/rc.d/init.d/functions
#
BINDIR=/opt/syslog-ng/sbin
syslog=syslog-ng
config=/opt/syslog-ng/etc/syslog-ng.conf
syslog_pid="/var/run/syslog-ng.pid"
#
# check config and programs
test -s ${config}         || {
    echo 1>&2 "${config} does not exist"
    if test "$1" == "stop" ; then exit 0 ; else exit 6 ; fi
}
test -x ${BINDIR}/$syslog || {
    echo 1>&2 "${BINDIR}/$syslog is not installed"
    if test "$1" == "stop" ; then exit 0 ; else exit 5 ; fi
}
#
case "$1" in
   start)
#       startproc -u nobody -p ${syslog_pid} ${BINDIR}/${syslog} -f $config
       daemon ${BINDIR}/${syslog} "-f $config"  
       echo  "Starting syslog-ng service"
       # create streams for log files here; note dq2 is a tag for the source.
       tail -F /opt/ddm/config/MWT2_UC/subscriptions.log | logger  -t dq2 -u /tmp/dq2logs.socket > /tmp/syslog-ng-tailer.log 2>&1 &
       ;;
   stop)
       echo "Shutting down syslog-ng service"
       # killproc ${syslog_pid} TERM
       killproc ${syslog} TERM 
       ;;
   restart)
       echo "restarting syslog-ng service"
       $0 stop
       $0 start
       ;;
     *)
       echo "Usage: $0 {start|stop|restart}"
       exit 1
       ;;
esac

Boot script for the DQ2 service host (Patrick's version)

Reflects changes in to the init.d script so that:

We can use chkconfig/service to manage the init.d script
The syslog-ng service runs as user nobody
The tail command runs as user nobody


#! /bin/sh
#
# chkconfig: 345 99 99
# description: syslog-ng-ddm v2
### BEGIN INIT INFO
# Provides: syslog-ng-ddm
# Default-Start: 3 4 5
# Default-Stop: 1 2 6
# Description: syslog-ng
#
#
# Source function library
. /etc/rc.d/init.d/functions
#
BINDIR=/opt/syslog-ng/sbin
syslog=syslog-ng
config=/opt/syslog-ng/etc/syslog-ng.conf
syslog_pid="/var/run/syslog-ng.pid"
#
# check config and programs
test -s ${config}         || {
    echo 1>&2 "${config} does not exist"
    if test "$1" == "stop" ; then exit 0 ; else exit 6 ; fi
}
test -x ${BINDIR}/$syslog || {
    echo 1>&2 "${BINDIR}/$syslog is not installed"
    if test "$1" == "stop" ; then exit 0 ; else exit 5 ; fi
}
#
case "$1" in
   start)
       daemon --user nobody ${BINDIR}/${syslog} "-f $config"  
       echo  "Starting syslog-ng service"
       # create streams for log files here
       # Note: should check if tail's are already running before starting up them up
       su nobody -s /bin/bash -c "tail -F /opt/dq2_0.2.12/config/UTA_SWT2/subscriptions.log | logger  -t dq2 -u /tmp/dq2logs.socket > /tmp/syslog-ng-tailer.log 2>&1 &"
       ;;
   stop)
       echo "Shutting down syslog-ng service"
       # killproc ${syslog_pid} TERM
       killproc ${syslog} TERM 
       ;;
   restart)
       echo "restarting syslog-ng service"
       $0 stop
       $0 start
       ;;
     *)
       echo "Usage: $0 {start|stop|restart}"
       exit 1
       ;;
esac

Configuration of central logging host

This will be uct3-edge6.uchicago.edu. * /working/syslog-ng/dq2/logfiles - this can contain, eg., dq2 related logfiles from all dq2 site services instances.

This configuration file will go into /opt/syslog-ng/etc/syslog-ng.conf:


options {
   time_sleep(50);  # polling interval, in ms (helps reduce CPU)
   create_dirs(yes);  # create output directories
   use_fqdn(yes);  # use fully qualified domain names
   ts_format(iso);  # use ISO8601 timestamps (syslog-ng 2.0 only)
   #
   # for normal load
   flush_lines (10); # number of line to buffer before writing to disk
   log_fifo_size(100);
   #
   # for heavy load
   #flush_lines (1000); # number of line to buffer before writing to disk
   #log_fifo_size(1000);
   flush_timeout(500); # in ms
   #
   stats_freq(3600);
};
#
#
# define the source: any host sending to port 5142
source network {
   tcp(port(5142) max-connections(500));
   internal();  # internal syslog-ng messages
};
#
#
# Define the destination, automatically creating new directories
#    for each month and new host.
destination dq2logs {
     file ("/working/syslog-ng/dq2/logfiles/$YEAR.$MONTH/dq2.$HOST.log"
           perm(0644) dir_perm(0755) create_dirs(yes)
          template("$ISODATE $HOST $MSG\n") );
};
#
#
log { source(network);
     destination(dq2logs); flags (flow-control);
};

Bootscript for the central logging host

Note - this is a special central logging host - it also hosts OSG-VTB gatekeeper services, so we may also collecting globus-gatekeeper, gridftp, and container logfiles.


#! /bin/sh
#
# Source function library
. /etc/rc.d/init.d/functions
#
BINDIR=/opt/syslog-ng/sbin
syslog=syslog-ng
config=/opt/syslog-ng/etc/syslog-ng.conf
syslog_pid="/var/run/syslog-ng.pid"
#
# check config and programs
test -s ${config}         || {
    echo 1>&2 "${config} does not exist"
    if test "$1" == "stop" ; then exit 0 ; else exit 6 ; fi
}
test -x ${BINDIR}/$syslog || {
    echo 1>&2 "${BINDIR}/$syslog is not installed"
    if test "$1" == "stop" ; then exit 0 ; else exit 5 ; fi
}
#
case "$1" in
   start)
#       startproc -u nobody -p ${syslog_pid} ${BINDIR}/${syslog} -f $config
       daemon ${BINDIR}/${syslog} "-f $config"
       echo  "Starting syslog-ng service"
       ;;
   stop)
       echo "Shutting down syslog-ng service"
       killproc ${syslog} TERM
       # should the tails get killed off too? If so, what is the best way to do this? Possibly 
       #  should move the tail commands to a separate boot script that runs after this script
       ;;
   restart)
       echo "restarting syslog-ng service"
       $0 stop
       $0 start
       ;;
     *)
       echo "Usage: $0 {start|stop|restart}"
       exit 1
       ;;
esac

Validation

Write a message into the dq2 socket on the source host (the dq2 service host):

[root@uct2-grid1 syslog-ng]# /usr/bin/logger -t test -u /tmp/dq2logs.socket "this is a test message"

On the destination side (the central logging host), check to see that the message arrived. This directory /working/syslog-ng/dq2/logfiles/2007.03 was created. Then [root@uct3-edge6 2007.03]$grep test dq2.uct2-grid1.uchicago.edu.log yields:

2007-03-27T08:39:24-05:00 uct2-grid1.uchicago.edu test: this is a test message

Message arrived.

Repeating this test with a very long (>400 bytes) message shows that messages are getting truncated. The syslog-ng docs say that the default message length limit is 8192 bytes, but this does not seem to match experiment. I am currently investigating... Charles

The truncation is in the "logger" program. Patch at http://repo.mwt2.org/viewvc/patches/patch-logger

References

-- RobGardner - 20 Mar 2007

Topic revision: r12 - 06 Jun 2007, CharlesWaldman

Logging

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback