LogSurfer and LogSurfer+ Resources

Contents

Introduction
Logsurfer+ features
Download
Documentation
Mailing List
Configuration examples
Links

Introduction

 Logsurfer is a program for monitoring system logs in real-time, and reporting on the occurrence of events. It is similar to the well-known swatch program on which it is based, but offers a number of advanced features which swatch does not support.

Logsurfer is capable of grouping related log entries together - for instance, when a system boots it usually creates a high number of log messages. In this case, logsurfer can be setup to group boot-time messages together and forward them in a single Email message to the system administrator under the subject line "Host xxx has just booted". Swatch just couldn't do this properly.

Logsurfer is written in C - this makes it extremely efficient, an important factor when sites generate a high amount of log traffic. I have used logsurfer at a site where a logging server was recording more than 500,000 events per day - and Logsurfer had no trouble keeping up with this load. Swatch, on the other hand, is based on perl and runs into trouble even when dealing with a much smaller rate of log traffic.

Logsurfer+ Features

Logsurfer+ is a branched version of the standard Logsurfer package from DFN-CERT, it has been modified to add a few features to improve what can be done with it.

Logsurfer+ 1.7 Features

Many of the features in this release are designed to allow Logsurfer to work as a log aggregator - to quickly and efficiently detect complex events and place summarization messages either into plain files or back into syslog.
  • Added -e option to begin processing from the current end of the input log file ( normally used with -f )
  • Put double-quotes around regex expressions in dump file
  • If the context argument to a pipe or report action is "-" then the current context contents are piped into the command this should shorten most context definitions
  • Added new action "echo" which simply echo's the output on stdout, or to a file with optional >file or >>file first argument. This is more efficient than invoking an external process for simple echo actions.
  • Added a macro construct in context action fields, if "$lines" exists in a context action (such as a command line) it will be substituted by the number of lines in the context
  • Added syslog action to send a message into syslog. The first argument to the action must be <facility>:<level>, the second argument is the string to send to syslog. Note that the log lines stored in a context are not forwarded into syslog.

Logsurfer+ 1.6 Features

  • An optional parameter at the end of context definitions ( just before action ) specifying the minimum number of lines collected which needs to be satisfied before performing the action. This min_lines argument can be used for detecting events such as firewall attacks where we are only interested in events which generate more than x log entries ( like packet drops from a single source IP address ).
  • Added -t command line option to explicity timeout contexts when exiting, therefore running the action for all contexts. The default is off, so contexts don't all trigger their actions when logsurfer is shut down.
  • Changed context rule execution so that we only store lines in a context if the context has an action of 'pipe' or 'report'. In other words, don't store lines in memory which won't ever be used. The number of matching lines in the context is still incremented. This allows contexts to be created which can notify if we don't see an event, such as regular "syslog pings" from hosts.

Download

 Download Logsurfer+ (190 kbytes):
    MD5Sum for these source packages: f3536af5bd612dde9880f7018213f9bd

Packages

  • Logsurfer+ packages are available for most Sun/Solaris systems from SunFreeWare.Com

Logsurfer+ patches

These patches can be applied to the standard DFN-CERT Logsurfer v1.5b to bring it up to the level of Logsurfer+ :

Startup/Shutdown scripts (/etc/init.d/logsurfer)

You will need to modify these according to how you system is set up, including location of the configuration file and log file.

Documentation

 Logsurfer+ documentation (mostly good for the standard Logsurfer as well)

Logsurfer+ man pages:

Logsurfer configuration examples

 Most of these depend on Logsurfer+ v1.6 or higher code. See also EMF's page here for many more configuration examples - these will work for all versions of standard Logsurfer and Logsurfer+

Mailing List

 There isn't enough interest in Logsurfer for it to have its own mailing list. If you want to post questions, comments, or suggestions then please join the Loganalysis mailing list and post Logsurfer stuff there. A number of Logsurfer fans (inculding myself) are members of this list.

Logsurfer links

 Links to other sites with Logsurfer information.