LogSurfer Software and Resources
IntroductionLogsurfer is a program for monitoring system logs in real-time, and reporting on the occurrence of events. It is similar to the well-known swatch program on which it is based, but offers a number of advanced features which swatch does not support.
Logsurfer is capable of grouping related log entries together - for instance, when a system boots it usually creates a high number of log messages. In this case, logsurfer can be setup to group boot-time messages together and forward them in a single Email message to the system administrator under the subject line "Host xxx has just booted". Swatch just couldn't do this properly.
Logsurfer is written in C - this makes it extremely
efficient, an important factor when sites generate a high
amount of log traffic. I have used logsurfer at a site
where a logging server was recording more than 500,000
events per day - and Logsurfer had no trouble keeping up
with this load. Swatch, on the other hand, is based on
perl and runs into trouble even when dealing with a much
smaller rate of log traffic.
Logsurfer 1.8 Features (5 September 2011)A bug fix and a couple of extra command-line options:
- Fixed double free() in exec.c/prepare_exec(), thanks to reports from Gregor Kopf of Recurity Labs, Jan Kohlrausch of DFN_CERT, and Wolfgang Ley
- Updated README & DISCLAIMER files removing DFN-CERT copyright
- Now just Logsurfer, not Logsurfer+ any more
- -D command line option for daemon mode. Warning: closes stdin, stdout, & stderr, therefore no error messages
- -F command line option to auto re-open log file
Logsurfer 1.7 FeaturesMany of the features in this release are designed to allow Logsurfer to work as a log aggregator - to quickly and efficiently detect complex events and place summarization messages either into plain files or back into syslog.
- Added -e option to begin processing from the current end of the input log file ( normally used with -f )
- Put double-quotes around regex expressions in dump file
- If the context argument to a pipe or report action is "-" then the current context contents are piped into the command this should shorten most context definitions
- Added new action "echo" which simply echo's the output on stdout, or to a file with optional >file or >>file first argument. This is more efficient than invoking an external process for simple echo actions.
- Added a macro construct in context action fields, if "$lines" exists in a context action (such as a command line) it will be substituted by the number of lines in the context
- Added syslog action to send a message into syslog. The first argument to the action must be <facility>:<level>, the second argument is the string to send to syslog. Note that the log lines stored in a context are not forwarded into syslog.
Logsurfer 1.6 Features
- An optional parameter at the end of context definitions ( just before action ) specifying the minimum number of lines collected which needs to be satisfied before performing the action. This min_lines argument can be used for detecting events such as firewall attacks where we are only interested in events which generate more than x log entries ( like packet drops from a single source IP address ).
- Added -t command line option to explicity timeout contexts when exiting, therefore running the action for all contexts. The default is off, so contexts don't all trigger their actions when logsurfer is shut down.
- Changed context rule execution so that we only store lines in a context if the context has an action of 'pipe' or 'report'. In other words, don't store lines in memory which won't ever be used. The number of matching lines in the context is still incremented. This allows contexts to be created which can notify if we don't see an event, such as regular "syslog pings" from hosts.
DownloadDownload Logsurfer (190 kbytes):
- Logsurfer packages are available for most Sun/Solaris systems from SunFreeWare.Com
Startup/Shutdown scripts (/etc/init.d/logsurfer)You will need to modify these according to how you system is set up, including location of the configuration file and log file.
- An Introduction to Logsurfer - an introductory paper I wrote for Sys Admin magazine, published in March 2004.
Logsurfer man pages:Manual pages for the most current version of Logsurfer (v1.8):
Logsurfer configuration examplesMost of these require the latest version of Logsurfer, and may not work on older versions.
- DFN-CERT Logsurfer examples - these should work for all versions of Logsurfer
Logsurfer LinksLinks to other sites with Logsurfer information.
- Logsurfer SourceForge files
- Logurfer SourceForge Home - a bit out of date
- A paper by James E Prewett on using logsurfer to analyse linux cluster log files
- EMF's logsurfer resources - lots of useful configuration recipes for any version of Logsurfer