Fido Manual

A file watcher and notification utility

INTRODUCTION

Fido is a highly configurable log watch utility. It can peruse multiple files simultaneously for user defined patterns. It supports extended regular expressions. Fido works in a manner similar to UNIX tail but it checks for file truncation so it will continue to function after log rotation.

The name comes from the proverbial dog of yore. During the Second World War, an Italian kiln worker found a street dog and nursed him back to health. He named the dog Fido from the latin word for “faithful one.” It was a good choice.

Every day Fido would follow the kiln worker to the public square where he would catch a bus to work. Fido would wait faithfully for his master to return at night. One day, the kiln worker was killed in an airstrike. His dog waited patiently for his return. His master never came. For the rest of his life, Fido went daily to the square and patiently waited for his master to get off the bus. He continued this pattern faithfully until his death in 1958. This program is a tribute to the dog.

RUNNING FIDO
The format for running fido is this:
fido [options]

Fido is built with GNU’s long options. Each option can be invoked in either long or short form. Both long and short can be used in the same startup. The program supports the following options:

-V
–version
VERSION, prints version number to screen.

-h
–help
HELP, prints the fido help message which closely resembles this section of the manual.

-f /path/file
–file=/path/file
FILE, an alternative config file (default: /etc/fido/fido.conf)

-C
–config
CONFIG, displays the default settings and exits. This option works best when its invoked with -f it gives you a chance to inspect the configuration of that file

-d
–daemon
DAEMON, runs the background as a daemon. This is the default setting and one you will probably want to use. Non-daemon mode is good for config troubleshooting or for development testing.

-l arg
–log=arg
LOG, this allows you to set the logging option. The choices for ‘arg’ are ‘syslog’ or ‘/path/to/file’ If you select the former, it will use the system’s syslog mechanism. If you select the latter, it will create a log file and write to that.

-p /path/file
–pid=/path/file
PID, set an alternative path to a process ID file. The default pid is: /var/run/fido.pid

On RedHat systems, it’s recommended that you install the RedHat startup scripts and run fido from service, i.e., service fido start / service fido stop. The startup scripts are located in the utils directory of the source code. There is also a script to help you install them. As root, run:

utils/install-startup

CONFIGURATION

Fido is configured with a fido.conf file. By default it loops through the following files in search of a configuration:

/etc/fido/fido.conf
/etc/fido.conf
/usr/local/etc/fido/fido.conf
/usr/local/etc/fido.conf

At run time, fido uses the first first fido.conf file it finds. If you have one in /etc/fido/fido.conf and /usr/local/etc/fido/fido.conf, then it will only read the first one. You can set an alternative configuration file with -f /path/file.

Since fido can be configured in one-to-many mode (one fido can monitor several files), some of its directives can be set at both the global level and the file level. In this section of the manual we’ll detail the configuration directives. If a directive is GLOBAL, then it’s set outside of a file block. If it’s FILE, then it’s set inside a file block. And if it’s BOTH, then it can be set globally or inside a file block. If a both directive is used inside a file block, then it will override the global directive for that block.

daemon = true | false

GLOBAL: This directive tells fido to run in the background as a UNIX daemon. Generally speaking, you’ll want to run it in this manner. In the foreground, it’s useful for config testing or development debugging. Generally speaking, ‘false’ is more beneficial to fido developers than fido users.

rulesdir = /path/to/directory

GLOBAL: This directive tells fido where to look for a rules file that was specified inside a file block. If your rulesdir is set to ‘/etc/fido/rules’ and you specify ‘rules = downloads.conf’ inside a file block, then fido will look for /etc/fido/rules/downloads.conf

user = name

GLOBAL: This directive tells fido under which UNIX name it should run. As a rule, you should run fido as the least privileged user possible. You will need adequate permission to set a pid file and log to disk. Fortunately, fido allows you to change the path of both those files.

group = name

GLOBAL: Like the user directive, this one tells fido under which group name it should run. You will need the necessary privileges to run under the group you select.

log = ‘/path/to/file’ OR syslog

BOTH: This directive tells fido how to log its messages. If you specify a file, it will create a file and log its messages there. If you select syslog, then it will use the system’s syslog mechanism. This directive can be set at both the global level and the file block level. It’s possible to log global messages in syslog and file block messages inside a specified file.

pid = ‘/path/to/file’

GLOBAL: This directive tells file where to write it’s pid file. The default value is /var/run/fido.pid. If you don’t have permission to write to that directory, then you can certainly specify an alternative location for the pid file, i.e., pid = /home/jeff/var/fido.pid

action = /path/to/script

FILE: When fido matches a pattern inside a file it’s watching, it responds by issuing a command. The action directive tells it what to run. If your rules directive has a prefix label (see rules), then fido will pass that as an argument. Here’s an example: LABEL: .*fido-.*tar.gz.* If fido matches that pattern inside an apache access log, it will run /path/to/script LABEL

capture = true | false

FILE: This directive tells fido to capture the output of the action command inside the logging mechanism. If your action is echo “whoo hoo!” | sendmail -v [email protected]then fido will write all the sendmail verbosity to its log. (Good for debugging action commands)

rules = filename.conf | regex

FILE: This directive tells fido what it’s looking for inside a file it watches. If your rules point to a file inside $rulesdir, then fido will read them into a list and loop through that list looking for a match. Alternatively, you can specify a single value for rules and set it in the config file, i.e., rules = LABEL: .*OutOfMemoryError.*

Comments
Fido treats # and ‘ as comment designations. You can place them on a line by themselves or they can trail a directive. Like all config file comments, this allows you to make secret notes for yourself.
# This is a comment
key = value # and so is this
key = value ' this is, too, but we don't like this style as much
Globals
A fido configuration file is broken into two parts, global and file blocks. Global directives are simply ones that are not inside a file block. These are ‘key = value’ pairs that are detailed above. By convention, global values are placed at the top of the file while file blocks are placed below it. In reality, it doesn’t matter where you place your globals as long as they’re outside a file block.
File Blocks
Inside the configuration file, a file block specifies a file that we parse and issue a response. Any file can be configured as a block as long as you follow this format: it’s a path and file followed by brackets. Inside the brackets you can define directives. Here’s an example:
/usr/local/jboss/server/haha/log/console.log {
  action = /home/jboss/bin/haha
  rules  = MEMORY: .*OutOfMemoryError.*
  log    = /home/jboss/var/fido.log
}

The rules assignment can also be a file name. In the example below, we set rules to downloads.txt. Fido will look for that file inside its rules directory (The default rules directory is /etc/fido/rules).

rules = downloads.txt

If that’s not enough flexibility, you can also configure a direct path to the file:

rules = /home/jboss/etc/jboss.rules

This provides an opportunity to discuss the rules directive in further detail. It’s a little tricky. If the value assigned to rules is not a readable file, then fido assumes the value is a rule. That’s the case in the sample above. I could have put the rule inside a file like this:

HAHA: .*I said, "Ha Ha!".*

Fido will look for that rule inside its $rulesdir. If we assume the default, then downloads.txt is located in /etc/fido/rules/downloads.txt.

Rules Files

If you’re only concerned with matching a single pattern, then you probably don’t need a rules file. You can place a regular expression in the rules directive inside the block:

/usr/local/jboss/server/haha/log/console.log {  
  action = /home/jboss/bin/haha  
  rules  = MEMORY: .*OutOfMemoryError.*  
  log   = /home/jboss/var/fido.log
}

In this case you probably don’t need an optional label. After all the purpose of the label is to designate the match. If present, a label is passed to your action program as the first arg like this:

 /home/jboss/bin/haha MEMORY

If you don’t need a label, then format the directive like this:

  rules = .*OutOfMemoryError.*
A rules file provides the benefit of multiple patterns. Each line inside the file will be inserted into a list. Fido will loop through in search of a match. This is where labels come in handy. The accompanying label will help you determine what was found. Here’s an example file:
#
# Track and count software downloads
SIEGE:  .*siege-.*tar.gz.*
FIDO:   .*fido-.*tar.gz.*
WACKY:  .*wackyd-.*tar.gz.*
DICK:   .*dick.*tar.gz.*
SPROXY: .*sproxy-.*tar.gz.*
CONFIG: .*JoeDog-Config.*
STATS:  .*JoeDog-Stats.*
GETOPT: .*php-getopt.*
WACKY:  .*JoeDog-Wacky.*
PBAR:   .*JoeDog-ProgressBar.*

Rules Labels

A rules label works well in a rules file. It gives you a chance to identify the match and perform actions based on the match. Fido will pass the label to as the first argument to the program you specified with the action direction. Consider this label:

 

SIEGE:  .*siege-.*tar.gz.*

If your action is /usr/bin/haha, then fido will run /usr/bin/haha SIEGE on a match. (NOTE: fido removes the colon).

The format for a rules label is this:

[A-Z0-9_-]+:

This means a label can contain UPPERCASE letters, numbers an underscore (‘_’) or a hyphen (‘-‘). That series is followed by a colon (‘:’). SIEGE_01: is a legitimate label.

Posted in | 2 Comments

Leave a Reply




Recent Comments

  • Mirko: Wow! This trick saved my day :) thanks a lot
  • The Spaniard: greatest team ever
  • Mike Smith: I find that Dunning-Kruger explains a lot. The difficult part is getting help to those that suffer from...
  • Sohan: I am using node-js http server. I created http request to hit the server and log the message. In that case...
  • Jeff Fulmer: I don’t know what “simple http server” means. If you’re using apache out of the...