Counting Downloads With Fido

I wanted to illustrate how to use fido with an example. Today we’re going to use it to count software downloads on this site. Exciting! This will be simple since we only have one data source. A few years ago, I move my software from an FTP repository onto this web server. To quantify software downloads, we can simply monitor the http access log.

Here’s our fido configuration for the log file:

/var/log/httpd/access_log {
 rules  = downloads.conf
 action = /usr/local/bin/tally
 log    = syslog
}

This tells fido to monitor the access_log in real time. Its pattern match rules are in a file called downloads.conf When fido finds a match, it will execute a program called tally. Finally, the last directive tells fido to use syslog to log its activity.

In order to understand what we’re looking for, you should take a look at the software repository. It contains multiple versions and helpful links to the latest releases and betas. We want to match them all.

Let’s take a look at our downloads.conf file. Since we didn’t specify a full path to the file, fido knows to look for it under $sysconfdir/etc/fido/rules. If you configured it to use /etc, then the rules are found in /etc/fido/rules/downloads.conf. Here’s the file:

#
# Track and count downloads from the website
SIEGE:  .*siege-.*tar.gz.*
FIDO:   .*fido-.*([rpm]|[tar.gz]).*
WACKY:  .*wackyd-.*tar.gz.*
DICK:   .*dick.*tar.gz.*
SPROXY: .*sproxy-.*tar.gz.*
CONFIG: .*JoeDog-Config.*
STATS:  .*JoeDog-Stats.*
GETOPT: .*php-getopt.*
WACKY:  .*JoeDog-Wacky.*
PBAR:   .*JoeDog-ProgressBar.*

Each line begins with an optional label. If a label is present, fido will pass it (minus the colon) to the action program. In the example above, if the JoeDog-Config perl module is downloaded, then fido will run /usr/local/bin/tally CONFIG. For more on labels, see the fido user’s manual.

Continue reading Counting Downloads With Fido



Invalid command ‘TypesConfig’

Ah but the joys of trying to match the missing module with its obtuse apache error. In this case, we tried to use the TypesConfig directive but the module wasn’t loaded at runtime. Here’s the error:

# service httpd configtest
Syntax error on line 107 of /etc/httpd/conf/httpd.conf:
Invalid command 'TypesConfig', perhaps misspelled or defined by a module
not included in the server configuration

In this case, we were missing the mime module. You can add that module in your httpd.conf file with the following directive:

LoadModule mime_module modules/mod_mime.so

Happy apaching!



Newlines In WordPress

Did you ever want to add a new line to a WordPress entry but it gives you a new paragraph? Instead of this:

– haha
– papa
– mama

You get this:

– haha

– papa

– mama

I hate that. It adds extra space between each line. Fortunately, there’s an easy fix. In order to produce the first list without spaces between each line, just hold the shift key while you hit return.



Invalid command ‘order’

It would be nice if apache told you which module you were missing. Fortunately, there’s the Internets! Hey, this site is on the Internets let’s see if we can help. I just ran ‘service httpd checkconfig’ and received the following error:

# service httpd configtest
Syntax error on line 92 of /etc/httpd/conf/httpd.conf:
Invalid command 'Order', perhaps misspelled or defined by a module 
not included in the server configuration

After a brute force attempt at adding modules, it became clear that I was missing the following module: authz_host_module. I added that in httpd.conf with the following directive:

LoadModule authz_host_module modules/mod_authz_host.so

You can also compile that module into the binary with the following flag: –enable-authz-host  (in most cases that’s compiled by default but I’m using RedHate’s binary so it was necessary to add it at run time).



Helpful Perl Functions

The following pair of functions are ones that I use often. As far as I’m concerned, they should be included in perl. This post serves as both a personal place holder and an opportunity to share with the Internets. Chances are you found them at the sweet end of a Google search.

Method: trim
Params: $string
Return:  $str
Usage:   $str = trim($str);

# This function trims white space from the
# front and back of parameter $string.
sub trim() {
  my $thing = shift;
  $thing =~ s/#.*$//; # trim trailing comments
  $thing =~ s/^s+//; # trim leading whitespace
  $thing =~ s/s+$//; # trim trailing whitespace
  return $thing;
}

 

Php offers a useful utility function called ’empty’ which determines whether or not a variable is, well, empty. Here’s the equivalent function is perl:

Method: empty
Params: $string
Returns: boolean
Usage:    if (!empty($string)) { print “Whoo hoo!”; }

sub empty { ! defined $_[0] || ! length $_[0] }

 

I often use timestamps as unique identifiers or in transaction logging. The Internets are full of perl modules that provide timestamp functionality but I generally prefer to roll my own. Why? Mainly for portability. If a script relies on the basic perl library, then it runs on any server with perl installed.

Method: timestamp
Params: none
Returns: $string
Usage:    print timestamp() . “n”;

# returns a string in the following format:
# YYYYMMDDHHMMSS
sub timestamp() {
  my $now   = time;
  my @date  = localtime $now;
  $date[5] += 1900;
  $date[4] += 1;
  my $stamp = sprintf(
    "%04d%02d%02d%02d%02d",
     $date[5],$date[4],$date[3], $date[2], $date[1], $date[0]
  );
  return $stamp;
}

NOTE: The above function was corrected to include seconds.