Fido 1.1.2 Release

I like C. It’s probably my favorite language. Yeah, yeah, sure, it doesn’t have support for objects but that doesn’t mean you can’t write your own. Fido was written in C but it has an object-oriented architecture. This design makes it easy to debug memory problems and simple to re-use code. (I use the same hash table in all my projects.) So I like C but it also sucks. Version 1.1.1 of this application contains a bug that wouldn’t have compiled in other languages. C was all, “What? You want to do something stupid? Okay, fine.”

As a result, fido-1.1.1 doesn’t trigger an alert when a file reaches its target age while logging is turned on. All fido users should upgrade to fido-1.1.2.tar.gz


Posted in Uncategorized | Leave a comment

Siege 3.0.7 Release

Here’s the format for a location header,  Location: absolute_url

Unfortunately, many developers don’t care about standards and Internet Exploder is famous for letting them get away with it. When siege followed the letter of the law, I was inundated with bug reports that weren’t bugs at all. If siege is confused by Location: /haha that’s your developer’s problem, not mine. Against my better judgement and beginning with siege-3.0.6, I started constructing absolute_urls from relative paths. Unfortunately, my parser missed a usecase: localhost. Siege 3.0.6 will barf on this:

Location: http://localhost/haha_or_whatever

Technically, I didn’t miss localhost. If you look at url.c:459 you’ll see this:

// XXX: do I really need to test for localhost?

It didn’t occur to me that people would run siege on the same server as their webserver.  My bad. There are many tests besides load tests.

All siege users running version 3.0.6 should upgrade to siege-3.0.7.tar.gz

Posted in Siege | Leave a comment

sh Script: Read lines and omit comments and blanks

I perform remote operations on a series of servers. Rather than maintain the server list in several scripts, I consolidated it into a single file called servers.txt  Exciting! The second I did that, I raised my own bar. You’d expect a config file parser to omit comments and blank lines, right? I do.

The anticipated time to write that parser was longer than I expected. In order to save you time, dear reader, I decided to post it here. This sh script reads a file into an array while skipping comment lines and blanks.

Here’s a sample config file:

# Comment line 
# Wed Feb 25 19:28:54 EST 2014

And here’s a script to parse it:

let X=0
while read line ; do
  if [[ ! $line =~ [^[:space:]] ]] ; then
  [ -z "`echo $line | grep '^#'`" ] || continue
  let X=$X+1
done < servers.txt
for (( i=0; i<${#SRV[@]}; i++ )); do
  echo ${SRV[$i]}

Here’s the output from the script:

$ sh ./haha

Here’s another way to coax the data out of the $SRV array. You can convert it into a space separated string and loop through it in a traditional manner:

for S in $SRV ; do
  echo $S

After you guys vet this in the comments, I’ll add it to the sh scripting cheat sheet. Happy hacking.

UPDATE: A reader sends me a one-liner which implements similar functionality.  If you don’t require an indexed array, then it’s only drawback is its perl dependency.

SRV=$(egrep '[^[:space:]$]' servers.txt|egrep -v '^#'|perl -pe 's/^\s+//')
for S in $SRV ; do
  echo $S
Posted in Programming, sh | Leave a comment

It Knows Me Better Than I Know Myself….

robotI write a lot of software with which I interact. If it’s easy for me, then it’s easy for you. I try to keep it easy for me. JoeDog’s Pinochle is the first program against which I’ve competed. It’s been a surreal experience.

The program was designed to be competitive against me. Tonight it took two out of three games. The damn thing knows me inside and out. And why not? I wrote it. And while I can exploit some knowledge of its inter-workings, I can’t predict all its behavior. It was designed to learn bidding from experience.

Bidding is the hardest aspect of this game. The team that wins the bid has an incredible opportunity to earn a lot of points. At the same time, overbids come at a large price. A failure to make the bid means the bid is deducted from your score.

When the game was first released, its bids were implemented programmatically. I like to think I’m a pretty good programmer but that version of the game played like a moran. To improve it, I had the game play itself hundreds of thousands of times. It would store those results and use them to generate future bids.

This implementation has resulted in a much more competitive program. Now it bids more aggressively — much more aggressively. It bids like me which is odd because I didn’t tell it to do that. I told it to learn from its experience and as a result of that experience, its personality morphed into mine.


Posted in Applications, Java, Pinochle, Programming | Leave a comment


Today I’m pleased to announce the first public release of JoeDog’s Pinochle. It’s a computerized version of the classic card game. It plays a four-player variation in which you are paired with a computer player against two computer players. Exciting!

This project is notable for several reasons: 1.) It’s the first time I’ve released software with a graphical interface and 2.) it’s the first major project I’ve completed in java.

JoeDog’s Pinochle is the culmination of hundreds of hours of work over the past several years. The groundwork was laid on planes and trains. It offered an enjoyable way to pass the time as I sat in traveling tubes. Last September I finally achieved a functioning version of the game. Since then, I’ve honed its ability to play a descent game.

Because pinochle maintains a strict set of rules for governing play, most of the “intelligence” in this game was implemented programmatically. Unfortunately, its original ability to assess and bid a hand was very weak. In order to improve that, I’ve built experience into the game. It played itself for hundreds of hours and stored those outcomes. Now when it bids a hand, it consults past experience to shape future results. A skilled human can still beat it but give me time. It will get better with each ensuing release.


I’m currently honing the game’s ability to beat the pants off you people. That may take some time. Once I’ve built adequate intelligence into the game I’d like to add more variation. A double-deck version will be added at some later date. I’d also like to add three and five player variations.


This game is currently licensed under the terms of shareware. It contains some code that was published without licensing terms and the author has not answered my inquiries about its license. It’s probably in the public domain but until I get verification I can’t release it under an open source license. If you’d like a copy of the code, send me an email. Until then, enjoy the binaries.

Posted in Applications, Pinochle | Leave a comment

Because I like esc-esc completion, that’s why

Your JoeDog got a new laptop – happy nice time!

He bought if from System 76 with Ubuntu 13.10 preinstalled. No major work to get this bad boy configured, right? Well….

In a previous life Your JoeDog was a UNIX guy. He earned his chops on AIX before moving to an HPUX shop. On commercial UNIX, AT&T’s korn shell is king. In the 1990s, it showed its flexibility by accomodating both vi and emacs editing. But c’mon. Any UNIX geek worth his salt ran it in vi-mode. Along the way, JoeDog developed a habit of completing filenames by hitting the esc key twice. /usr/lo esc-esc and he was in local. Bash weenies arrive at the same location by hitting the tab key. Only one stroke they say. Perhaps. But Your JoeDog gets emotional satisfation from banging esc twice. [pop, pop]

Now unfortunately, Ubuntu didn’t include ksh on his shiny new laptop. No problem, ‘apt-get install ksh’ Unfortunately that version didn’t support esc-completion. No problem, ‘apt-get install pdksh’ And ah-way we guh …. WTF? That didn’t support it either.

That’s okay. Your JoeDog is pretty adept with a compiler. He snagged pdksh from source and rolled his own. There’s a couple things to remember when you do this: 1. make sure the shell you install is in /etc/shells or you won’t be permitted to login. 2. You need to add ‘set -o vi-esccomplete’ to your $HOME/.profile in order to achieve esc-esc completion.

During the build, the following error was encountered:

gcc -c -DHAVE_CONFIG_H -I. -I. -g -O table.c
./ "gcc -E -DHAVE_CONFIG_H -I. -I." < ./ > tmpsiglist.out
sort: cannot read: +2n: No such file or directory
make: *** [siglist.out] Error 1

Well that sucks. Your JoeDog punched that error into the Google machines and boy did he get an earful. There were plenty of people who encountered that error. They were frustrated and turned to the Internets for help. Hey, anybody know how to correct this? Oh, the Internets had opinions all right. You know what they had to say? “ksh sucks, use bash!” Well, that’s certainly one thought. Here’s another one: fsck you! Flies gotta fly. Bees gotta be. And UNIX grey beards gotta esc-esc.

So the Internets weren’t much help but the sort’s documentation helped shed some light. To fix the error above, you need to go POSIX. So before you run ‘make’ run this command:

export _POSIX2_VERSION=199209

Are you with us so far? If you’re an esc-esc guy, we’re pretty sure you are. With his new pdksh in place, Your JoeDog was esc-esc’ing all over the place … until he tried to log in to a new Xsession. WTF? Basically, he was locked out of his brand new laptop.

To get into single-user mode in Ubuntu, hold down the shift key while it’s booting. That will provide a menu through which you can navigate to the root shell. Once you’re in the root shell, you’ll have to mount file systems read-write in order to fix the errors that locked you out of your system. Here’s how you mount root as read-write:

mount -o remount,rw /

Here’s what Your JoeDog discovered inside his $HOME/.xsession-errors file:

/usr/sbin/lightdm-session: 15: set: Illegal option -o vi-esccomplete


So the directive that provides the feature he wants prevents him from logging in. If that’s not a Catch-22, then what is? Since we rolled our own shell, let’s just alter its code so it does our bidding. Unfortunately, the pdksh source wasn’t the most intuitive read but Your JoeDog slogged through it so you won’t have to.

On line 1142 of vi.c you’ll find this:

 case Ctrl('['): /* some annoying at&t ksh's */
    if (!Flag(FVIESCCOMPLETE))
        return -1; 
 case '\\': /* at&t ksh */

Do you see that? Try searching for the word “annoying” in the code. Okay, to do esc-esc completion by default without resorting to ‘set -o vi-esccomplete’ change that block of code so it looks like this:

 case Ctrl('['): /* some annoying at&t ksh's */
 case '\\': /* at&t ksh */

Now esc-esc falls down to esc-\ and its behavior is the same.



1. Download pdksh-5.2.14

2. Unzip the tarball: gunzip -9cd pdksh-5.2.14.tar.gz | tar -xvf –

3. Edit vi.c

On line 1142 of vi.c you’ll find this:

case Ctrl('['): /* some annoying at&t ksh's */
    if (!Flag(FVIESCCOMPLETE))
        return -1; 
 case '\\': /* at&t ksh */

Make it look like this:

 case Ctrl('['): /* some annoying at&t ksh's */
 case '\\': /* at&t ksh */

4. Run: ./configure

5. Run: export _POSIX2_VERSION=199209

6. Run: make

7. Run: cp ksh /bin/pdksh

8. Run: cp ksh.1 /usr/share/man/man1/pdksh.1

9. VERY IMPORTANT: add /bin/pdksh to /etc/shells

10. In /etc/password change your shell to /bin/pdksh

Happy double-escaping.

Posted in Environment, System | 2 Comments

An Old Dog Learns A New Trick

Beginning with version 3.0.6-beta2, siege reacts differently to –reps=once.

In the past, when you invoked –reps=once, each siege user would invoke each URL in the file exactly one time. If urls.txt contained 100 files and you ran -c10 –reps=once, siege would finish its business with 1000 hits.

That was then.

This is now: siege runs each URL in the file exactly once. If you run -c10 –reps=once, then siege will split the file among all 10 users and hit each URL one time. Whereas in the past, you’d finish with 1000 hits, you now finish with 100 hits.

This should give you greater control by making tests more precise.

Posted in Applications, Siege | Leave a comment

Why Do Investors Love Amazon?

What’s happening at Amazon isn’t suppose to happen in modern finance. Shares are rising as profits are falling:

Amazon shares are up around 150 percent since mid-2010, which perhaps not coincidentally was the last time the company had sizable profits. In other words, investors really decided they loved the company only when net income began to slide.

Any fool can run a profitable company but it takes a gutsy person to build the world’s largest retailer….

[New York Times - All Amazon Is Missing Is a Profit]

Posted in Community, Technology | Leave a comment

Fido Learns A New Trick

I use Mondoarchive to create Linux recovery disks. Each server writes ISO images to a shared volume on a weekly basis. If any file inside that directory is older than seven days, then a server failed to create an ISO. In order to monitor this directory for failure, I added a new feature to fido. Exciting!

Starting with version 1.1.0 (click to download), fido can monitor a file or directory to see if it — or any file inside it — is older than a user configurable period of time. If fido discovers a file whose modification date exceeds the configured time, it fires an alert.

The following example illustrates how to configure the use case above:

/export {
  rules = exceeds 8 days
  exclude = ^\.|^lccns178$|^lccns179$|^lccns335$|lccns336$
  throttle = 12 hours
  action = /etc/fido/

This file block applies to “/export” which is a directory. Since it’s a directory, the rules apply to every file inside it. In this case ‘rules’ is pretty straight forward. We’re looking for files that exceed eight days in age. This rule will always follow this format: exceeds [int] [modifier]. The modifier can be seconds, minutes, hours or days. If you take the long view — if you’re concerned about events far into the future — then you’ll have to do some math. We don’t designate years so you’ll have to use 1825 days if you want to be alerted five years out.

We also find a new feature inside this block. ‘exclude’ takes a regular expression and tells fido which files to ignore. Currently, ‘exclude’ only works inside a file block with an exceeds rule but I plan to make better use of it.

Finally we notice one final feature that we’ve never seen before. The ‘throttle’ directive tells fido how long to wait between alerts. In this scenario, fido will trigger an alert the second it finds a file which exceeds 8 days. If the problem is not addressed within twelve hours, it will fire another alert. Alerts will continue in twelve hour intervals until the problem is corrected.

I hope you enjoy these features. If there are enhancements you’d like to see, feel free to contact me either in the comments or by email.

Posted in Applications, Fido | Leave a comment

Gub’mint (IT) Mule

bureaucratsSean Gallagher has an interesting piece on (ars)technica. He asks, “Why do government IT projects fail so hard and so often?” Gallagher provides several reasons, most of which are symptoms of a large organization. Let’s examine that list.

1. The government uses antiquated technologies. Its bureaucracy is slow to move and slow to adapt. Older technologies remain long after their life cycle expires largely because the approval process for new ones is long and arduous. You can imagine many frustrating meetings that end with, “Fsck it. We’ll put it on XP.”

2. Its user base is really large. Gallagher cits as one example a DOD email rollout that touched 1.5 million users. That’s an astonishing number for an in-house IT department. Certainly there are web companies with more users — there are 425 million GMail accouts, for instance — but Google does web for profit. The army’s IT department is an expense.

3. Flawed metrics. Gallagher notes that many government IT dashboards are filled with nice metrics that contain a lot of nines. Unfortunately, those nines have little bearing on end user experience. If department CIOs measured things that mattered, they’d be filled with zeros.

I’m sure these are valid criticisms but how do they vary from other large organizations? I work for a large corporation and this week my company finally moved my laptop off Windows XP. There’s no way the entire organization will be XP-free by end-of-life-cycle. It took a Great Recession to convince management that Linux was a viable alternative to HP-UX.

Our user base is 15,000 and every internal roll-out contains some glitches or problems. The army rolls out applications to a user base that is two orders of magnitude larger than ours. Where we roll out in a carefully controlled environment, they have to provide service to all corners of the world. Some of those corners are pre-fab barracks on an Afghanistan mountain top.

I’m yet to meet a person in my company who likes our out-sourcing partner. The bean counters like how little they cost, but they’re not happy with their services either. Yet if you look at this partner’s dashboard, you’ll find it’s filled with as many nines as those government CIOs. Faulty metrics aren’t limited to the public sector.

The Affordable Care website famously crashed during its rollout. Yet on the surface it wasn’t subject to many of government IT’s shortcomings. Most of the work was handled by a private partner. They used apache webservers on Linux. The site was fronted by the Akamai CDN which greatly reduces load by moving content close to the users. Most importantly, the site wasn’t tied to antiquated government infrastructure. Yet it failed. Why?

When you examine the site you find it’s simply not optimized for heavy traffic. The pages are too heavy and they contain too many elements. Fifty-six javascript files? Really? Then we learn the system was tested under load that was an order of magnitude less then what they received on October 1st. The site was basically slash-dotted.

Certainly there are good failures and bad failures. Collapsing under the weight of your own popularity is a good one. Still, with better planning and better coding the Affordable Care site could have experienced a more successful roll-out. Those operations require a high level of expertise which brings us to what I suspect is the real reason government IT projects fail: constrained by the tax payer’s dime, government can’t attract the talent necessary to service a very large user base. In other words, we get what we pay for.

Posted in On The Job, Technology | Leave a comment

Recent Comments

  • Sohan: I am using node-js http server. I created http request to hit the server and log the message. In that case...
  • Jeff Fulmer: I don’t know what “simple http server” means. If you’re using apache out of the...
  • Sohan: MY server is simple http server. Can you tell me what thresol paramter or property is used in siegerc file.
  • Jeff Fulmer: Ninety-five percent of the time, this error occurs because you’ve configured more simulated users...
  • Sohan: Hi, Resolved by commenting time parm in siege.rc Cheers