You guys asked for it and Your JoeDog delivered! The problem is you guys asked for it fifteen years ago and Your JoeDog delivered yesterday.
parser = true
Siege was originally designed with performance in mind. Every thread basically dumped its HTML into the same bucket. This made parsing the content impossible without a design change. The choices were to wrap the reads with thread locks or create individual buckets for every thread. The former option would dramatically decrease performance and the latter option will require a lot of RAM. Imagine, if you will, running 255 browser tabs at once. “Meh,” Your JoeDog thought, “RAM is cheap now.”
We want to get the new version into your hands as soon as possible so it’s available with a few caveats:
- Don’t use gzip’d encoding – it will handle it but it’s not ready for prime time.
- Don’t expect cache validation to work – it requires a design overhaul
- Don’t expect it to parse poorly formatted HTML
- Element requests are sequential. We plan to make them concurrent.
There’s still a lot of work until 4.0.0 is ready for Prime Time but it can serve you well as long as your keep its limitations in mind.
[SIEGE RELEASE: 4.0.0-beta2]
Your JoeDog overhauled siege’s cookie handling. It was re-written in an object-oriented manner which makes maintenance and debugging simpler. That’s good for me, Your JoeDog, but what about you, Your Siege User?
Siege 3.1.4 will have two noticeable changes. Individual configuration files have moved and cookies will now persist between runs. In order to be tidy, we created $HOME/.siege The resource file was moved from $HOME/.siegerc to $HOME/.siege/siege.conf A new file will also be added to that directory. Your cookies will be stored in $HOME/.siege/cookies.txt At some point we’ll probably move the urls.txt file there as well.
We’d appreciate if you could test it. You can find it here: Siege-3.1.4b2
By now you’ve heard that we had another mass shooting in the United States. It was our 351st of the year — fourteen more and we get a free ice cream! The shooting was in San Bernardino and #SanBernadino was trending on Twitter. The world already thinks we’re violent, guys. Do we need them to think we’re stupid, too? Learn to spell the town that’s currently under fire.
Wars and shootings are how America learns geography. Nobody knew where Baghdad was until we bombed it.
So this particular shooter left an office holiday party in a fit of rage. He returned with his wife and they shot up the place. So this much is clear: Office parties suck! Let’s face it. The only thing we have in common is we applied for a job in the same company. Now we have assholes showing up with guns. Your JoeDog would rather stay at home and hack some code.
We learned this particular office asshole was stockpiling ammunition and bombs. When police searched his vehicle, they found 6000 rounds of ammunition and a dozen pipe bombs. Now that’s a guy who hates his co-workers.
Some of you may think Your JoeDog’s a bit flippant about this whole ordeal, telling jokes and dropping snark. For the past thirty years we’ve tried “thoughts and prayers.” When someone shot up a public place, we gave thoughts and prayers. Over the last thirty years we gave a lot of thoughts and a lot of goddamn prayers.
Well guess what? They don’t fscking work. Your JoeDog’s just going to do what he does best: Toss spit balls from the back of the room. You know what’s annoying and gross? Spitballs. If we plaster the people who own this country with enough of them maybe they’ll finally act. God knows thoughts and prayers don’t work.
Your JoeDog has moved … again.
We were on a t.small instance on Amazon Web Services running RedHat Enterprise Linux. That instance cost us $0.086 / hour — that’s 8.6¢ an hour. The monthly billing total was around $65.00. A t.small instance includes 1 CPU, 8GB of disk and 2GB of RAM. Two gigs was a lot of memory in 2004, but contemporary software goes Oliver Twist on that shit. “Please, sir, may I have some more?”
Your JoeDog wanted more. We had trouble managing our application stack within the confines of two gigs of RAM. Your JoeDog checked its options on the Amazon pricing chart and found something interesting: Amazon Linux is priced far below RedHat Enterprise. A t.small server running that OS was only $0.026 per month. With prices that low, it offered room for a server upgrade. For $0.052 / hour, we could get a t.medium server which has 2 CPU and 4GB of RAM. That was more acceptable.
Yes, more memory is more awesome but there’s an ever better reason to switch from RHEL7. That OS uses systemd as its startup mechanism. Amazon uses SYSVINIT. (Earlier version of RHEL used SYSVINIT. Systemd is new to RHEL7)
We cutover to the new server a few hours ago. It was a simple move since Your JoeDog was using an Amazon RDS database. We just copied content and configs to the new server and we were off and running.
Is Amazon Linux right for you? Probably. For starters, it’s similar. It has the look and feel of a RedHat variant. It uses RPM for package management and it’s directory hierarchy is similar. As mentioned above, it’s startup mechanism is SYSVINIT rather than systemd. In this sense, it’s closer to RHEL6 than RHEL7.
For the most part, the “enterprise” premium provides enterprise support. Some software vendors won’t provide any support unless you run their product on a “certified” Linux. If you don’t have that particular requirement, then Amazon’s flavor should serve you as well (at a fraction of the cost).
Siege is available through most major Linux distributors. That makes us feel like important uppity-ups. It also cuts down on the time we spend walking n00bs through the compile process. Unfortunately, it comes with a downside. Because most people use distributed versions, it takes a while for news of buggie bugs to reach us.
We applied a patch to version 3.1.1 that introduced one such bug. Versions 3.1.1 and 3.1.2 incorrectly handle HTTP POST. Instead of following protocol, those versions make this request:
POST /test POST ha=1&ho=2 HTTP/1.0
Well, that’s embarrassing! Version 3.1.3 fixes that mess. Your JoeDog regrets the SNAFU.
This latest version does another thing! It may look like a bug, but it’s a feature. Beginning with siege-3.1.3 you will no longer be able to run siege with more than 1000 users without changing your configuration. This allows us to Dog-splain in the rc file why you’re probably doing something foolish.
The siegerc file now contains a limit directive which caps the number of users at 1000 by default. Frankly, that’s probably still too high. If the limit directive is not present in that file, then siege defaults to a hardcoded limit. If you want to run siege with more than 1000 users, you’ll have to set that ceiling in the rc file yourself:
limit = 1001
Please understand, if you jack that up and make a mess, we’re not going to be very sympathetic.
A few years ago, Your JoeDog was with Mrs. JoeDog in McSorley’s Old Ale House. For those who’ve never been to McSorley’s it’s an interesting place. The East Village pub is well-known for its very large beer selection. The choices are overwhelming. Since 1854 they’ve been serving McSorley’s light and McSorley’s dark. Truth be known, the “dark” is just light with added syrup. But here’s the cool part: a “beer” is actually two beers. If you and a companion order a couple, the bartender brings you four.
McSorley’s etiquette holds that tables should be shared. You shouldn’t sit at an empty table unless all the others are full. On that late Saturday afternoon a few years ago, Mr. and Mrs. JoeDog sat beside a man from Frankfurt, Germany. We engaged him in a conversation about German cars and German beer. Finally he said, “All you Americans want to talk to Germans about is cars and beer.” Basking in the warm caress of a second round Your JoeDog said, “And after a few more of these, we’ll probably ask about you-know-who.”
It turns out there’s more to Germany that cars, beer and Hitler. Who knew?
Recently Your JoeDogs found the tables turned when they went to London to watch their beloved New York Jets defeat the hated Miami Dolphins. The game was part of the NFL’s international series. London pub culture is a lot like McSorleys. Table sharing isn’t mandatory, but it’s pretty common. It didn’t take long before we noticed a trend similar to what that German man discovered in America. There’s one thing Londoners seem to want to discuss with Americans. That one thing is guns.
Why do you have so much gun violence? Why do you have so many guns? The answer is: I don’t fscking know. Now if Your JoeDog had a modicum of interest in guns these inquiries may have been amusing. He may have enjoyed the engagement. But like that German in McSorley’s Old Ale House, Your JoeDog grew tired fast. Yes, America is armed to the teeth. Yes, America loves its guns. Your JoeDog is not part of that culture. The most lethal weapon he’s ever fired was a toy water gun.
Can we talk about German beer and German cars instead?
With next to no fanfare you’re JoeDog released siege-3.1.2.
Awesome! What’s new? We moved an include directive from one file to another. Exciting! Wait – what?
Basically this means siege-3.1.2 should compile in more environments than siege-3.1.1.
Oh, well that’s something … I guess.
When it snows, the snow is everywhere. It’s on the roads, it’s on the sidewalks but it’s also on the media. Local news covers it. National news covers it. Cable news and newspapers cover it. If you want to know about non-snow issues at the height of a snow storm, you’re fscked. Everybody’s covering snow.
They report it because people are interested and it’s easy to do. Send a reporter into the street. See that white shit falling on his head? That’s Goddamn snow! How much is going to fall? We don’t know, between one and a million inches.
The pope is in the US right now and he’s on all the channels. There’s probably newsworthy events also taking place — you know, things that affect our lives — but we don’t know about them. Why? Because everyone’s covering the Goddamn pope. The pope is basically snow.
Last night, NBC News covered a bus accident in Seattle and devoted the rest of the broadcast to the pope. Did he cure a leper? No, he got in his stupid Popemobile and cruised down Fifth Avenue to St. Patrick’s Cathedral. What did he do when he arrived there? He went inside.
Is this newsworthy? Your JoeDog’s not a religious sort but he’s pretty certain the pope’s been to church before yesterday. He pretty much lives in one.
It’s cool that the pope recognizes climate change and irritates the moralizing wing of the Republican party but Your JoeDog can’t wait until he gets back on that plane to Leipzig or wherever the hell he’s from…
The Times posed this question a few days ago but Your JoeDog is catching up: “Whatever happened to German-America?” The short answer is this: two Goddamn global wars. After those bitter conflicts they didn’t feel like being German any more.
In the late 80s, Your JoeDog moved into an Upper East Side apartment in Yorkville. That’s a Manhattan neighborhood also known as Germantown. It stretches from the East River to Lexington Avenue. It was there that he witnessed first hand the dying of the German-American light.
Back then, it was filled with German stores, delis, bakeries and bars. The Viennese were there, too. Their pastries could brighten any morning. With two exceptions, these businesses were in their dying throes. The owners were old and the help was even older.
On Third Avenue, there were yellow pre-war tenement buildings. On top of those buildings you could find swastikas formed with brown bricks against a yellow background. They remained on those buildings until the early 21st Century. When they were laid by German-American construction crews, Hitler was not yet revealed as evil. Yet they remained on display long after the world knew he was a monster. The brown emblem was eventually blasted away but you can still see the Nazi symbol thanks to its brick outline.
Now almost nothing is left of Germantown except the venerable Heidelberg Restaurant on 86th and Second. Your JoeDog still visits that bar several times a year. A little while back we met an old Czech woman at the bar. She nursed a Jaegermeister with a beer and finished both drinks at the same time.
She was an ethnic German from the Sudetenland. In 1945, she hid under a bridge as the Red Army marched over it. She was there with her sisters and a cousin. The Red Army was raping its way across Eastern Europe then. We weren’t sure if she was unscathed but they didn’t find her on that particular day.
She emigrated to New York in 1948 and never lived more than a few blocks from her original apartment. I told her Barack Obama — a German-American — lived in this neighborhood, about a block from my old apartment. This greatly excited her and she announced it to everyone at the bar. Then I reminisced about the Old Neighborhood and that made her even more excited. Finally she met someone who still remembered it.
Your JoeDog plans to put the brakes on you people.
The number one we email message we receive goes something like this: “When we hit the server with 60 billion users we get socket timeouts. How do we correct that?” Short answer: configure your web server with a pool of 60 billion threads.
If you run more siege users than you have web handlers, requests start to queue up and sockets start to timeout. Out of the box, apache has a pool of 256 handlers. So if you run siege with 1024 users, 768 of them are waiting for a thread to handle their request.
So this is where the brakes come in. Your JoeDog plans to add a 255 thread limit in the siegerc file. If you configure more users than that, it will warn you and reduce the pool to 255. You can override the limit inside the file but in the comments you’ll find this lecture splaining why you shouldn’t do that without configuring your site for high capacity.