up arrow Concurrency and the Single Siege

We’re frequently asked about concurrency. When a siege is finished, one of its characteristics is “Concurrency” which is described with a decimal number. This stat is known to make eyebrows furl. People want to know, “What the hell does that mean?”

In computer science, concurrency is a trait of systems that handle two or more simultaneous processes. Those processes may be executed by multiple cores, processors or threads. From siege’s perspective, they may even be handled by separate nodes in a server cluster.

When the run is over, we try to infer how many processes, on average, were executed simultaneously the web server. The calculation is simple: total transactions divided by elapsed time. If we did 100 transactions in 10 seconds, then our concurrency was 10.00.

Bigger is not always better

Generally, web servers are prized for their ability to handle simultaneous connections. Maybe your benchmark run was 100 transactions in 10 seconds. Then you tuned your server and your final run was 100 transactions in five seconds. That is good. Concurrency rose as the elapsed time fell.

But sometimes high concurrency is a trait of a poorly functioning website. The longer it takes to process a transaction, the more likely they are to queue.  When the queue swells, concurrency rises. The reasons for this rise can vary. An obvious cause is load.  If a server has more connections than thread handlers, requests are going to queue. Another is competence – poorly written apps can take longer to complete then well-written ones.

We can illustrate this point with an obvious example. I ran siege against a two-node clustered website. My concurrency was 6.97. Then I took a node away and ran the same run against the same page. My concurrency rose to 18.33. At the same time, my elapsed time was extended 65%.

Sweeping conclusions

Concurrency must be evaluated in context. If it rises while the elapsed time falls, then that’s a Good Thing™. But if rises while the elapsed time increases, then Not So Much™. When you reach the point where concurrency rises and elapsed time is extended, then it might be time to consider more capacity.

 

  • http://liataja.com Ahmad

    is it possible to calculate the concurrency manually? or only calculated by the system inside? thanks

    • http://www.joedog.org/ JoeDog

      Concurrency is a simple formula: It’s the sum of all transaction times divided by the total elapsed time. That tells us how many processes were handled simultaneously.

      • http://liataja.com Ahmad

        I am sorry, I am still confused. isn’t it like a formula for the transaction rate?

        in an experiment to test image files using 10 simultaneous requests for 60 seconds and resulted 703 transactions and concurrency : 3.73 for Apache web server. It also resulted 695 transactions and 4.13 concurency for Nginx. How do I prove it? thanks

        • http://www.joedog.org/ JoeDog

          I edited the previous comment for more clarity. For concurrency we take the sum of all transaction times (from connection to socket close) divided by the elapsed time (how long siege ran).

          The transaction rate is the count of all requests divided by elapsed time. You can see the calculations in the file data.c

          It’s not clear what you’re trying to prove in the scenario you provided.

          • http://liataja.com Ahmad

            Ooo i seee.. the key is “sum of all transaction times”, Thankyou very much..

  • Yaoyu He

    What is the difference between transaction rate and ‘concurrency’? And how is it calculated? Please kindly help note.

    • http://www.joedog.org/ JoeDog

      Concurrency is the sum of all transaction times (from connection to socket close) divided by the elapsed time (how long siege ran).

      Transaction rate is the count of all requests divided by elapsed time.

      • MahdiXareie

        I really admire your patience 🙂

  • MahdiXareie

    I think runing siege with -t is a good way to compare system performance .
    by keeping test time constant we can easily check for higher transactions counts and optimize our web server settings for the best.
    thanks for great post ^_^

  • fzzzt

    I know this post is old, but it still seems relevant to the application, which is still useful!

    I don’t understand your definition of concurrency, even after reading the comments. I would assume it means “how many requests the server can handle at once”, which is sort of the definition of concurrency, however that assumption seems to be wrong given the math. Maybe it’s just a poorly named metric given my understanding. I understand the transaction rate, and it seems like what you describe in the article as concurrency (100 transactions in 10 seconds) is actually describing the transaction rate. I would think an example of your concurrency value would be more like “100 transactions took 10 seconds in 12 seconds”, meaning a rate of 10 and “concurrency” of 8.33, I think, which doesn’t really make much sense to me.

    Say the concurrency of a server is, in reality, 10–meaning it can actually handle 10 requests at once due to RAM limits (it’s mod_php ;)), and it takes 2 seconds per request, to make the math easier. If we run siege with 5 requests, it takes 2 seconds to run, has a rate of 5 and a concurrency value of 5. If we run siege with 10 requests, it should _still_ finish in roughly 2 seconds, have a rate of 5 and I think a concurrency value of 10 (10*2/2).

    Now, if we run it with 20 requests, the first 10 will happen the same as before, the next 10 will be queued and take twice as long, so overall it takes 4 seconds. The rate would be 5 still, from 20/4, but the concurrency would be 15 (10*2+10*4/4)? So “concurrency” rose, even though it isn’t handling more at once. Take 50 requests (10 sec total): rate is (50/10)=5 still, concurrency is ((10*2 + 10*4 + 10*6 + 10*8 + 10*10)/10) = 30. It seems more like an average completion time.

    I guess either I don’t understand the calculation or I don’t understand the purpose or usefulness of the concurrency value. I probably don’t get the calculation, but it doesn’t seem like this would actually indicate what it’s supposed to indicate. I’m guessing it has something with repeating requests over time rather than blasting them all at once, and the theory that requests shifted later would complete more quickly, or something. Even after drawing out some time diagrams I don’t think I would get the correct value, so I must be missing something.

    To actually determine the concurrency, I would think you need to keep hitting with more requests in synchronized waves until you start filling up the queue (i.e., until the completion time increases significantly), and detect at which point that happens with some sort of delta threshold, so you would really need something like bombard. Using the above server as an example: 1 req = 2s, 2 req = 2s, 5 req = 2s, 10 req = 2sec, 15 req = 4s (greater than 2s, so we back off until we hit 2s again), 13 req = 4s, 11 req = 4s, 10 req = 2s, levels out at 10, with a delta of <1s. Something like that. Of course this is more procedural, less math and extrapolation, and maybe isn't even possible to do reliably, I dunno…

    Anyway, these are great tools, even if I don't understand this one part of them. Thanks. 🙂