COP 4610: Operating Systems & Concurrent Programming

Queueing Theory & Other Analysis Techniques


The objective of these notes is to make you aware of the existence of analytic techniques for predicting OS performance, to give you a flavor of how they work, and to give some examples of simple results. We are limited in how deep we can go without making assumptions about background in probability theory. The only actual result we expect everyone to remember and be able to apply is Little's Result. There is a very nice "picture proof" of this result accessible via a link further below, which we go over in class. I hope you will read and follow it. You may even see it on an examination.

The ".jpg" figures in these notes are reproduced for the use of students in this course from William Stallings' textbook and on-line notes on Queueing Analysis ( You can read that document if you would like to see all the words that go with the figures.

There are some other nice notes, due to G.A. Vignaux, at These include a very good explanation of statistical equilibrium, Markov processes, and Little's Result. I strongly recommend looking at them, as they are better than what Stallings wrote on the subject.

OS Performance Analysis Techniques

  1. Build the new/modified system, instrument it, and test it
  2. Try to extrapolate from performance of the most similar existing systems
  3. Build a mathematical model, and analyze it (Queueing Theory)
  4. Build a software simulation model, instrument it, and run it

Stallings' textbook may give the impression that there is no interesting mathematical theory underlying operating systems. That is not true. The textbook simply plays down the theory, to broaden the potential readership.

There are several areas with well developed theory, some bits of which pop up here and there in Stallings text. These examples include:

Queueing theory, originally developed by industrial engineers for optimizing the flow of work in businesses, has also been very useful for modeling the flow of work in computers and networks. Without much background in probability, we will not be able to do much with queueing theory in this course. However, there is one simple and very useful result that does not require much knowledge of probability. That is called Little's Result. We will look at queueing theory at a very superficial level, and in particular at one simple result from queueing theory, known as Little's Result.

Queueing theory is mentioned on pp 414-416 in the text.

More complex systems can be studied using simulations. Simulation is discussed on pp 417-419 in text.

Projected Versus Actual Response Time

Projection can be inaccurate. The figure compares actual response time of a system against a projection made from actual data up to a load of 0.5. The projection is based on fitting a 3rd-order polynomial to the known data.

Queueing Theory

Single Server Queue

The most basic queuing model is the single-server queue.

Single Server Queue

maximum input rate = 1 / average service time:

Queues become very large (grow exponentially) as utilization (ρ) approaches 1.


Multiserver Queue

Multiserver Queue

Theoretical maximum utilization (N * ρ) is N * 100%, so theoretical maximum input rate is N / average service time.

Multiple Single-Server Queues

Queueing System Notation (1)

Note that "r" above (taken from texbook) is not residence time, but the number of resident work items.

Also, the terms "traffic intensity" and "utilization" are generally interpreted as meaning the same thing.

Queueing System Notation (2)

Queueing Theory Questions

How to outputs relate to inputs? e.g., how their averages relate?

X/Y/N Classification of Queueing Models

e.g., M/M/1 refers to single-server model with Poisson arrival rate and exponential service times

Poisson Distribution: Probablity Density

The Poisson distribution is for a discrete-valued random variable, such as the number of arrivals that occur within a fixed lengh (e.g., unit) time interval.

(Inverse) Exponential Distribution: Formulae

Here, 1 / λ is the mean value.

Exponential Distribution: Probability Distribution Function

Exponential Distribution: Probability Density Function

It can be shown that the exponential distribution is the only distribution that has the "memoryless" property. That is, the probability that the time until the next arrival at the (any) current time is independent of when the last preceding arrival occurred.

Basic M/G/1 (General) Queueing Relationships

When standard deviation = mean, the service time is exponential, so we have M/M/1.

When standard deviation is zero, the service time is constant, so we have M/D/1.

Little's Result/Law

This is a very valuable observation that has a broad range of applications and is worth pondering. I has been stated in many ways.

John Bentley, in his Programming Pearls (a very good book, which I hope to finish reading some day!) gives an example:

Bentley also cites the following, attributing them to Peter Denning:

A Good Explanation

A very nice explanation of Little's Law is given at

Please read it. It is better than anything Stallings wrote on the subject, or these notes. I have only provided a link to it, rather than embedding it here, out of respect for the author's copyright.

Mean Queue Size for Single-Server Queue

The figure shows the relationship of average queue size to utilization, for several different distributions. The coefficient of variation is std deviation of service time divided by the mean service time, and is a measure of variability in the distribution.

The graph for mean residence time looks very similar. This makes sense, if we figure the longer the queue the longer the residence time.

M/G/1 Service Times

M/M/1 Service Times

M/D/1 Service Times

Example of a Network of Queues

Traffic Partitioning

Traffic Merging

Simple Tandem Queue (Sequencing)

Jackson's Theorem

Network of m nodes with independent exponential service.

Outside arrivals have a Poisson rate (i.e., exponential inter-arrival times).

Once served at a node, an item goes immediately to one of the other nodes with a fixed probability, or out of the system.

Says a network of queues, each node may be analyzed separately from the others using the M/M/1 or M/M/N model, and the results combined using ordinary statistical methods.

Example Queuing Theory Application: DB Server

ρ = λ Ts
    = (20 arrivals/minute)(0.6 seconds/transmission)/(60 sec/min)
    = 0.2

avg response time = Tr
    = Ts/(1 - ρ)
    = 0.6/(1 - 0.2) = 0.75 seconds

© 2002, 2005 T. P. Baker & Florida State University. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without written permission. (Last updated by $Author: cop4610 $ on $Date: 2002/09/26 20:06:06 $.)