FSU Seal - 1851

COT 5405
Advanced Algorithms
Chris Lacher
Notes 1: Intro

Algorithm


Algorithm Concepts


Insert Sort

Insert-Sort ( array of numbers A )
{
  for (j = 2; j <= length(A); ++j)
  {
    // Loop Invariant: A[1..j-1] is sorted
    key = A[j];
    // insert A[j] into A[1..j-1]
    i = j - 1;
    while (i > 0 and A[i] > key)
    {
      A[i+1] = A[i];
      i = i-1;
    }
    A[i+1] = key;
  }
  return;
}
  1. Initialization
    LI[2] true
  2. Maintenance
    LI[j-1] implies LI[j]
  3. Conclusion
    LI[n] proves correctness

Analysis of Insertion Sort

                                     incr cost   times
Insert-Sort ( array of numbers A )   ---------   -----
{
  for (j = 2; j <= length(A); ++j)       c1        n
  {
    key = A[j];                          c2        n-1
    i = j - 1;                           c3        n-1
    while (i > 0 and A[i] > key)         c4        SUM[2,n] tj
    {
      A[i+1] = A[i];                     c5        SUM[2,n] (tj -1)
      i = i-1;                           c6        SUM[2,n] (tj -1)
    }
    A[i+1] = key;                        c7        n-1
  }
  return;
}

where tj is the number of times the while loop header is executed for that particular j (dependent on input data instance).

Observations: The value of tj-1 is the number of elements of A[1..j-1] that are greater than key, obviously a data-dependent value.

Total Cost = c1n + c2(n -1) + c3(n -1) + c4Σ2n tj + c5Σ2n (tj -1) + c6Σ2n (tj -1) + c7 (n -1)

Best Case Cost

= (c1 + c2 + c3 + c4 + c7)n - (c2 + c3 + c4 + c7)
= An + B, for some constants A and B.

Worst Case Cost

= c1n + c2(n -1) + c3(n -1) + c4Σ2n j + c5Σ2n (j -1) + c6Σ2n (j -1) + c7 (n -1)
= An2 + Bn + C for some constants A, B, and C (using the fact that Σ1n j = n(n -1)/2)

Average Case Cost

= c1n + c2(n -1) + c3(n -1) + c4Σ2n (j /2) + c5Σ2n (j /2 -1) + c6Σ2n (j /2 -1) + c7 (n -1)
= An2 + Bn + C for some constants A, B, and C (using the fact that Σ1n (j /2) = n(n -1)/4)


Asymptotic Notation

Theorem 1 (transitivity).
(1) If f(n) is in O(g(n)) and g(n) is in O(h(n)) then f(n) is in O(h(n))
(2) If f(n) is in Ω(g(n)) and g(n) is in Ω(h(n)) then f(n) is in Ω(h(n))
(3) If f(n) is in Θ(g(n)) and g(n) is in Θ(h(n)) then f(n) is in Θ(h(n))

Theorem 2 (anti-symmetry). f(n) is in O(g(n)) iff g(n) is in Ω(f(n)).

Theorem 3 (symmetry). f(n) is in Θ(g(n)) iff g(n) is in Θ(f(n)).

Theorem 4 (reflexivity). A function is asymptotically related to itself: f(n) is in O(f(n)), f(n) is in Ω(f(n)), and f(n) is in Θ(f(n)).

In particular, of the three, Θ defines an equivalence relation on functions while O and Ω define an anti-symmetric pair of relations on functions that is analogous to the pair of order relations (<=, >=) on numbers. Thus, Θ is analogous to '=' (equality), while O is analogous to '<=' (less than or equal to) and Ω is analogous to '>=' (greater than or equal to). There is even a form of the dichotomy property of order relations:

Theorem 5 (dichotomy). If f(n) <= O(g(n)) and g(n) <= O(f(n)) then f(n) = Θ(g(n)).

All of these theorems can be restated in an equivalent form using set notation. For example, Theorem 5 restates as follows:

Theorem 5a (dichotomy). If O(f(n)) subset_of O(g(n)) and O(g(n)) subset_of O(f(n)) then Θ(f(n)) = Θ(g(n))).

We will use the notation of equality and inequality, as shown in the slide, as a notational device that helps reinforce our perception of the nature of these three relations. (We recognize that this is an abuse of notation. See the discussion of notation "abuse" and notation "misuse" in [Cormen].)


Insertion Sort Asymptotics