FSU Seal - 1851 Course Organizer
COT 5405 Advanced Algorithms
Fall Semester 2009
Root View: Course Components
Syllabus The course syllabus establishes course policies on grading, attendance, and exams. The syllabus should be read in detail at the beginning of the semester.
Course Notes These are supplemental to the assigned reading and are released through the calendar. Note: The course notes are undergoing revision during the semester.
Calendar The course calendar: coverage, assignments, lecture notes all released here.
Assignments Assignments are intended to be enhance the learning experience by exertcising and grounding concepts. Assignments will be released through the calendar. Note that assignment deadlines will be enforced and that successful completion of most assignments requires starting at least two weeks in advance of the deadline. We expect polished, thoughtfully prepared work and will assess accordingly.
My FSU The FSU/Blackboard Portal, where you will find this course. The course site is the main communication resource for the class. Here you can get help, talk to other students, retrieve your grades, and generally keep up with course news and announcements.
Office Hours Instructor office hours and contact information
Effective Aug 24 - Dec 11, 2009 (excluding Sep 7, Nov 11, and Nov 25-27)
Extras: Miscellaneous Resources and References
The lecture notes from the pre-requisite courses
FAQ Frequently Asked Questions harvested from discussion boards
Make A tutorial on the make utility and makefiles
C++ Style C++/Java Coding Style and Standards
Assign1 Sol
Assign2 Sol
Assign3 Sol
Note: Assignment 1 Problem 3(e) (Problem 4-5(e) in the text) is a fact used in the proof that AVL trees have height = Θ(log n).
COT 5405 Course Calendar
WeekDateTopicsAssignments Due Date
1 Details 8/25 Course Syllabus, Asymptotics Assignment 1 9/13
2 Details 9/1 Recurrences    
3 Details 9/8 Sorting & Searching Assignment 2 9/27
4 Details 9/15 Positional and Associative Data Structures    
5 Details 9/22 UUSet, Table, Associative Array Assignment 3 10/11
6 Details 9/29 Ordered Set, Map; Binary Trees and Iterators; Binary Search Trees    
7 Details 10/6 Graph Representations and Basic Algorithms Assignment 4 10/25
8 Details 10/13 Advanced Graph Algorithms    
9 Details 10/20 Advanced Graph Algorithms (cont)    
10 10/27 Midterm Exam: Covers weeks 1-9 See Exam Schedule for details
11 Details 11/3 Introduction to Genetic Algorithms    
12 Details 11/10 Introduction to Neural Networks    
13 Details 11/17 Backpropagation Learning    
14 Details 11/24 TBA    
15 Details 12/1 TBA    
16 Details 12/8 Final Exam: Covers weeks 1-16 See Exam Schedule for details
Note: Otherwise uncited references to chapters, sections, exercises, and problems refer to the course textbook Introduction to Algorithms, T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, second edition, MIT press, 2001 (ISBN 0-262-03293-7).
Details: Week 1
Topics: Introduction
Growth of Functions
Analyzing Algorithms
Objectives: At the end of this class the student should be able to:
  1. Describe the course policies and grading criteria.
  2. Define algorithm in the strict sense and relate the definition to the meaning of the word in the text.
  3. Define Big O, Big Theta, Big Omega, little o, and little omega and state and prove the basic relationships among them.
  4. Describe the algorithm hypotheses and body for InsertionSort.
  5. Prove correctness of InsertionSort.
  6. Provide runtime analysis for InsertionSort.
Reading: Course Syllabus Textbook: Chapters 1, 2, 3
Suplemental: Notes 1: Introduction to Algorithms
Exercises: 1.2-3, 2.1-3, 2.2-2, 2.3-3, 3.1-3, 3.1-4, 3.2-3
Assignment: Assignment 1 Problem 1: Problem 3-1 on p. 57
Assignment 1 Problem 2: Problem 3-2 on p. 58
Details: Week 2
Topics: Recurrences
Objectives: At the end of this class the student should be able to:
  1. Apply the substitution method and tree method to obtain asymptotic solutions of recurrences.
  2. Solve the Fibonacci recurrence exactly (using the characteristic polynomial) and asymptotically
  3. State the Master Method for finding asymptotic solutions for recurrences.
  4. Apply the Master method to find asymptotic bounds on certain recurrences.
Reading: Textbook: Chapters 4, 6, 7
Suplemental: Notes 2: Recurrences
Exercises: 4.3-1, 4.3-3, 6.1-1, 6.1-3, 6.1-5, 6.4-3, 6.4-4,
Assignment: Assignment 1 Problem 3: Problem 4-5 on p. 86
Assignment 1 Problem 4: Problem 6-1 on p. 142
Assignment 1 Problem 5: Problem 7-4 on p. 162
Details: Week 3
Topics: Sorting & Searching
Objectives: At the end of this class the student should be able to:
  1. Describe the algorithm hypotheses and body for InsertionSort, SelectionSort, MergeSort, HeapSort, and QuickSort
  2. Give the best, average, and worst case runtimes for InsertionSort, SelectionSort, MergeSort, HeapSort, and QuickSort, and outline the proofs.
  3. Prove correctness of InsertionSort, SelectionSort, MergeSort,HeapSort, and QuickSort
  4. Derive and prove the best case asymptotic runtime for a comparison sort, and apply the results to InsertionSort, SelectionSort, MergeSort, HeapSort and QuickSort
  5. Describe the pros/cons of choosing one of these sorts over another in a given practical setting.
  6. Describe the algorithm hypotheses and body for SequentialSearch and BinarySearch
  7. Prove correctness of SequentialSearch and BinarySearch.
  8. Derive the worst and average case runtime of SequentialSearch and BinarySearch
Reading: Textbook: Textbook: Chapters 6, 7, 8.1
Suplemental: Notes 3: Sorting
Notes 4: Searching
Exercises: 7.1-1, 7.1-3, 7.2-3, 7.2-4, 7.4-2
Halting of BinarySearch
Correctness of BinarySearch
Runtime analysis of BinarySearch
Runspace analysis of BinarySearch
Assignment: Assignment 1 Problem 6: Prove correctness (including halting) of SelectionSort (use loop invariants)
Assignment 1 Problem 7: Provide worst and average case runtime analysis of SelectionSort
Assignment 1 Problem 8: Provide runspace analysis of SelectionSort
Assignment 2 Problem 1: Prove correctness of SequentialSearch
Assignment 2 Problem 2: Worst, average, and best case runtime analysis of SequentialSearch
Assignment 2 Problem 3: State and prove correct the iterative UpperBound algorithm (see notes)
Assignment 2 Problem 4: Give a complete runtime analysys for iterative UpperBound
Assignment 2 Problem 5: Explain why "short circuit bailout" in any of the binary search algorithms is not cost effective
Details: Week 4
Topics: Positional and Associative Data Structures
Objectives: At the end of this class the student should be able to:
  1. Define each of the following ADTs in terms of public interface or operations:
    1. Vector
    2. List
    3. Deque
    4. Stack
    5. Queue
    6. Set (ordered | unordered)
    7. MultiSet (ordered | unordered)
    8. Table / Map / Associative Array (ordered | unordered)
    9. MultiMap (ordered | unordered)
  2. Classify these ADTs as positional or associative
  3. Give runtime requirements for the operations in the positional ADTs above
  4. Discuss tradeoffs for various runtime requirements for the operations in the associative ADTs above
Reading: Textbook: Chapters 10, 12
Suplemental: Notes 5: Sequential ADTs and Data Structures
Notes 6: Associative ADTs
Exercises: 10.2-1, 10.2-5, 10.3-1, 12.1-1, 12.3-3
Assignment: Assignment 2 Problem 6: Exercise 10.2-7 on p. 209
Assignment 2 Problem 7: Exercise 12.1-3 on p. 256
Assignment 2 Problem 8: Devise an algorithm that uses a Deque for control and such that using Push/Pop at the same end of the deque implements depth-first search and using Push/Pop at opposite ends of the deque implements breadth-first search, otherwise the two algorithms are identical.
Details: Week 5
Topics: UUSet, Table, Associative Array
Objectives: At the end of this class the student should be able to:
  1. Define and give examples of hash function
  2. Show the structure of hash table implemented with chaining
  3. Define uniform hashing
  4. State the expected access time for hash tables, and prove these assertions
  5. Given a specific hash function and parameters defining a hash table, illustrate the internal (private) hash table structure after sequences of insert and remove operations
Reading: Textbook: Chapter 11
Suplemental: Notes 7: Searching in Hash Structures
Exercises: 11.2-2, 11.2-3
Assignment: Assignment 3 Problem 1: Exercise 11.2-3 on p. 229
Assignment 3 Problem 2: Exercise 11.2-5 on p. 229
Details: Week 6
Topics: Ordered Set, Map
Binary Trees
Binary Tree Iterators
Binary Search Trees as ordered binary trees
Sets as Binary Search Trees
AVL [Height-Balanced] Binary Search Trees
Objectives: At the end of this class the student should be able to:
  1. Discuss representation of trees and binary trees using dynamically created nodes
  2. Discuss binary tree iterators
  3. Explain why the increment operator ++ for a binary tree iterator has amortized constant runtime
  4. Define Binary Search Tree (BST) in terms of order on a Binary Tree
  5. Show that binary tree iterators encounter BST vertices in sorted order
  6. Describe the BST Search, Insert, and Delete algorithms
  7. Estimate the asymptotic runtime of BST Search
  8. Define height-balanced trees and show that they are special cases of binary trees
  9. Describe the BST implementation of Set
  10. Describe runtimes for the various Set operations when implemented as a height-balanced BST.
Reading: Textbook: Appendix B, Chapter 12
Suplemental: Notes 8: Binary Trees
Assignment: Assignment 3 Problem 3: For the following binary tree, complete the table showing each step in a traversal and the number of edges covered by each step (begin, next, ... , next):
                      iteration step  current location  no of edge moves
         Q            --------------  ----------------  ----------------
      /     \         1. initialize
     W       E        2. ++
      \     / \       3. ++
       R   T   Y      ...
                      7. ++           (null)
Assignment 3 Problem 4: Describe an order in which vertices of a BST can be saved to ensure that reconstructing a BST from the saved data will result in the same tree. Argue correctness of your assertion.

Assignment 3 Problem 5: Use the result of Assignment 1 Problem 3(e) [problem 4-5(e) in the textbook] to prove: Height-balanced binary trees satisfy height(n) = Θ(log n), where n = number of vertices in the tree.
Details: Week 7
Topics: Graphs, Digraphs, Breadth First Search (BFS), Depth First Search (DFS)
Objectives: At the end of this class the student should be able to:
  1. Describe and implement adjacency list representations of graphs and digraphs
  2. Describe and implement adjacency matrix representations of graphs and digraphs
  3. State and analyze the breadth first search algorithm
  4. State and analyze the depth first search algorithm
  5. State and analyze two distinct versions of the topological sort algorithm
  6. Define Graph, Directed Graph (Digraph), and Network
  7. Describe adjacency matrix representations of graphs, digraphs, and networks
  8. Describe adjacency list representations of graphs, digraphs, and networks
  9. Discuss how additional vertex and/or edge data can be assoicated with these representations
  10. Describe the BFS algorithm on a graph or digraph G with Queue control
  11. Explain why the runtime of BFS is O(V + E) (where V is the number of vertices of G and and E is the number of edges of G)
  12. Define the BFS Tree of a BFS search of G
  13. Explain the relationship between the BFS tree and shortest paths in G
Reading: Chapter 22 of [Cormen]
Suplemental: Notes 9: Fundamental Graph Algorithms
Exercises: Modify the code for the DFS class so that it prints each edge of a digraph along with its type (tree, back, forward, cross). How would this work for an undirected graph?
Exercise 22.1-1 on p. 530
Exercise 22.2-2 on p. 538
Assignment: Assignment 4 Problem 1: Implement the BFS class for some category of adjacency list representations of graphs and digraphs. You can invent your own or use [LIB]/igraph. Graphs or digraphs should be instantiatable from a file containing the number of vertices followed by any number of vertex pairs representing edges.

Assignment 4 Problem 2: Implement the DFS class for the same category of adjacency list representations of graphs and digraphs.

Assignment 4 Problem 3: Implement Topological Sort, the version that builds on DFS, for the same class of digraphs.
Details: Week 8
Topics: Depth First Search (DFS), Topological Sort
Objectives: At the end of this class the student should be able to:
  1. Describe the DFS algorithm with Stack control
  2. Explain why the runtime of DFS is O(V + E)
  3. Define the DFS Forrest of a DFS search
  4. Define the DFS [discovered,finished] interval for the vertices of G, and explain the Parenthesis Theorem
  5. Describe the Topological Sort problem for a Digraph
  6. Explain how DFS can produce a Topological Sort
Reading: Textbook: Chapters
Suplemental: Notes 9 (cont): Graph Algorithms
Exercises: Exercise 22.4-1 on p. 551
Exercise 22.4-5 on p. 552
Details: Week 9
Topics: Minimum Spanning Tree and Shortest Path Algorithms
Objectives: At the end of this class the student should be able to:
  1. Define Prim's algorithm as a class similar to BFS
  2. Define Kruskal's algorithm
  3. Derive the asymptotic runtime of Kruskal's algorithm
  4. Derive the asymptotic runtime of Prim's algorithm
  5. Explain why the runtime of Prim is asympticically slower than BFS
Reading: Textbook: Chapters
Suplemental: Notes 10: Graph Algorithms 2
Exercises: Experiment with traces of Prim and Kruskal on specific weighted graphs. Can you find cases where:
   MST is not unique
   Prim and Kruskal give different MST
Assignment: Assignment 4 Problem 4: Modify your solution to problem 2 to obtain Prim's algorithm. Test and experiment.

Assignment 4 Problem 5: Use your experience-gathered knowledge to propose a class-based implementation of Kruskal's algorithm. Test as usual, and discuss the results.

Notes for Assignment 4:
  1. Your program(s) should be working code, so that it is clear how to compile and run them. (A makefile would be helpful for multifile projects.)
  2. Your program(s) should be able to instantiate a graph from a file that begins with (1) one line of documentation, then (2) specifies the number of vertices and then (3) has any number of edges (with weights if it is a weighted graph), specifying from, to, [and weight]. Example files:
    directed graph: cycle of length 5
    0 1 1 2 2 3 3 4 4 0 
    undirected graph: cycle of length 5
    0 1
    1 2
    2 3
    3 4
    4 0
    weighted graph (undirected): cycle of length 5
    0 1 1.5
    1 2 2.3
    2 3 6.0
    3 4 2.1
    4 0 0.5
  3. Note that the first line of the file serves as both documentation and as the signal for what kind of entity it is by the first character in the line:
    1. 'g' or 'G' or 'u' or 'U' implies Graph (undirected)
    2. 'd' or 'D' implies directed graph
    3. 'w' or 'W' implies weighted graph
    4. 'n' or 'N' implies network (weighted directed graph)
    Thus you can just read the first non-clearspace character in the file and that tells you what you are building. Then skip the rest of that line and start reading numerical data, which is interpreted depending on the type of entity represented. (We are not doing any network algorithms, but I threw that in to allow for a complete set of possibilites.) You can assume the files are correctly configured!
Details: Week 11
Topics: Introduction to Genetic Algorithms
Objectives: At the end of this class the student should be able to:
  1. State the generic genetic algorithm (GA)
  2. Define these terms in the context of genetic algorithms: population, fitness function, fitness-proportionate selection, crossover, mutation, fitness optimization criteria
  3. Give a detailed trace of an example GA optimization
  4. Define schema / schemata
  5. For a schema, define: order, defining length
  6. State and explain the Schema Theorem
Suplemental: Notes GA: Introduction to Genetic Algorithms
Details: Week 12
Topics: McCulloch-Pitts Neuron; Perceptrons & Perception Learning Rule; Supervised Learning; Analog Units; Adaline
Objectives: At the end of this class the student should be able to:
  1. Describe an artificial neuron in terms of input, processing [combining function] and output [firing function]
  2. Describe a connection between two artificial neurons in terms of functionality [synaptic function or weight]
  3. Describe a neural network and give examples of feedforward [acyclic] and feedback [recurrent] neural topologies
  4. Describe the principal categorization of learning algorithms: supervised and unsupervised. Also describe reinforcement learning and place it into the major categorization.
  5. Describe and derive the "delta-rule" learning mechanism for the linear classifyer architecture [single-layer, feed-forward, linear synapses, sigmoidal firing function] [similar to Adaline]
Suplemental: Notes NN: Introduction to Neural Networks (Chapters 2,3,6,7,8,9; skim the others)
Exercises: Read public sources such as Wikipedia to enhance your ability with the stated objectives above
Assignment: Find (legally copyable) source code for Backpropagation learning and install on your workstation. Run learning tests for the 4-2-4 and 8-3-8 encoder/decoder problems.