Course Organizer

COT 5405Advanced Algorithms

Fall Semester 2009

Root View:Course ComponentsSyllabus The course syllabus establishes course policies on grading, attendance, and exams. The syllabus should be read in detail at the beginning of the semester. Course Notes These are supplemental to the assigned reading and are released through the calendar. Note: The course notes are undergoing revision during the semester.Calendar The course calendar: coverage, assignments, lecture notes all released here. Assignments Assignments are intended to be enhance the learning experience by exertcising and grounding concepts. Assignments will be released through the calendar. Note that assignment deadlines will be enforced and that successful completion of most assignments requires starting at least two weeks in advance of the deadline.We expect polished, thoughtfully prepared work and will assess accordingly.My FSU The FSU/Blackboard Portal, where you will find this course. The course site is the main communication resource for the class. Here you can get help, talk to other students, retrieve your grades, and generally keep up with course news and announcements. Office Hours Instructor office hours and contact information

Effective Aug 24 - Dec 11, 2009 (excluding Sep 7, Nov 11, and Nov 25-27)

Extras:Miscellaneous Resources and ReferencesCOP4531

MAD2104

MAD3105

The lecture notes from the pre-requisite courses FAQ Frequently Asked Questions harvested from discussion boards Make A tutorial on the make utility and makefiles C++ Style C++/Java Coding Style and Standards Assign1 Sol

Assign2 Sol

Assign3 Sol

Note:Assignment 1 Problem 3(e) (Problem 4-5(e) in the text) is a fact used in the proof that AVL trees have height = Θ(log n).

COT 5405 Course CalendarWeekDateTopicsAssignmentsDue Date1 Details 8/25 Course Syllabus, Asymptotics Assignment 1 9/13 2 Details 9/1 Recurrences 3 Details 9/8 Sorting & Searching Assignment 2 9/27 4 Details 9/15 Positional and Associative Data Structures 5 Details 9/22 UUSet, Table, Associative Array Assignment 3 10/11 6 Details 9/29 Ordered Set, Map; Binary Trees and Iterators; Binary Search Trees 7 Details 10/6 Graph Representations and Basic Algorithms Assignment 4 10/25 8 Details 10/13 Advanced Graph Algorithms 9 Details 10/20 Advanced Graph Algorithms (cont) 10 10/27 Midterm Exam: Covers weeks 1-9 See Exam Schedule for details 11 Details 11/3 Introduction to Genetic Algorithms 12 Details 11/10 Introduction to Neural Networks 13 Details 11/17 Backpropagation Learning 14 Details 11/24 TBA 15 Details 12/1 TBA 16 Details 12/8 Final Exam: Covers weeks 1-16 See Exam Schedule for details

Note: Otherwise uncited references to chapters, sections, exercises, and problems refer to the course textbookIntroduction to Algorithms, T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, second edition, MIT press, 2001 (ISBN 0-262-03293-7).

Details:Week 1Topics:Introduction

Growth of Functions

Analyzing Algorithms

Objectives:At the end of this class the student should be able to:

- Describe the course policies and grading criteria.
- Define
algorithmin the strict sense and relate the definition to the meaning of the word in the text.- Define
Big O,Big Theta,Big Omega,little o, andlittle omegaand state and prove the basic relationships among them.- Describe the algorithm hypotheses and body for InsertionSort.
- Prove correctness of InsertionSort.
- Provide runtime analysis for InsertionSort.
Reading:Course Syllabus Textbook: Chapters 1, 2, 3 Suplemental:Notes 1: Introduction to Algorithms

FormulasExercises:1.2-3, 2.1-3, 2.2-2, 2.3-3, 3.1-3, 3.1-4, 3.2-3 Assignment:Assignment 1 Problem 1: Problem 3-1 on p. 57

Assignment 1 Problem 2: Problem 3-2 on p. 58

Details:Week 2Topics:Recurrences

Objectives:At the end of this class the student should be able to:

- Apply the substitution method and tree method to obtain asymptotic solutions of recurrences.
- Solve the Fibonacci recurrence exactly (using the characteristic polynomial) and asymptotically
- State the Master Method for finding asymptotic solutions for recurrences.
- Apply the Master method to find asymptotic bounds on certain recurrences.
Reading:Textbook: Chapters 4, 6, 7 Suplemental:Notes 2: Recurrences

FormulasExercises:4.3-1, 4.3-3, 6.1-1, 6.1-3, 6.1-5, 6.4-3, 6.4-4, Assignment:Assignment 1 Problem 3: Problem 4-5 on p. 86

Assignment 1 Problem 4: Problem 6-1 on p. 142

Assignment 1 Problem 5: Problem 7-4 on p. 162

Details:Week 3Topics:Sorting & Searching

Objectives:At the end of this class the student should be able to:

- Describe the algorithm hypotheses and body for InsertionSort, SelectionSort, MergeSort, HeapSort, and QuickSort
- Give the best, average, and worst case runtimes for InsertionSort, SelectionSort, MergeSort, HeapSort, and QuickSort, and outline the proofs.
- Prove correctness of InsertionSort, SelectionSort, MergeSort,HeapSort, and QuickSort
- Derive and prove the best case asymptotic runtime for a comparison sort, and apply the results to InsertionSort, SelectionSort, MergeSort, HeapSort and QuickSort
- Describe the pros/cons of choosing one of these sorts over another in a given practical setting.
- Describe the algorithm hypotheses and body for SequentialSearch and BinarySearch
- Prove correctness of SequentialSearch and BinarySearch.
- Derive the worst and average case runtime of SequentialSearch and BinarySearch
Reading:Textbook: Textbook: Chapters 6, 7, 8.1 Suplemental:Notes 3: Sorting

Notes 4: Searching

FormulasExercises:7.1-1, 7.1-3, 7.2-3, 7.2-4, 7.4-2

Halting of BinarySearch

Correctness of BinarySearch

Runtime analysis of BinarySearch

Runspace analysis of BinarySearchAssignment:Assignment 1 Problem 6: Prove correctness (including halting) of SelectionSort (use loop invariants)

Assignment 1 Problem 7: Provide worst and average case runtime analysis of SelectionSort

Assignment 1 Problem 8: Provide runspace analysis of SelectionSort

Assignment 2 Problem 1: Prove correctness of SequentialSearch

Assignment 2 Problem 2: Worst, average, and best case runtime analysis of SequentialSearch

Assignment 2 Problem 3: State and prove correct the iterative UpperBound algorithm (see notes)

Assignment 2 Problem 4: Give a complete runtime analysys for iterative UpperBound

Assignment 2 Problem 5: Explain why "short circuit bailout" in any of the binary search algorithms is not cost effective

Details:Week 4Topics:Positional and Associative Data Structures Objectives:At the end of this class the student should be able to:

- Define each of the following ADTs in terms of public interface or operations:

- Vector
- List
- Deque
- Stack
- Queue
- Set (ordered | unordered)
- MultiSet (ordered | unordered)
- Table / Map / Associative Array (ordered | unordered)
- MultiMap (ordered | unordered)
- Classify these ADTs as
positionalorassociative- Give runtime requirements for the operations in the positional ADTs above
- Discuss tradeoffs for various runtime requirements for the operations in the associative ADTs above
Reading:Textbook: Chapters 10, 12 Suplemental:Notes 5: Sequential ADTs and Data Structures

Notes 6: Associative ADTs

Exercises:10.2-1, 10.2-5, 10.3-1, 12.1-1, 12.3-3 Assignment:Assignment 2 Problem 6: Exercise 10.2-7 on p. 209

Assignment 2 Problem 7: Exercise 12.1-3 on p. 256

Assignment 2 Problem 8: Devise an algorithm that uses a Deque for control and such that using Push/Pop at the same end of the deque implements depth-first search and using Push/Pop at opposite ends of the deque implements breadth-first search, otherwise the two algorithms are identical.

Details:Week 5Topics:UUSet, Table, Associative Array Objectives:At the end of this class the student should be able to:

- Define and give examples of
hash function- Show the structure of hash table implemented with chaining
- Define
uniform hashing- State the expected access time for hash tables, and prove these assertions
- Given a specific hash function and parameters defining a hash table, illustrate the internal (private) hash table structure after sequences of insert and remove operations
Reading:Textbook: Chapter 11 Suplemental:Notes 7: Searching in Hash Structures

Exercises:11.2-2, 11.2-3

Assignment:Assignment 3 Problem 1: Exercise 11.2-3 on p. 229

Assignment 3 Problem 2: Exercise 11.2-5 on p. 229

Details:Week 6Topics:Ordered Set, Map

Binary Trees

Binary Tree Iterators

Binary Search Trees as ordered binary trees

Sets as Binary Search Trees

AVL [Height-Balanced] Binary Search TreesObjectives:At the end of this class the student should be able to:

- Discuss representation of trees and binary trees using dynamically created nodes
- Discuss binary tree iterators
- Explain why the increment operator ++ for a binary tree iterator has amortized constant runtime
- Define Binary Search Tree (BST) in terms of order on a Binary Tree
- Show that binary tree iterators encounter BST vertices in sorted order
- Describe the BST Search, Insert, and Delete algorithms
- Estimate the asymptotic runtime of BST Search
- Define height-balanced trees and show that they are special cases of binary trees
- Describe the BST implementation of Set
- Describe runtimes for the various Set operations when implemented as a height-balanced BST.
Reading:Textbook: Appendix B, Chapter 12 Suplemental:Notes 8: Binary Trees

Exercises:

Assignment:Assignment 3 Problem 3: For the following binary tree, complete the table showing each step in a traversal and the number of edges covered by each step (begin, next, ... , next): iteration step current location no of edge moves Q -------------- ---------------- ---------------- / \ 1. initialize W E 2. ++ \ / \ 3. ++ R T Y ... 7. ++ (null)Assignment 3 Problem 4: Describe an order in which vertices of a BST can be saved to ensure that reconstructing a BST from the saved data will result in the same tree. Argue correctness of your assertion.

Assignment 3 Problem 5: Use the result of Assignment 1 Problem 3(e) [problem 4-5(e) in the textbook] to prove:Height-balanced binary trees satisfy height(n) = Θ(log n), where n = number of vertices in the tree.

Details:Week 7Topics:Graphs, Digraphs, Breadth First Search (BFS), Depth First Search (DFS) Objectives:At the end of this class the student should be able to:

- Describe and implement adjacency list representations of graphs and digraphs
- Describe and implement adjacency matrix representations of graphs and digraphs
- State and analyze the breadth first search algorithm
- State and analyze the depth first search algorithm
- State and analyze two distinct versions of the topological sort algorithm
- Define Graph, Directed Graph (Digraph), and Network
- Describe adjacency matrix representations of graphs, digraphs, and networks
- Describe adjacency list representations of graphs, digraphs, and networks
- Discuss how additional vertex and/or edge data can be assoicated with these representations
- Describe the BFS algorithm on a graph or digraph G with Queue control
- Explain why the runtime of BFS is O(V + E) (where V is the number of vertices of G and and E is the number of edges of G)
- Define the BFS Tree of a BFS search of G
- Explain the relationship between the BFS tree and shortest paths in G
Reading:Chapter 22 of [Cormen] Suplemental:Notes 9: Fundamental Graph Algorithms Exercises:Modify the code for the DFS class so that it prints each edge of a digraph along with its type (tree, back, forward, cross). How would this work for an undirected graph?

Exercise 22.1-1 on p. 530

Exercise 22.2-2 on p. 538Assignment:Assignment 4 Problem 1: Implement the BFS class for some category of adjacency list representations of graphs and digraphs. You can invent your own or use [LIB]/igraph. Graphs or digraphs should be instantiatable from a file containing the number of vertices followed by any number of vertex pairs representing edges.

Assignment 4 Problem 2: Implement the DFS class for the same category of adjacency list representations of graphs and digraphs.

Assignment 4 Problem 3: Implement Topological Sort, the version that builds on DFS, for the same class of digraphs.

Details:Week 8Topics:Depth First Search (DFS), Topological Sort

Objectives:At the end of this class the student should be able to:

- Describe the DFS algorithm with Stack control
- Explain why the runtime of DFS is O(V + E)
- Define the DFS Forrest of a DFS search
- Define the DFS [discovered,finished] interval for the vertices of G, and explain the Parenthesis Theorem
- Describe the Topological Sort problem for a Digraph
- Explain how DFS can produce a Topological Sort
Reading:Textbook: Chapters Suplemental:Notes 9 (cont): Graph Algorithms Exercises:Exercise 22.4-1 on p. 551

Exercise 22.4-5 on p. 552Assignment:

Details:Week 9Topics:Minimum Spanning Tree and Shortest Path Algorithms Objectives:At the end of this class the student should be able to:

- Define Prim's algorithm as a class similar to BFS
- Define Kruskal's algorithm
- Derive the asymptotic runtime of Kruskal's algorithm
- Derive the asymptotic runtime of Prim's algorithm
- Explain why the runtime of Prim is asympticically slower than BFS
Reading:Textbook: Chapters Suplemental:Notes 10: Graph Algorithms 2 Exercises:Experiment with traces of Prim and Kruskal on specific weighted graphs. Can you find cases where:

MST is not unique

Prim and Kruskal give different MST

Assignment:Assignment 4 Problem 4: Modify your solution to problem 2 to obtain Prim's algorithm. Test and experiment.

Assignment 4 Problem 5: Use your experience-gathered knowledge to propose a class-based implementation of Kruskal's algorithm. Test as usual, and discuss the results.

Notes for Assignment 4:

- Your program(s) should be working code, so that it is clear how to compile and run them. (A makefile would be helpful for multifile projects.)
- Your program(s) should be able to instantiate a graph from a file that begins with (1) one line of documentation, then (2) specifies the number of vertices and then (3) has any number of edges (with weights if it is a weighted graph), specifying from, to, [and weight]. Example files:
directed graph: cycle of length 5 5 0 1 1 2 2 3 3 4 4 0 undirected graph: cycle of length 5 5 0 1 1 2 2 3 3 4 4 0 weighted graph (undirected): cycle of length 5 5 0 1 1.5 1 2 2.3 2 3 6.0 3 4 2.1 4 0 0.5- Note that the first line of the file serves as both documentation and as the signal for what kind of entity it is by the first character in the line:
Thus you can just read the first non-clearspace character in the file and that tells you what you are building. Then skip the rest of that line and start reading numerical data, which is interpreted depending on the type of entity represented. (We are not doing any network algorithms, but I threw that in to allow for a complete set of possibilites.) You can assume the files are correctly configured!

- 'g' or 'G' or 'u' or 'U' implies Graph (undirected)
- 'd' or 'D' implies directed graph
- 'w' or 'W' implies weighted graph
- 'n' or 'N' implies network (weighted directed graph)

Details:Week 11Topics:Introduction to Genetic Algorithms Objectives:At the end of this class the student should be able to:

- State the generic genetic algorithm (GA)
- Define these terms in the context of genetic algorithms:
population,fitness function,fitness-proportionate selection,crossover,mutation,fitness optimization criteria- Give a detailed trace of an example GA optimization
- Define
schema/schemata- For a schema, define:
order,defining length- State and explain the Schema Theorem
Reading:Suplemental:Notes GA: Introduction to Genetic Algorithms Exercises:

Assignment:

Details:Week 12Topics:McCulloch-Pitts Neuron; Perceptrons & Perception Learning Rule; Supervised Learning; Analog Units; Adaline Objectives:At the end of this class the student should be able to:

- Describe an artificial neuron in terms of input, processing [
combining function] and output [firing function]- Describe a connection between two artificial neurons in terms of functionality [
synaptic functionorweight]- Describe a neural network and give examples of feedforward [
acyclic] and feedback [recurrent] neural topologies- Describe the principal categorization of learning algorithms:
supervisedandunsupervised. Also describereinforcementlearning and place it into the major categorization.- Describe and derive the "delta-rule" learning mechanism for the linear classifyer architecture [single-layer, feed-forward, linear synapses, sigmoidal firing function] [similar to Adaline]
Reading:Suplemental:Notes NN: Introduction to Neural Networks (Chapters 2,3,6,7,8,9; skim the others) Exercises:Read public sources such as Wikipedia to enhance your ability with the stated objectives above

Assignment:Find (legally copyable) source code for Backpropagation learning and install on your workstation. Run learning tests for the 4-2-4 and 8-3-8 encoder/decoder problems.