Homework  #3 Bayesian Decision Theory and Parameter Estimation

Visual Perception Modeling and Its Applications

CIS 4930/5930, Spring 2001

Department of Computer Science, Florida State University

¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾

Due:  Week 11, Monday, March 19, 2001 Points: 100

 

  1. Summarize the Bayesian decision theory. State clearly the assumptions and show that the Bayesian decision theory is optimal in minimizing the average cost.
  2. Answer the following questions based on the maximum likelihood estimation.

a)      State clearly the assumptions of maximum likelihood estimation.

b)      Show that the maximum likelihood estimation for a uni-variate Gauassian distribution with m and s2 as unknown parameters.

c)      Suppose we have 4 samples for each category in a two-category classification problem, estimate the mi and si2 using the maximum likelihood. Here we assume that the true densities are Gaussain.

Sea Bass   

Salmon  

d)      Find the optimal decision boundary given that the parameters found in c) are true values.

  1. Answer the following questions regarding Maximum likelihood estimate and Bayesian estimate.

a)      Compare Maximum likelihood estimate and Bayesian estimate. State clearly the differences in assumptions.

b)      Why in practice do both of them often given similar results?

c)      When do they produce different results?

  1. Answer the following questions regarding non-parametric techniques for density estimation.

a)      What are the advantages of non-parametric techniques over parametric methods?

b)      Explain Parzen window method.

c)      Explain the Kn-nearest neighbor estimation.