The today's objective is to create a Bayesian classifier which will classify a single real measurement into one of two classes. We will assume the 'zero-one' loss function.
The task is to design a classifier, which would distinguish between two letters K={'A', 'C'}, and that using only a single measurement:
x = (sum of pixel intensities in the left half of the image) - (sum of
pixel intensities in the right half of the image)
This measurement (so-called feature) assigns a real number to each image. Let us assume that the percentage of letters 'A' and 'C' in a given dataset is known, i.e. we know the a priori probabilities p(A) and p(C). Further, let us assume that the conditional probabilities p(x|A) and p(x|C) are also known, that is the probabilities that a value x is measured if the letter is 'A' or 'C' respectively. Let probabilities p(x|A) and p(x|C) be Gaussian distributions with mean value and variance given for each class. Your task is to design a Bayesian classifier (strategy), i.e. a function k=q(x), which will minimize the risk (expected loss).
In this problem, the set of decisions D coincides with the set of hidden states K and the loss function W takes just two values: 0 for correctly recognized and 1 for incorrectly recognized letter. In this special case the classificator q expresses as (lecture notes Bayesian decision theory, slides 22-23)
q(x) = argmax_k
p(k|x) = argmax_k p(k,x)/p(x) = argmax_k p(k,x) = argmax_k p(k) p(x|k),
where p(k|x) is called a posteriori probability of class k given the measurement x. Symbol argmax_k denotes finding k maximizing the argument.
Recall, that density of Gaussian (aka normal) probability of one variable is given by
where μ is mean value and σ is standard deviation (σ2 is variance).
Note: Recommended to go through tasks 1 and 4 before the exercise.
images |
- |
10x10x160, 160 images of size 10x10 pixels, depicting letters 'A' and 'C' |
D1 |
- |
a structure with parameters of the distribution p(x|A) = N(D1.Mean, D1.Sigma^2) |
D1.Mean |
- |
mean value of p(x|A) |
D1.Sigma |
- |
standard deviation of p(x|A) |
D1.Prior |
- |
probability p(A), i.e. the percentage of 'A's in the dataset |
D2 |
- |
a structure with parameters of distribution p(x|C) = N(D2.Mean, D2.Sigma^2) |
D2.Mean |
- |
mean value of p(x|C) |
D2.Sigma |
- |
standard deviation of p(x|C) |
D2.Prior |
- |
probability p(C), i.e. the percentage of 'C's in the dataset |
labels |
- |
1x160, vector with ground-truth classification (to be used ONLY for evaluation of the classification error) |
Alphabet |
- |
1x2, Alphabet = 'AC', the alphabet |
risk |
- |
risk/error of optimal Bayesian strategy |
epsA |
- |
probability p(x|A) integrated over LC, where LC is the set where x is classified as 'C'. |
epsC |
- |
probability p(x|C) integrated over LA, where LA is the set where x is classified as 'A'. |
interA |
- |
one or two intervals defining the set LA |
Hint: To integrate exponential functions you can use normcdf function from Statistics toolbox.
Contact the lab assistants if you
have any questions.
This task is not compulsory. Work on bonus tasks deepens on your knowledge about the subject. Successful solution of a bonus task(s) will be positively reflected during the examination.
The data for this task can be downloaded from data_33rpz_cv02_bonus.mat. The task is analogous to the previous one, only there are now two measurements (features) and three classes. The second measurement is
y = (sum of pixel intensities in the upper half of the image) - (sum of pixel intensities in the lower half of the image)
The alphabet (i.e. the classes) now consists of three letters, 'A', 'C', and
'T'.
The task is to classify the input images into
one of the three classes. Use both measurements x and y
here.
The density function p(x) = p(x|A)+p(x|C)+p(x|T) is visualized below (drawn
using pgmm function from
the toolbox)
Output:
Hint:
Multivariate Gaussian
distribution density is given by
Created by Martin Urban, last update 18.7.2011