Code In Python

# Download the faces dataset - you should not need to change its contents.

faces = datasets.fetch_olivetti_faces()

print(faces.images.shape)

print(faces.data.shape)

downloading Olivetti faces from https://ndownloader.figshare.com/files/5976027 to /Users/izaanys/scikit_learn_data
(400, 64, 64)
(400, 4096)

The data is stored in
the faces variable. faces.images stores each
face image; from the first print statement, we see there
are 400400 face images and each one
is 64×6464×64 pixels. faces.data stores the
same data but the face images have been unwound into a
length 642=4096642=4096 vector. This latter one will be
easier to work with, but harder to use for visualization.

The next cell plots the first 1515 images in the faces
data set.

In [11]:

# Plot the first 15 faces

fig = plt.figure(figsize=(8, 6))

for i in range(15):

 ax = fig.add_subplot(3, 5, i + 1, xticks=[], yticks=[])

 ax.imshow(faces.images[i], cmap=plt.cm.bone)

? QUESTION 1.9: (5
pts) Store faces.data in the
variable X (this part is done for you below) and compute
the mean of the data set, ??¯?R4096x¯?R4096, and
store ??¯x¯ in the variable x_bar.
Visualize ??¯x¯ by reshaping x_bar into
a 64×6464×64 numpy array/matrix and store this
reshaped array/matrix in the variable avg_face.

In [12]:

# This line will store faces.data in X

X = faces.data

# DO THIS: Put your code for computing x_bar here

# DO THIS: Reshape x_bar into a 64x64 numpy matrix/array and store in avg_face

avg_face = 

# This line will display the average face once you have stored it in the variable avg_face

plt.imshow(avg_face, cmap = 'bone')

? QUESTION 1.10: (5 pts) Mean
center the data set X and store the mean centered data in
the variable Z. Compute the SVD of Z and store the
output in the variables U, s, and Vt.

# Display the first 15 eigenfaces (i.e., the first 15
principal components) – no need to edit this cell.

eigenfaces = Vt[:15,:]

eigenfaces = np.reshape(eigenfaces, (15, 64, 64))

fig = plt.figure(figsize=(8, 6))

for i in range(15):

 ax = fig.add_subplot(3, 5, i + 1, xticks=[], yticks=[])

 ax.imshow(eigenfaces[i,:,:], cmap=plt.cm.bone)

? QUESTION 1.11: (5 pts) Take a
look at the first 1515 eigenfaces plotted in the previous
cell. Describe the different patterns that you see in the
eigenfaces. What characteristics of the faces do they correspond
to?

The larger the singular value, the more important the principal
component. Principal components with small singular values can be
discarded and will result in only a small reconstruction error when
we try to reconstruct the original faces. The call below plots the
singular values rescaled by 1/399??????v1/399.

# Plot the singular values – you should not need to change
this cell.

plt.plot(s/np.sqrt(399))

plt.grid(True)

plt.xlabel('Singular value index')

plt.ylabel('Singular value')

plt.title('Singular values in decreasing order');

From the plot we see that 5050 principal components
should do the trick. Let ??1,…,??400z1,…,z400 be the
centered face image vectors, i.e., ?????R4096zi?R4096 for
each ??=1,…,400i=1,…,400. Let ??1,…,??50v1,…,v50 be the
first 5050 principal components; remember
each ?????R4096vi?R4096 and they are orthonormal.
Therefore ??1,…,??50v1,…,v50 form an orthonormal basis
(ONB) for the subspace ??pca=span(??1,…,????)Vpca=span(v1,…,vn).
To compress the images, we compute the coordinates of each centered
image ????ziin the subspace ??pcaVpca, which are given
by:

Coordinates
of ???? in ??pca=(????·??1,????·??2,…,????·??50)?R50.Coordinates
of zi in Vpca=(zi·v1,zi·v2,…,zi·v50)?R50.

Note, if this works, we will have compressed each image by
nearly 9999% of its original size!

? QUESTION 1.12: (5 pts) Compute
the coordinates of each centered image ????zi in the
subspace ??pca=span(??1,…,??50)Vpca=span(v1,…,v50). Store the
compressed images in the variable Z_compressed, which should
be a numpy matrix or numpy array of
size 400×50400×50.

Compressing images is great, but at some point you want to be
able to go back and look at them. You can’t do that with the
compressed 5050-dimensional vectors. So we need to also be
able to uncompress them. This corresponds to projecting the
centered images onto ??pcaVpca and then adding back in the
average face that we computed back in Question 1.9.

? QUESTION 1.13: (10 pts) Do the
following: (1) For each centered image vector ????zi, compute
its projection onto ??pcaVpca, recalling the formula:

proj??pca????=(????·??1)??1+(????·??2)??2+?+(????·??50)??50.projVpcazi=(zi·v1)v1+(zi·v2)v2+?+(zi·v50)v50.

You should be able to use Z_compressed from Question
1.12 to help with this part. (2) Add back in the mean data
point, ??¯x¯, to each of the projected image vectors, i.e.,
compute and store:

??~??=proj??pca????+??¯.x~i=projVpcazi+x¯.

(3) Reshape and visualize the first 15 uncompressed faces
(use/copy code from above as needed). Compare to the original faces
that were plotted before Question 1.9. Comment on how well your
compression worked.

# (1) DO THIS: Project each centered face vector, z_i, onto
V_pca

# (2) DO THIS: Add back in x_bar to your projected vectors?

# (3) DO THIS: Reshape and visualize the first 15 uncompressed faces (use code from above to help)