PCA

• Given a data matrix $X$ $n\times d$ (zero-centered, the sample mean of column has been shifted no zeros), we want a linear transformation $A_L$, $d\times L$ such that $L<d$ and we have a new data matrix $\hat{X}=X{A}_L$ of dimension $n\times L$.

• We have that $X^{T}X$, $d\times d$ can be seen as the empirical sample covariance matrix of the dataset $X$

SVD

• $X=U\Sigma V^T$ ⇒ $X^{T}X=V{\Sigma }^2{V}^T$

• We can obtain $\hat{X}$ by picking the first $L$ columns of $V$ i.e. the largest $L$ singular vectors, $V_L$.

  • $\hat{X}=X{V}_L$