Spectral Norm

Definition 1 (Spectral norm)

The spectral norm of a matrix $A\in {\mathbb{R}}^{m\times n}$ is given by:

$|A1{|}_{\ast }:={\max }_{v\in {\mathbb{R}}^n\setminus 0}\frac{||Av|{|}_2}{||v|{|}_2}$

That is, the spectral norm is the largest factor by which a matrix can increase the norm of a vector on which it acts.

Also the spectral norm of a matrix is equal to its largest singular value.

Properties of the spectral norm

Let $A,B\in {\mathbb{R}}^{m\times n}$ be arbitrary matrices and $v\in {\mathbb{R}}^n$ be an arbitrary vector. As with all norms, the spectral norm is subadditive, meaning that it obeys the triangle inequality:

$|A+B{|}_{\ast }\le |A{|}_{\ast }+|B{|}_{\ast }$

The spectral norm is also submultiplicative in the sense that:

$|Av{|}_{\ast }\le |A{|}_{\ast }\cdot |v{|}_2$

and

$|AB{|}_{\ast }\le |A{|}_{\ast }\cdot |B{|}_{\ast }$

If we interpret a vector $v\in {\mathbb{R}}^n$ as a $1\times n$ matrix, then the ${\ell }_2$, spectral and Frobenius norms are equivalent: $|v{|}_2=|v{|}_{\ast }=|v{|}_F$.

Special cases of the spectral norm

For a rank-one matrix $A$, which can be written as an outer-product $A=u{v}^{\top }$, it holds that:

$|A{|}_{\ast }=|A{|}_F=|u{|}_2\cdot |v{|}_2$

A matrix $B\in {\mathbb{R}}^{m\times n}$ is semi-orthogonal if either:

$B^{\top }B=I_n\text{or}B{B}^{\top }=I_m$

A semi-orthogonal matrix has unit spectral norm: $|B{|}_{\ast }=1$.