【Study Notes】Mathematics for Machine Learning: Linear Algebra (Week 3)

Introduction

This is a study note of the course Mathematics for Machine Learning: Linear Algebra on the coursera. In this part, we will learn what matrices is, and operations in matrices. We can make transformations by using mactrices, and find the inverse of a matrix. In addition, the determinants will be explaned in this section.

Matrix

How matrices transform space

Given a vector $r=\begin{bmatrix}x\\y\end{bmatrix}$ , and a $2 \times 2$ matrix $A$ .
Start with $Ar = r'$
We know $r = \begin{bmatrix}x\\y\end{bmatrix}=x\hat {e_1} +y\hat{e_2}$
where $\hat {e_1}$ and $\hat{e_2}$ are basic vectors, that is $\hat {e_1}=\begin{bmatrix}1\\0\end{bmatrix}$ and $\hat {e_2}=\begin{bmatrix}0\\1\end{bmatrix}$.
Then, we get$\begin{aligned}Ar&=A(x\hat {e_1} +y\hat{e_2})\\ &=xA\hat {e_1} +yA\hat{e_2}\end{aligned}$
So we have made a transformation by using matrix $A$. We change the basic vector $\hat {e_1}$ to $A\hat {e_1}$, and $\hat {e_2}$ to $A\hat {e_2}$.

If we assume matrix $A = \begin{bmatrix}a &b\\c&d\end{bmatrix}$.
We get $A\hat {e_1}= \begin{bmatrix}a &b\\c&d\end{bmatrix} \begin{bmatrix}1\\0\end{bmatrix}= \begin{bmatrix}a\\c\end{bmatrix}$, and $A\hat {e_2}= \begin{bmatrix}a &b\\c&d\end{bmatrix} \begin{bmatrix}0\\1\end{bmatrix}= \begin{bmatrix}c\\d\end{bmatrix}$.
Finally, we change the basis to the columns of matrix $A$.
Then, we can say the basis after changing are the columns of matrix $A$.

Types of matrix transformations

Stretching

The matrix associated with a stretch by a factor $k$ along the x-axis is given by:
$\begin{bmatrix}k &0\\0&1\end{bmatrix}$
Similarly, a stretch by a factor $k$ along is given by:
$\begin{bmatrix}1 &0\\0&k\end{bmatrix}$

Squeezing

If the two stretches above are combined with reciprocal values, then the transformation matrix represents a squeeze mapping:
$\begin{bmatrix}k &0\\0&1/ k\end{bmatrix}$
A square with sides parallel to axes is transformed to a rectangle that has the same area as the square. The reciprocal stretch and compression leave the area invariant.

Rotation

For rotation by an angle θ clockwise about the origin the functional form is $x'=x\cos \theta +y\sin \theta$ and $y'=-x\sin \theta +y\cos \theta$. Written in matrix form, this becomes:
$\begin{bmatrix}\cos\theta &\sin\theta\\-\sin\theta&\cos\theta\end{bmatrix}$
Similarly, for a rotation counterclockwise about the origin, the functional form is $x'=x\cos \theta -y\sin \theta$ and $y'=x\sin \theta +y\cos \theta$ the matrix form is:
$\begin{bmatrix}\cos\theta &-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$

Shearing

$\begin{bmatrix}1 &k\\0&1\end{bmatrix}$
or
$\begin{bmatrix}1 &0\\k&1\end{bmatrix}$