1、Tracetrace of a square matrix A is defined as the sum of elements on the main diagonal of A .
Properties :
t r ( A ) = t r ( A T ) tr(A)=tr(A^T) t r ( A ) = t r ( A T ) t r ( A + B ) = t r ( A ) + t r ( B ) tr(A+B)=tr(A)+tr(B) t r ( A + B ) = t r ( A ) + t r ( B ) t r ( c A ) = c t r ( A ) tr(cA)=ctr(A) t r ( c A ) = c t r ( A ) if A and B are two m by n matrics, then: t r ( A T B ) = t r ( A B T ) = t r ( B T A ) = t r ( B A T ) = ∑ i , j A i j B i j tr(A^TB)=tr(AB^T)=tr(B^TA)=tr(BA^T)=\sum_{i,j}A_{ij}B_{ij} t r ( A T B ) = t r ( A B T ) = t r ( B T A ) = t r ( B A T ) = ∑ i , j A i j B i j
A is an m by n matrix and B is an n by m matrix, then: t r ( A B ) = t r ( B A ) tr(AB)=tr(BA) t r ( A B ) = t r ( B A ) Cyclic property: t r ( A B C D ) = t r ( B C D A ) = t r ( C D A B ) = t r ( D A B C ) tr(ABCD)=tr(BCDA)=tr(CDAB)=tr(DABC) t r ( A B C D ) = t r ( B C D A ) = t r ( C D A B ) = t r ( D A B C )
Similarity invariance: t r ( P − 1 A P ) = t r ( A ) tr(P^{-1}AP)=tr(A) t r ( P − 1 A P ) = t r ( A )
Trace of projection matrix is the dimension of the target space: P X = X ( X T X ) − 1 X T ⇒ t r ( P x ) = r a n k ( X ) P_X=X(X^TX)^{-1}X^T \Rightarrow tr(P_x)=rank(X) P X = X ( X T X ) − 1 X T ⇒ t r ( P x ) = r a n k ( X )
2、協方差σ x 2 = 1 n − 1 ∑ i = 1 n ( x i − x ˉ ) 2 \sigma_x^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2 σ x 2 = n − 1 1 ∑ i = 1 n ( x i − x ˉ ) 2 σ ( x , y ) = 1 n − 1 ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) \sigma(x,y)=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y}) σ ( x , y ) = n − 1 1 ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) Σ = [ σ ( x 1 , x 1 ) … σ ( x 1 , x d ) ⋮ ⋱ ⋮ σ ( x d , x 1 ) … σ ( x d , x d ) ] \Sigma=\left[ \begin{matrix} \sigma(x_1,x_1) & \dots & \sigma(x_1,x_d) \\ \vdots & \ddots & \vdots \\ \sigma(x_d,x_1) & \dots &\sigma(x_d, x_d) \end{matrix} \right] Σ = ⎣ ⎢ ⎡ σ ( x 1 , x 1 ) ⋮ σ ( x d , x 1 ) … ⋱ … σ ( x 1 , x d ) ⋮ σ ( x d , x d ) ⎦ ⎥ ⎤
X = [ x 1 x 2 … x n ] d ∗ n , x i ∈ R d X=\left[ \begin{matrix} x_1 & x_2 & \dots & x_n\end{matrix} \right]_{d*n}, x_i \in R^d X = [ x 1 x 2 … x n ] d ∗ n , x i ∈ R d
sample covariance S = X X T = [ x 1 … x n ] [ x 1 T ⋮ x n T ] T S=XX^T=\left[ \begin{matrix}x_1 & \dots & x_n \end{matrix} \right]\left[ \begin{matrix}x_1^T \\ \vdots \\ x_n^T \end{matrix} \right]^T S = X X T = [ x 1 … x n ] ⎣ ⎢ ⎡ x 1 T ⋮ x n T ⎦ ⎥ ⎤ T d ∗ d d*d d ∗ d i i i j j j i i i j j j
3、空間Euclidean space, norm，兩者是什麼關係，還有什麼空間，這是哪門課的概念？
兩點定一線
線段變直線
圓心+半徑 圓
直角都相等
過直線外一點有且只有一條平行線
參考：https://www.youtube.com/watch?v=_l7gyzguikE
4、正太分佈p ( x ) = 1 2 π σ e − ( x − μ ) 2 2 σ 2 p(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}} p ( x ) = 2 π σ 1 e − 2 σ 2 ( x − μ ) 2 p ( x ) = 1 ( 2 π ) 1 / k ∣ Σ ∣ e − ( x − μ ) T Σ − 1 ( x − μ ) p(x)=\frac{1}{\sqrt{(2\pi)^{1/k}|\Sigma|}}e^{-(x-\mu)^T\Sigma^{-1}(x-\mu)} p ( x ) = ( 2 π ) 1 / k ∣ Σ ∣ 1 e − ( x − μ ) T Σ − 1 ( x − μ )
5、距離、範數X X X x , y x,y x , y d ( x , y ) d(x,y) d ( x , y )
d ( x , y ) ≥ 0 d(x,y)\ge 0 d ( x , y ) ≥ 0 d ( x , y ) = 0 d(x,y)=0 d ( x , y ) = 0 x = y x=y x = y d ( x , y ) = d ( y , x ) d(x,y)=d(y,x) d ( x , y ) = d ( y , x ) d ( x , y ) ≤ d ( x , z ) + d ( z , y ) d(x,y)\le d(x,z)+d(z,y) d ( x , y ) ≤ d ( x , z ) + d ( z , y ) d ( x , y ) d(x,y) d ( x , y ) 抽象 最重要的屬性具體∣ ∣ x ∣ ∣ ||x|| ∣ ∣ x ∣ ∣ R n R^n R n ∣ ∣ x ∣ ∣ ≥ 0 , ∀ x ∈ R n ||x||\ge 0, \forall x \in R^n ∣ ∣ x ∣ ∣ ≥ 0 , ∀ x ∈ R n ∣ ∣ x ∣ ∣ = 0 ||x||=0 ∣ ∣ x ∣ ∣ = 0 x = 0 x=0 x = 0 ∣ ∣ α x ∣ ∣ = ∣ α ∣ ∣ ∣ x ∣ ∣ , ∀ α ∈ R , x ∈ R n ||\alpha x||=|\alpha|||x||, \forall \alpha \in R, x \in R^n ∣ ∣ α x ∣ ∣ = ∣ α ∣ ∣ ∣ x ∣ ∣ , ∀ α ∈ R , x ∈ R n ∣ ∣ x + y ∣ ∣ ≤ ∣ ∣ x ∣ ∣ + ∣ ∣ y ∣ ∣ , ∀ x , y ∈ R n ||x+y||\le ||x||+||y||, \forall x, y \in R^n ∣ ∣ x + y ∣ ∣ ≤ ∣ ∣ x ∣ ∣ + ∣ ∣ y ∣ ∣ , ∀ x , y ∈ R n
內積空間( x , y ) ∈ R (x,y)\in R ( x , y ) ∈ R
對稱性
對第一變元的線性性
正定性( x , y ) (x,y) ( x , y )
歐幾里得幾何學需要內積，但連續的概念不需要內積，甚至不需要距離。例如：社交圈的描述；學號的指定是“連續”的。
元素 規則
具體(加東西)---------------------內積（歐式空間、希爾伯特空間）--------------------範數（巴納赫空間）----------------距離--------------拓撲空間---------》抽象（減東西）
參考：http://open.163.com/newview/movie/free?pid=M8PTB0GHI&mid=M8PTBUHT0
6、for any matrix A A A A i A_i A i ∥ A ∥ 2 = ∥ A i T A i ∥ 2 = T r ( A T A ) \left\lVert A\right\rVert^2= \left\lVert A_i^TA_i\right\rVert^2=Tr(A^TA) ∥ A ∥ 2 = ∥ ∥ A i T A i ∥ ∥ 2 = T r ( A T A )
7、max Y T r ( Y T M Y ) s . t . Y T Y = I d ∗ d \max_Y \ Tr(Y^TMY) \\ s.t. \ Y^TY=I_{d*d} Y max T r ( Y T M Y ) s . t . Y T Y = I d ∗ d solution is maximum eigenvectors of M.min Y T r ( Y T M Y ) s . t . Y T Y = I d ∗ d \min_Y \ Tr(Y^TMY) \\ s.t. \ Y^TY=I_{d*d} Y min T r ( Y T M Y ) s . t . Y T Y = I d ∗ d solution is minmum eigenvectors of M.
(6,7不理解，先記住)
