冪迭代法求特徵值和特徵向量

冪迭代法求第k大的特徵值和特徵向量

數學表述

設矩陣A $$ A = \left[ \begin{matrix} X_{11}&\ldots&X_{1n} \newline X_{21} & \ldots& X_{2n} \newline &\vdots& \newline X_{n1}&\ldots &X_{nn} \newline \end{matrix} \right] $$ 求其最大特徵值對應的特徵向量$b_k = [v_1,v_2,\ldots,v_n]^T$: $$b_{k+1} = \frac{Ab_k}{||Ab_k ||}..........(1)$$

證明：見英文wiki Power_iteration

<!--more-->

代碼：

import numpy as np
def power_iteration(A, num_simulations: int):
    # Ideally choose a random vector
    # To decrease the chance that our vector
    # Is orthogonal to the eigenvector
    b_k = np.random.rand(A.shape[1])
  
    for _ in range(num_simulations):
        # calculate the matrix-by-vector product Ab
        b_k1 = np.dot(A, b_k)
 
        # calculate the norm
        b_k1_norm = np.linalg.norm(b_k1)
 
        # re normalize the vector
        b_k = b_k1 / b_k1_norm
 
    return b_k

例子

$$ A = \left[ \begin{matrix} 2&1\newline 1&2\newline \end{matrix} \right] $$

求其特徵值： $$ |A-\lambda I| = \left| \begin{matrix} 2-\lambda &1\newline 1 &2-\lambda\newline \end{matrix} \right| = 3 - 4\lambda + \lambda^2 $$ 求得最大特徵值$\lambda_2 = 3$，另一特徵值$\lambda_1 = 1$ $\lambda_2 = 3$ 時： $$ |(A-\lambda I)V| = 0 $$

$$ (A-3I)V = \left[ \begin{array} {cccc} -1 &1\newline 1 &-1\newline \end{array} \right] \left[ \begin{array} {cccc} v_1\newline v_2\newline \end{array} \right] = \left[ \begin{array} {cccc} 0\newline 0\newline \end{array} \right] $$

求得$v_1 = v_2$,歸一化後，$V_{\lambda=3} = [0.707,0.707]^T$

驗證代碼：

A = np.array([[2,1],[1,2]])
V = power_iteration(A, 100).reshape(1,-1)
V

array([[0.70710678, 0.70710678]])

求得特徵向量後，特徵值 $$ \lambda = VAV^T........(2) $$ 代碼

lbd = np.dot(V,np.dot(A,V.T))
lbd 

array([[3.]])

擴展

求次大特徵向量與特徵值: $$ B = A - \frac{\lambda}{||V||^2}V^TV ........(3) $$

$$ b_{k+1}' = \frac{Bb_k'}{||Bb_k' ||} ...........(4) $$

代碼

B = A - lbd / np.linalg.norm(V)**2 * np.dot(V.T,V)
V_2 = power_iteration(B, 100)
lbd1 = np.dot(V_2.T,np.dot(B,V_2))
lbd1

1.0000000000000002

以此類推，我們可以得到任意第k大特徵值。這在很多機器學習方法中，取前k重要特徵有着重要的作用。 歡迎訪問個人博客。轉載請註明出處。