隱馬爾可夫模型（Hidden Markov Model，HMM） ，是上世紀七十年代由美國數學家Leonard E. Baum等人提出來的，可以用於解決標註（tagging）問題，屬於一種生成模型（generative model），即直接生成觀測序列和狀態序列的聯合概率分佈。
狀態序列（state sequence）：模型假設由一個隱藏的馬爾可夫鏈生成一個不可觀測的序列，稱這個序列爲狀態序列，其中序列的每個時間步都代表着一個不可被觀測到的狀態
觀測序列（observation sequence）：狀態序列的每個狀態都會對應生成一個結果，這個結果是可以觀測得到的，將這個由狀態序列生成的結果序列稱爲觀測序列
序列
狀態序列：I = ( i 1 , i 2 , . . . , i T ) I = (i_1, i_2, ..., i_T) I = ( i 1 , i 2 , . . . , i T )
觀測序列：O = ( o 1 , o 2 , . . . , o T ) O = (o_1, o_2, ..., o_T) O = ( o 1 , o 2 , . . . , o T )
集合
狀態集合：Q = { q 1 , q 2 , . . . , q N } Q = \{q_1, q_2, ..., q_N\} Q = { q 1 , q 2 , . . . , q N }
觀測結果集合：V = ( v 1 , v 2 , . . . , v M ) V = (v_1, v_2, ..., v_M) V = ( v 1 , v 2 , . . . , v M )
概率矩陣
狀態轉移概率矩陣：A = [ a i j ] N × N A = [a_{ij}]_{N \times N} A = [ a i j ] N × N q i q_i q i q j q_j q j a i j = P r ( i t + 1 = q j ∣ i t = q i ) a_{ij} = Pr(i_{t+1} = q_j | i_{t} = q_i) a i j = P r ( i t + 1 = q j ∣ i t = q i )
觀測概率矩陣：B = [ b j ( k ) ] N × M B =[ b_j(k)]_{N \times M} B = [ b j ( k ) ] N × M q j q_j q j v k v_k v k b j ( k ) = P r ( o t = v k ∣ i t = q j ) b_j(k) = Pr (o_t = v_k |i_t = q_j ) b j ( k ) = P r ( o t = v k ∣ i t = q j )
初始狀態矩陣概率
初始狀態概率向量： $\pi = (\pi_i) $q i q_i q i π i = P r ( i 1 = q i ) \pi_i = Pr(i_1=q_i) π i = P r ( i 1 = q i )
有了以上的形式定義，我們可以使用λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π ) π \pi π A A A I I I I I I B B B O O O
假設有三個盒子，每個盒子裏有紅白兩種顏色的球，盒子裏的紅白球數目如下：
盒子標號
1
2
3
紅球數目
5
3
6
白球數目
5
7
4
抽取球的規則爲：首先從三個盒子裏面以概率選擇一個盒子，然後在這個盒子裏面隨機選擇一個球，記錄球的顏色後放回，並以當前盒子隨機轉移到下一個盒子。
挑選初始盒子的規則爲：0.5的概率挑選盒子1，0.25的概率挑選盒子2，0.25的概率挑選盒子3
轉移盒子的規則爲：如果當前是盒子1，則一定轉移到盒子2，如果當前是盒子2，則0.4的概率轉移到1，以0.6的概率轉移到3，如果當前爲盒子3，則以0.5的概率轉移到1，以0.5的概率轉移到2
重複這個過程五次，最後得到顏色的觀測序列爲：O = { 紅 ， 紅 ， 白 ， 白 ， 紅 } O= \{紅， 紅， 白， 白， 紅\} O = { 紅 ， 紅 ， 白 ， 白 ， 紅 }
狀態的集合爲：Q = { 盒 子 1 ， 盒 子 2 ， 盒 子 3 } , N = 3 Q = \{盒子1， 盒子2， 盒子3\}, N=3 Q = { 盒 子 1 ， 盒 子 2 ， 盒 子 3 } , N = 3
觀測的集合爲：V = { 紅 ， 白 } , M = 2 V = \{紅， 白\}, M=2 V = { 紅 ， 白 } , M = 2
狀態轉移概率矩陣：A = ( 0 1 0 0.4 0 0.6 0.5 0.5 0 )
A = \left( \begin{array}{ccc} 0 & 1 &0 \\ 0.4 & 0 & 0.6 \\ 0.5 & 0.5 & 0 \end{array} \right)
A = ⎝ ⎛ 0 0 . 4 0 . 5 1 0 0 . 5 0 0 . 6 0 ⎠ ⎞
觀測概率矩陣：B = ( 0.5 0.5 0.3 0.7 0.6 0.4 )
B = \left( \begin{array}{ccc} 0.5 & 0.5 \\ 0.3 & 0.7 \\ 0.6 & 0.4 \end{array} \right)
B = ⎝ ⎛ 0 . 5 0 . 3 0 . 6 0 . 5 0 . 7 0 . 4 ⎠ ⎞
π = ( 0.5 , 0.25 , 0.25 )
\pi = (0.5, 0.25, 0.25)
π = ( 0 . 5 , 0 . 2 5 , 0 . 2 5 )
根據以上定義，可以確定隱馬爾可夫模型λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π )
Rabiner(1989)提出隱馬爾可夫模型有下面三個基本問題：
Likelihood：即概率計算問題，給定模型λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π ) O O O P r ( O ∣ λ ) Pr(O|\lambda) P r ( O ∣ λ )
Learning：即學習問題，給定觀測序列O O O A A A B B B π \pi π
Decoding：即預測問題，給定模型λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π ) O O O I I I
概率計算問題定義如下：
給定模型λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π ) O = ( o 1 , o 2 , . . . , o T ) O = (o_1, o_2, ..., o_T) O = ( o 1 , o 2 , . . . , o T ) P r ( O ∣ λ ) Pr(O|\lambda) P r ( O ∣ λ )
一個最簡單的方法是按照概率公式直接計算，方法是列舉所有可能的狀態序列I = ( i 1 , i 2 , . . . , i T ) I = (i_1, i_2, ..., i_T) I = ( i 1 , i 2 , . . . , i T )
首先計算狀態序列的概率：P r ( I ∣ λ ) = π i 1 a i 1 i 2 a i 2 i 3 . . . a i T − 1 i T
Pr(I|\lambda) = \pi_{i_1} a_{i_1 i_2} a_{i_2 i_3} ... a_{i_{T-1} i_T}
P r ( I ∣ λ ) = π i 1 a i 1 i 2 a i 2 i 3 . . . a i T − 1 i T
然後計算給定狀態序列下，觀測序列的概率：
P r ( O ∣ I , λ ) = b i 1 ( o 1 ) b i 2 ( o 2 ) . . . b i T ( o T )
Pr(O|I, \lambda) = b_{i_1}(o_1) b_{i_2}(o_2) ...b_{i_T}(o_T)
P r ( O ∣ I , λ ) = b i 1 ( o 1 ) b i 2 ( o 2 ) . . . b i T ( o T )
所以可以求出狀態序列和觀測序列的聯合概率（所以說HMM是生成模型）：
P r ( O , I ∣ λ ) = P r ( I ∣ λ ) P r ( O ∣ I , λ ) = π i 1 b i 1 ( o 1 ) a i 1 i 2 b i ２ ( o ２ ) . . . a i T − 1 i T 2 b i T ( o T )
Pr(O, I|\lambda) = Pr(I|\lambda) Pr(O|I, \lambda) \\
= \pi_{i_1} b_{i_1}(o_1) a_{i_1 i_2} b_{i_２}(o_２)... a_{i_{T-1} i_T2} b_{i_T}(o_T)
P r ( O , I ∣ λ ) = P r ( I ∣ λ ) P r ( O ∣ I , λ ) = π i 1 b i 1 ( o 1 ) a i 1 i 2 b i ２ ( o ２ ) . . . a i T − 1 i T 2 b i T ( o T )
最後將可能的狀態序列求和就可以得出最終的觀測序列概率：P r ( O ∣ λ ) = ∑ I P r ( I ∣ λ ) P r ( O ∣ I , λ ) = ∑ i 1 , i 2 , . . . , i T π i 1 b i 1 ( o 1 ) a i 1 i 2 b i ２ ( o ２ ) . . . a i T − 1 i T 2 b i T ( o T )
Pr(O|\lambda) = \sum_I Pr(I|\lambda) Pr(O|I, \lambda) \\
= \sum_{i_1, i_2, ..., i_T} \pi_{i_1} b_{i_1}(o_1) a_{i_1 i_2} b_{i_２}(o_２)... a_{i_{T-1} i_T2} b_{i_T}(o_T)
P r ( O ∣ λ ) = I ∑ P r ( I ∣ λ ) P r ( O ∣ I , λ ) = i 1 , i 2 , . . . , i T ∑ π i 1 b i 1 ( o 1 ) a i 1 i 2 b i ２ ( o ２ ) . . . a i T − 1 i T 2 b i T ( o T ) N N N O ( T N T ) O(TN^T) O ( T N T )
這裏引入一種動態規劃（dynamic programming，DP）的算法來解決計算量問題，稱之爲前向後向算法（forward-backward algorithm） 。動態規劃算法使用表來存儲中間值以減少重複計算，以空間換時間，這裏的programming本來指的是表格的意思，DP可直譯爲動態的表格。
定義在時刻t，給定隱馬爾可夫模型λ \lambda λ o 1 , o 2 , . . . , o t o_1, o_2,...,o_t o 1 , o 2 , . . . , o t q i q_{i} q i 前向概率 ，即：α t ( i ) = P r ( o 1 , o 2 , . . . , o t , q i ∣ λ )
\alpha_t(i) = Pr(o_1, o_2,...,o_t,q_i| \lambda)
α t ( i ) = P r ( o 1 , o 2 , . . . , o t , q i ∣ λ ) P ( O ∣ λ ) P(O|\lambda) P ( O ∣ λ )
初始化： α 1 ( j ) = π j b j ( o 1 ) \alpha_1(j) = \pi_j b_j(o_1) α 1 ( j ) = π j b j ( o 1 )
遞推：α t + 1 ( i ) = [ ∑ j = 1 N α t ( j ) ] b i ( o t + 1 ) \alpha_{t+1}(i) = \left[ \sum_{j=1}^{N}\alpha_t(j) \right]b_i(o_{t+1}) α t + 1 ( i ) = [ ∑ j = 1 N α t ( j ) ] b i ( o t + 1 )
終止：P r ( O ∣ λ ) = ∑ i = 1 N α T ( i ) Pr(O|\lambda) = \sum_{i=1}^{N}\alpha_T(i) P r ( O ∣ λ ) = ∑ i = 1 N α T ( i )
定義在時刻t，給定隱馬爾可夫模型λ \lambda λ q i q_{i} q i o t + 1 , o t + 2 , . . . , o T o_{t+1}, o_{t+2},...,o_T o t + 1 , o t + 2 , . . . , o T 後向概率 ，即：β t ( i ) = P r ( o t + 1 , o t + 2 , . . . , o T ∣ i t = q i , λ )
\beta_t(i) = Pr(o_{t+1}, o_{t+2},...,o_T|i_t = q_i, \lambda)
β t ( i ) = P r ( o t + 1 , o t + 2 , . . . , o T ∣ i t = q i , λ ) P ( O ∣ λ ) P(O|\lambda) P ( O ∣ λ )
初始化： β T ( i ) = 1 , i = 1 , 2 , . . . , N \beta_T(i) = 1,\quad i=1,2,...,N β T ( i ) = 1 , i = 1 , 2 , . . . , N
遞推：β t ( i ) = ∑ j = 1 N a i j b j ( o t + 1 ) β t + 1 ( j ) , i = 1 , 2 , . . . , N \beta_{t}(i) = \sum_{j=1}^{N} a_{ij} b_j(o_{t+1}) \beta_{t+1}(j),\quad i=1, 2, ..., N β t ( i ) = ∑ j = 1 N a i j b j ( o t + 1 ) β t + 1 ( j ) , i = 1 , 2 , . . . , N
終止：P r ( O ∣ λ ) = ∑ i = 1 N π i b i ( o 1 ) β 1 ( i ) Pr(O|\lambda) = \sum_{i=1}^{N}\pi_i b_i(o_1)\beta_1(i) P r ( O ∣ λ ) = ∑ i = 1 N π i b i ( o 1 ) β 1 ( i )
模型訓練的問題定義如下：
給定觀測序列O = ( o 1 , o 2 , . . . , o T ) O = (o_1, o_2, ..., o_T) O = ( o 1 , o 2 , . . . , o T ) λ = ( A , B , π ) \lambda = (A, B, \pi) λ = ( A , B , π )
隱馬爾可夫模型的訓練算法稱爲Baum-Welch 算法，是EM 算法的一個特例。EM算法是一種迭代的算法，算法分爲兩步，第一步求期望Expectation，第二步最大化期望Maximization。簡單來講就是初始化概率的估計值，然後使用這些估計值來計算一個更好的估計，不斷的迭代提升。
MSE vs EM （https://machinelearningmastery.com/expectation-maximization-em-algorithm/）
Maximum likelihood estimation is an approach to density estimation for a dataset by searching across probability distributions and their parameters.
It is a general and effective approach that underlies many machine learning algorithms, although it requires that the training dataset is complete, e.g. all relevant interacting random variables are present. Maximum likelihood becomes intractable if there are variables that interact with those in the dataset but were hidden or not observed, so-called latent variables.
The expectation-maximization algorithm is an approach for performing maximum likelihood estimation in the presence of latent variables. It does this by first estimating the values for the latent variables, then optimizing the model, then repeating these two steps until convergence. It is an effective and general approach and is most commonly used for density estimation with missing data, such as clustering algorithms like the Gaussian Mixture Model.
EM算法詳細步驟如下：
完全數據定義爲：( Q , I ) = ( o 1 , o 2 , . . . , o T , i 1 , i 2 , . . . , i T ) (Q, I) = (o_1, o_2, ..., o_T, i_1, i_2, ..., i_T) ( Q , I ) = ( o 1 , o 2 , . . . , o T , i 1 , i 2 , . . . , i T ) log P r ( O , I ∣ λ ) \log Pr(O, I|\lambda) log P r ( O , I ∣ λ )
Q Q Q Q Q Q Q ( λ , λ ^ ) = ∑ I log P r ( O , I ∣ λ ) P r ( O , I ∣ λ ^ ) Q(\lambda, \hat{\lambda}) = \sum_I \log Pr(O,I|\lambda)Pr(O, I|\hat{\lambda}) Q ( λ , λ ^ ) = ∑ I log P r ( O , I ∣ λ ) P r ( O , I ∣ λ ^ )
這裏$ \hat{\lambda}是 模 型 ∗ 當 前 的 估 計 值 ∗ ， 是模型*當前的估計值*， 是 模 型 ∗ 當 前 的 估 計 值 ∗ ， 更好的估計值 。
這裏的Q函數實際上省略了常數因子$ \frac{1}{Pr(O|\hat{\lambda})}， 按 照 定 義 ， ，按照定義， ， 按 照 定 義 ， ， 展 開 得 到 ，展開得到 ， 展 開 得 到
其中P r ( O , I ∣ λ ) = π i 1 b i 1 ( o 1 ) a i 1 i 2 b i 2 o 2 . . . a i T − 1 i T b i T ( o T ) Pr(O, I|\lambda) = \pi_{i_1}b_{i_1}(o_1)a_{i_1 i_2}b_{i_2}{o_2}...a_{i_{T-1}i_T}b_{i_T}(o_T) P r ( O , I ∣ λ ) = π i 1 b i 1 ( o 1 ) a i 1 i 2 b i 2 o 2 . . . a i T − 1 i T b i T ( o T )
所以log P r ( O , I ∣ λ ) = log π i 1 + ∑ t = 1 T − 1 a i t i t + 1 + ∑ t = 1 T b i t ( o t ) \log Pr(O,I|\lambda) = \log \pi_{i_1} + \sum_{t=1}^{T-1} a_{i_{t}i_{t+1}} + \sum_{t=1}^{T} b_{i_t}(o_t) log P r ( O , I ∣ λ ) = log π i 1 + ∑ t = 1 T − 1 a i t i t + 1 + ∑ t = 1 T b i t ( o t )
帶入到Q函數，Q ( λ , λ ^ ) = ∑ I log π i 1 P r ( O , I ∣ λ ^ ) + ∑ I ( ∑ t = 1 T − 1 log a i t i t + 1 ) P r ( O , I ∣ λ ^ ) + ∑ I ( ∑ t = 1 T log b i t ( o t ) ) P r ( O , I ∣ λ ^ ) Q(\lambda, \hat{\lambda}) = \sum_I \log \pi_{i_1}Pr(O, I|\hat{\lambda}) + \sum_I\left( \sum_{t=1}^{T-1} \log a_{i_{t}i_{t+1}} \right)Pr(O, I|\hat{\lambda}) + \sum_I\left( \sum_{t=1}^{T} \log b_{i_t}(o_t) \right)Pr(O, I|\hat{\lambda}) Q ( λ , λ ^ ) = ∑ I log π i 1 P r ( O , I ∣ λ ^ ) + ∑ I ( ∑ t = 1 T − 1 log a i t i t + 1 ) P r ( O , I ∣ λ ^ ) + ∑ I ( ∑ t = 1 T log b i t ( o t ) ) P r ( O , I ∣ λ ^ )
這個步驟就是求能夠極大化Q函數的模型參數，即A A A B B B π \pi π
從上面的Q函數表達式可以看出，要求的模型參數分別位於三個項中，所以可以逐個對各項最大化。這是一個約束優化問題，因爲是概率，所以要滿足歸一性的約束條件，運用拉格朗日乘子法可以解決這個問題。
第一項可以寫爲：∑ I log π i 1 P r ( O , I ∣ λ ^ ) = ∑ i = 1 N log π i P r ( O , i 1 = i ∣ λ ^ )
\sum_I \log \pi_{i_1}Pr(O, I|\hat{\lambda}) = \sum_{i=1}^{N} \log \pi_i Pr(O, i_1=i|\hat{\lambda})
I ∑ log π i 1 P r ( O , I ∣ λ ^ ) = i = 1 ∑ N log π i P r ( O , i 1 = i ∣ λ ^ ) π i \pi_i π i ∑ i = 1 N π i = 1 \sum_{i=1}^N \pi_i = 1 ∑ i = 1 N π i = 1 ∑ i = 1 N log π i P r ( O , i 1 = i ∣ λ ^ ) + γ ( ∑ i = 1 N π i − 1 )
\sum_{i=1}^{N} \log \pi_i Pr(O, i_1=i|\hat{\lambda}) + \gamma \left( \sum_{i=1}^N \pi_i - 1 \right)
i = 1 ∑ N log π i P r ( O , i 1 = i ∣ λ ^ ) + γ ( i = 1 ∑ N π i − 1 ) π i \pi_i π i ∂ ∂ π i [ ∑ i = 1 N log π i P r ( O , i 1 = i ∣ λ ^ ) + γ ( ∑ i = 1 N π i − 1 ) ] = 0
\frac{\partial}{\partial \pi_i}\left[ \sum_{i=1}^{N} \log \pi_i Pr(O, i_1=i|\hat{\lambda}) + \gamma \left( \sum_{i=1}^N \pi_i - 1 \right) \right] = 0
∂ π i ∂ [ i = 1 ∑ N log π i P r ( O , i 1 = i ∣ λ ^ ) + γ ( i = 1 ∑ N π i − 1 ) ] = 0 P r ( O , i 1 = i ∣ λ ^ ) π i + γ = 0 P r ( O , i 1 = i ∣ λ ^ ) + γ π i = 0
\frac{Pr(O,i_1=i|\hat{\lambda})}{\pi_i} + \gamma = 0 \\
Pr(O,i_1=i|\hat{\lambda}) + \gamma \pi_i = 0
π i P r ( O , i 1 = i ∣ λ ^ ) + γ = 0 P r ( O , i 1 = i ∣ λ ^ ) + γ π i = 0 ∑ i = 1 N π i = 1 \sum_{i=1}^N \pi_i = 1 ∑ i = 1 N π i = 1 ∑ i = 1 N P r ( O , i 1 = i ∣ λ ^ ) + ∑ i = 1 N γ π i = 0 P r ( O ∣ λ ^ ) + γ = 0
\sum_{i=1}^{N} Pr(O,i_1=i|\hat{\lambda}) +\sum_{i=1}^{N} \gamma \pi_i = 0 \\
Pr(O|\hat{\lambda}) + \gamma = 0
i = 1 ∑ N P r ( O , i 1 = i ∣ λ ^ ) + i = 1 ∑ N γ π i = 0 P r ( O ∣ λ ^ ) + γ = 0 γ \gamma γ γ = − P r ( O ∣ λ ^ )
\gamma = - Pr(O|\hat{\lambda})
γ = − P r ( O ∣ λ ^ ) P r ( O , i 1 = i ∣ λ ^ ) + γ π i = 0 Pr(O,i_1=i|\hat{\lambda}) + \gamma \pi_i = 0 P r ( O , i 1 = i ∣ λ ^ ) + γ π i = 0 π i \pi_i π i π i = P r ( O , i 1 = i ∣ λ ^ ) P r ( O ∣ λ ^ )
\pi_i = \frac{Pr(O,i_1=i|\hat{\lambda})}{Pr(O|\hat{\lambda})}
π i = P r ( O ∣ λ ^ ) P r ( O , i 1 = i ∣ λ ^ )
第二項可以寫成（這裏前面兩個求和是對於轉移矩陣A A A T T T I I I ∑ I ( ∑ t = 1 T − 1 log a i t i t + 1 ) P r ( O , I ∣ λ ^ ) = ∑ i = 1 N ∑ j = 1 N ∑ t = 1 T − 1 log a i j P r ( O , i t = i , i t + 1 = j ∣ λ ^ )
\sum_I\left( \sum_{t=1}^{T-1} \log a_{i_{t}i_{t+1}} \right)Pr(O, I|\hat{\lambda}) = \sum_{i=1}^{N} \sum_{j=1}^{N} \sum_{t=1}^{T-1} \log a_{ij} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda})
I ∑ ( t = 1 ∑ T − 1 log a i t i t + 1 ) P r ( O , I ∣ λ ^ ) = i = 1 ∑ N j = 1 ∑ N t = 1 ∑ T − 1 log a i j P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) ∑ j = 1 N a i j = 1 \sum_{j=1}^N a_{ij} = 1 ∑ j = 1 N a i j = 1
添加拉格朗日乘子：∑ i = 1 N [ ∑ j = 1 N ∑ t = 1 T − 1 log a i j P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ ( ∑ j = 1 N a i j − 1 ) ]
\sum_{i=1}^{N} \left[ \sum_{j=1}^{N} \sum_{t=1}^{T-1} \log a_{ij} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) + \gamma \left( \sum_{j=1}^N a_{ij} - 1 \right) \right]
i = 1 ∑ N [ j = 1 ∑ N t = 1 ∑ T − 1 log a i j P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ ( j = 1 ∑ N a i j − 1 ) ] a i j a_{ij} a i j ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) a i j + γ = 0 ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ a i j = 0
\frac{\sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda})}{a_{ij}} + \gamma = 0 \\
\sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) + \gamma a_{ij} = 0
a i j ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ = 0 t = 1 ∑ T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ a i j = 0 ∑ j = 1 N ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + ∑ j = 1 N γ a i j = 0 ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ ) + γ = 0
\sum_{j=1}^{N} \sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) + \sum_{j=1}^{N} \gamma a_{ij} = 0 \\
\sum_{t=1}^{T-1} Pr(O, i_t=i|\hat{\lambda}) + \gamma = 0
j = 1 ∑ N t = 1 ∑ T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + j = 1 ∑ N γ a i j = 0 t = 1 ∑ T − 1 P r ( O , i t = i ∣ λ ^ ) + γ = 0 γ \gamma γ γ = − ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ )
\gamma = -\sum_{t=1}^{T-1} Pr(O, i_t=i|\hat{\lambda})
γ = − t = 1 ∑ T − 1 P r ( O , i t = i ∣ λ ^ ) ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ a i j = 0 \sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) + \gamma a_{ij} = 0 ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) + γ a i j = 0 a i j a_{ij} a i j a i j = ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ )
a_{ij} = \frac{\sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) }{\sum_{t=1}^{T-1} Pr(O, i_t=i|\hat{\lambda})}
a i j = ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ ) ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ )
第三項可以寫爲：∑ I ( ∑ t = 1 T log b i t ( o t ) ) P r ( O , I ∣ λ ^ ) = ∑ j = 1 N ∑ t = 1 T log b j ( o t ) P r ( O , i t = j ∣ λ ^ )
\sum_I\left( \sum_{t=1}^{T} \log b_{i_t}(o_t) \right)Pr(O, I|\hat{\lambda}) = \sum_{j=1}^{N} \sum_{t=1}^{T} \log b_j(o_t) Pr(O, i_t = j|\hat{\lambda})
I ∑ ( t = 1 ∑ T log b i t ( o t ) ) P r ( O , I ∣ λ ^ ) = j = 1 ∑ N t = 1 ∑ T log b j ( o t ) P r ( O , i t = j ∣ λ ^ ) ∑ k = 1 M b j ( k ) = 1 \sum_{k=1}^M b_j(k) = 1 ∑ k = 1 M b j ( k ) = 1 ∑ j = 1 N ∑ t = 1 T log b j ( o t ) P r ( O , i t = j ∣ λ ^ ) + γ ( ∑ k = 1 M b j ( k ) − 1 )
\sum_{j=1}^{N} \sum_{t=1}^{T} \log b_j(o_t) Pr(O, i_t = j|\hat{\lambda}) + \gamma \left( \sum_{k=1}^M b_j(k) - 1\right)
j = 1 ∑ N t = 1 ∑ T log b j ( o t ) P r ( O , i t = j ∣ λ ^ ) + γ ( k = 1 ∑ M b j ( k ) − 1 ) b j ( k ) b_j(k) b j ( k ) o t = v k o_t=v_k o t = v k b j ( o t ) b_j(o_t) b j ( o t ) b j ( k ) b_j(k) b j ( k ) I ( o t = v k ) I(o_t =v_k) I ( o t = v k ) ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k ) + γ b j ( k ) = 0
\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})I(o_t =v_k) + \gamma b_j(k) = 0
t = 1 ∑ T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k ) + γ b j ( k ) = 0 γ = − ∑ t = 1 T P r ( O , i t = j ∣ λ ^ )
\gamma = -\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})
γ = − t = 1 ∑ T P r ( O , i t = j ∣ λ ^ ) b j ( k ) = ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k ) ∑ t = 1 T P r ( O , i t = j ∣ λ ^ )
b_j(k) = \frac{\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})I(o_t =v_k)}{\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})}
b j ( k ) = ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k )
經上面的推導，最後得出三個表達式，分別爲：π i = P r ( O , i 1 = i ∣ λ ^ ) P r ( O ∣ λ ^ ) a i j = ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ ) b j ( k ) = ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k ) ∑ t = 1 T P r ( O , i t = j ∣ λ ^ )
\pi_i = \frac{Pr(O,i_1=i|\hat{\lambda})}{Pr(O|\hat{\lambda})} \\
a_{ij} = \frac{\sum_{t=1}^{T-1} Pr(O, i_t=i, i_{t+1}=j|\hat{\lambda}) }{\sum_{t=1}^{T-1} Pr(O, i_t=i|\hat{\lambda})}\\
b_j(k) = \frac{\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})I(o_t =v_k)}{\sum_{t=1}^{T} Pr(O, i_t = j|\hat{\lambda})}
π i = P r ( O ∣ λ ^ ) P r ( O , i 1 = i ∣ λ ^ ) a i j = ∑ t = 1 T − 1 P r ( O , i t = i ∣ λ ^ ) ∑ t = 1 T − 1 P r ( O , i t = i , i t + 1 = j ∣ λ ^ ) b j ( k ) = ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) ∑ t = 1 T P r ( O , i t = j ∣ λ ^ ) I ( o t = v k ) γ t ( i ) \gamma_t(i) γ t ( i ) ξ t ( i , j ) \xi_t(i, j) ξ t ( i , j ) π i = γ 1 ( i ) a i j = ∑ t = 1 T − 1 ξ t ( i , j ) ∑ t = 1 T − 1 γ t ( i ) b j ( k ) = ∑ t = 1 , o t = v k T γ t ( j ) ∑ t = 1 T γ t ( j )
\pi_i = \gamma_1(i) \\
a_{ij} = \frac{\sum_{t=1}^{T-1} \xi_t(i, j) }{\sum_{t=1}^{T-1} \gamma_t(i)}\\
b_j(k) = \frac{\sum_{t=1,o_t=v_k}^{T} \gamma_t(j)}{\sum_{t=1}^{T} \gamma_t(j)}
π i = γ 1 ( i ) a i j = ∑ t = 1 T − 1 γ t ( i ) ∑ t = 1 T − 1 ξ t ( i , j ) b j ( k ) = ∑ t = 1 T γ t ( j ) ∑ t = 1 , o t = v k T γ t ( j )
關於這兩個概率γ t ( i ) \gamma_t(i) γ t ( i ) ξ t ( i , j ) \xi_t(i, j) ξ t ( i , j )
1、給定模型$\lambda 和 觀 測 和觀測 和 觀 測 ， 在 時 刻 t 處 於 狀 態 ，在時刻t處於狀態 ， 在 時 刻 t 處 於 狀 態 KaTeX parse error: No such environment: align at position 8:
\begin{̲a̲l̲i̲g̲n̲}̲
\gamma_t(i) &=… 和 觀 測 和觀測 和 觀 測 ， 在 時 刻 t 處 於 狀 態 ，在時刻t處於狀態 ， 在 時 刻 t 處 於 狀 態 的 且 時 刻 t + 1 處 於 狀 態 的且時刻t+1處於狀態 的 且 時 刻 t + 1 處 於 狀 態 KaTeX parse error: No such environment: align at position 8:
\begin{̲a̲l̲i̲g̲n̲}̲
\xi_t(i, j) &=…
以上就是Baum-Welch算法，實際上就是EM算法在學習HMM上的具體實現。
維特比算法（Viterbi algorithm ）使用動態規劃（DP）解決HMM預測問題（decoding），即使用DP求概率最大路徑，一條路徑對應着一個狀態序列。
引入兩個變量δ \delta δ Ψ \Psi Ψ
定義在時刻t t t i i i ( i 1 , i 2 , . . . , i t ) (i_1, i_2,...,i_t) ( i 1 , i 2 , . . . , i t ) δ t ( i ) = max i 1 , i 2 , . . . , i t − 1 P r ( i t = i , i t − 1 , . . . , i 1 , o t , . . . , o 1 ∣ λ ) , i = 1 , 2 , . . . , N
\delta_t(i) = \max _{i_1, i_2,...,i_{t-1}} Pr(i_t=i, i_{t-1},...,i_1,o_t,...,o_1|\lambda), \quad i=1,2,...,N
δ t ( i ) = i 1 , i 2 , . . . , i t − 1 max P r ( i t = i , i t − 1 , . . . , i 1 , o t , . . . , o 1 ∣ λ ) , i = 1 , 2 , . . . , N KaTeX parse error: No such environment: align at position 8:
\begin{̲a̲l̲i̲g̲n̲}̲
\delta_{t+1}(i… t t t i i i ( i 1 , i 2 , . . . , i t − 1 , i ) (i_1, i_2,...,i_{t-1}, i) ( i 1 , i 2 , . . . , i t − 1 , i ) t − 1 t-1 t − 1 Ψ t ( i ) = arg max 1 ≤ j ≤ N [ δ t − 1 ( j ) a j i ] , i = 1 , 2 , . . . , N
\Psi_t(i) = \arg \max_{1\le j \le N} [\delta_{t-1}(j) a_{ji}], \quad i=1,2,...,N
Ψ t ( i ) = arg 1 ≤ j ≤ N max [ δ t − 1 ( j ) a j i ] , i = 1 , 2 , . . . , N
輸入：模型λ = ( A , B , π ) \lambda = (A, B, \pi ) λ = ( A , B , π ) O = ( o q , o 2 , . . . , o T ) O = (o_q, o_2,...,o_T) O = ( o q , o 2 , . . . , o T )
輸出：最優路徑（狀態序列）I ∗ = ( i 1 ∗ , i 2 ∗ , . . . i T ∗ ) I^*=(i_1^*, i_2^*,... i_T^*) I ∗ = ( i 1 ∗ , i 2 ∗ , . . . i T ∗ )
(1) 初始化。δ 1 ( i ) = π i b i ( o 1 ) , i = 1 , 2 , . . . , N Ψ 1 ( i ) = 0 , i = 1 , 2 , . . . , N
\delta_1(i) = \pi_i b_i(o_1), \quad i=1,2,...,N \\
\Psi_1(i) = 0, \quad i=1,2,...,N
δ 1 ( i ) = π i b i ( o 1 ) , i = 1 , 2 , . . . , N Ψ 1 ( i ) = 0 , i = 1 , 2 , . . . , N
(2) 遞推。對t = 2 , 3 , . . . , T t=2,3,...,T t = 2 , 3 , . . . , T δ t ( i ) = max 1 ≤ j ≤ N [ δ t − 1 ( j ) a j i ] b i ( o t ) Ψ t ( i ) = arg max 1 ≤ j ≤ N [ δ t − 1 ( j ) a j i ] , i = 1 , 2 , . . . , N
\delta_t(i) = \max_{1\le j \le N}[\delta_{t-1}(j) a_{ji}] b_i(o_{t}) \\
\Psi_t(i) = \arg \max_{1\le j \le N} [\delta_{t-1}(j) a_{ji}], \quad i=1,2,...,N
δ t ( i ) = 1 ≤ j ≤ N max [ δ t − 1 ( j ) a j i ] b i ( o t ) Ψ t ( i ) = arg 1 ≤ j ≤ N max [ δ t − 1 ( j ) a j i ] , i = 1 , 2 , . . . , N P ∗ = max 1 ≤ i ≤ N = δ T ( i ) i T ∗ = arg max 1 ≤ i ≤ N [ δ T ( i ) ]
P^* = \max_{1 \le i \le N} = \delta_T(i) \\
i_T^* = \arg \max_{1 \le i \le N} [\delta_T(i)]
P ∗ = 1 ≤ i ≤ N max = δ T ( i ) i T ∗ = arg 1 ≤ i ≤ N max [ δ T ( i ) ] t = T − 1 , T − 2 , . . . , 1 t=T-1, T-2,...,1 t = T − 1 , T − 2 , . . . , 1 i t ∗ = Ψ t + 1 ( i t + 1 ∗ )
i_t* = \Psi_{t+1}(i_{t+1}^*)
i t ∗ = Ψ t + 1 ( i t + 1 ∗ ) I ∗ = ( i 1 ∗ , i 2 ∗ , . . . i T ∗ ) I^*=(i_1^*, i_2^*,... i_T^*) I ∗ = ( i 1 ∗ , i 2 ∗ , . . . i T ∗ )
