我們這裏所說的透視，簡單地說就是將三維物理空間中的點經過與光學中心的連線投影到某個面上，這個面可以是平面，也可以是曲面，如圖 1 是一個簡單的示例。平面投影透視的應用最爲廣泛，我們絕大多數的相機鏡頭都是基於這種方式設計的，不過這不代表鏡頭就一定是平面的。其特點是空間中的直線在平面上的投影依然是直線，這樣在某種意義上我們可以認爲是沒有畸變的，當然近大遠小作爲一種物理性質始終不可避免。這種方式實際上就類似於小孔成像，爲了方便我們可以把該投影平面當作是最終的成像平面。但這種方式的缺點也很明顯，由於成像點到成像平面中心的距離與入射角是正切的關係，其有效視角通常比較小。
爲了擴大鏡頭的視角，我們可以採用球面投影透視。而球面是一個曲面，我們很難設計一個球面的感光器件，所以，我們還需要一個從球面到平面的映射過程，這通常是基於折射的原理的來實現的。如果折射角小於入射角，那麼我們就可以把更大的視角壓縮到一張平面圖像上了，就是魚眼鏡頭的基本設計原理。如圖 2 所示，空間中的一點經過球面投影透視投影到了 P P P P 1 P_1 P 1 P 2 P_2 P 2
一般來說，我們會以鏡頭的光學中心作爲原點建立座標參考系，並且令成像平面爲 z = 1 z=1 z = 1 X = ( X , Y , Z ) {\bf{X}}=(X,Y,Z) X = ( X , Y , Z ) x = ( x , y , 1 ) = 1 Z ( X , Y , Z ) = 1 Z X . (1) {\bf{x}} = \left( {x,y,1} \right) = \frac{1}{Z}\left( {X,Y,Z} \right) = \frac{1}{Z}{\bf{X}}. \tag{1} x = ( x , y , 1 ) = Z 1 ( X , Y , Z ) = Z 1 X . ( 1 ) H {\bf{H}} H ( X 2 Y 2 Z 2 ) = ( h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ) ( X 1 Y 1 Z 1 ) . (2) \left( {\begin{array}{c}
{{X_2}}\\
{{Y_2}}\\
{{Z_2}}
\end{array}} \right) = \left( {\begin{array}{c}
{{h_{11}}}&{{h_{12}}}&{{h_{13}}}\\
{{h_{21}}}&{{h_{22}}}&{{h_{23}}}\\
{{h_{31}}}&{{h_{32}}}&{{h_{33}}}
\end{array}} \right)\left( {\begin{array}{c}
{{X_1}}\\
{{Y_1}}\\
{{Z_1}}
\end{array}} \right). \tag{2} ⎝ ⎛ X 2 Y 2 Z 2 ⎠ ⎞ = ⎝ ⎛ h 1 1 h 2 1 h 3 1 h 1 2 h 2 2 h 3 2 h 1 3 h 2 3 h 3 3 ⎠ ⎞ ⎝ ⎛ X 1 Y 1 Z 1 ⎠ ⎞ . ( 2 ) x 2 = 1 Z 2 X 2 = 1 Z 2 H X 1 = Z 1 Z 2 H x 1 . (3) {{\bf{x}}_2} = \frac{1}{{{Z_2}}}{{\bf{X}}_2} = \frac{1}{{{Z_2}}}{\bf{H}}{{\bf{X}}_1} = \frac{{{Z_1}}}{{{Z_2}}}{\bf{H}}{{\bf{x}}_1}. \tag{3} x 2 = Z 2 1 X 2 = Z 2 1 H X 1 = Z 2 Z 1 H x 1 . ( 3 ) H \bf{H} H Z 1 {Z_1} Z 1 Z 1 = 1 {Z_1=1} Z 1 = 1 x 2 = h 11 x 1 + h 12 y 1 + h 13 h 31 x 1 + h 32 y 1 + h 33 , y 2 = h 21 x 1 + h 22 y 1 + h 23 h 31 x 1 + h 32 y 1 + h 33 . (4) {x_2} = \frac{{{h_{11}}{x_1} + {h_{12}}{y_1} + {h_{13}}}}{{{h_{31}}{x_1} + {h_{32}}{y_1} + {h_{33}}}},{\rm{ }}{y_2} = \frac{{{h_{21}}{x_1} + {h_{22}}{y_1} + {h_{23}}}}{{{h_{31}}{x_1} + {h_{32}}{y_1} + {h_{33}}}}.\tag{4} x 2 = h 3 1 x 1 + h 3 2 y 1 + h 3 3 h 1 1 x 1 + h 1 2 y 1 + h 1 3 , y 2 = h 3 1 x 1 + h 3 2 y 1 + h 3 3 h 2 1 x 1 + h 2 2 y 1 + h 2 3 . ( 4 ) Z 2 {Z_2} Z 2 Z 2 {Z_2} Z 2
考慮一種特殊的情況，假如令 h 31 = h 32 = 0 , h 33 = 1 h_{31}=h_{32}=0, h_{33}=1 h 3 1 = h 3 2 = 0 , h 3 3 = 1 A = ( a 11 a 12 a 13 a 21 a 22 a 23 0 0 1 ) . {\bf{A}}= \left( {\begin{array}{c}
{{a_{11}}}&{{a_{12}}}&{{a_{13}}}\\
{{a_{21}}}&{{a_{22}}}&{{a_{23}}}\\
0&0&1
\end{array}} \right). A = ⎝ ⎛ a 1 1 a 2 1 0 a 1 2 a 2 2 0 a 1 3 a 2 3 1 ⎠ ⎞ . Z 2 = 1 Z_2=1 Z 2 = 1 z z z z = 1 z=1 z = 1 z = 1 z=1 z = 1
從透視變換到仿射變換，實際反映了變換參數自由度的縮減。對於透視變換，其總共有 9 個可變化的參數，因此靈活性最高，通過變換可改變物體在平面上的基本形狀。而仿射變換隻剩下了 6 個可變參數，使得原本的非線性變換轉換爲線性變換，因此無法改變物體中線條的平行性，在一定程度上保留了物體的基本形狀。同樣，我們還可以繼續縮減變換參數的自由度，例如只允許物體在平面上平移、旋轉以及寬高等比例縮放，這時物體的形狀除了大小以外沒有任何改變，因此稱爲剛體（Rigid）或歐氏（Euclidean）變換，即( x 2 y 2 1 ) = ( s ⋅ cos θ − s ⋅ sin θ t x s ⋅ sin θ s ⋅ cos θ t y 0 0 1 ) ( x 1 y 1 1 ) . (5) \left( {\begin{array}{c}
{{x_2}}\\
{{y_2}}\\
1
\end{array}} \right) = \left( {\begin{array}{c}
{s \cdot \cos \theta }&{ - s \cdot \sin \theta }&{tx}\\
{s \cdot \sin \theta }&{s \cdot \cos \theta }&{ty}\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{{x_1}}\\
{{y_1}}\\
1
\end{array}} \right). \tag{5} ⎝ ⎛ x 2 y 2 1 ⎠ ⎞ = ⎝ ⎛ s ⋅ cos θ s ⋅ sin θ 0 − s ⋅ sin θ s ⋅ cos θ 0 t x t y 1 ⎠ ⎞ ⎝ ⎛ x 1 y 1 1 ⎠ ⎞ . ( 5 ) ( x 2 y 2 1 ) = ( 1 0 t x 0 1 t y 0 0 1 ) ( s 0 0 0 s 0 0 0 1 ) ( cos θ − sin θ 0 sin θ cos θ 0 0 0 1 ) ( x 1 y 1 1 ) . (6) \left( {\begin{array}{c}
{{x_2}}\\
{{y_2}}\\
1
\end{array}} \right) = \left( {\begin{array}{c}
1&0&{tx}\\
0&1&{ty}\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
s&0&0\\
0&s&0\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{\cos \theta }&{ - \sin \theta }&0\\
{\sin \theta }&{\cos \theta }&0\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{{x_1}}\\
{{y_1}}\\
1
\end{array}} \right).\tag{6} ⎝ ⎛ x 2 y 2 1 ⎠ ⎞ = ⎝ ⎛ 1 0 0 0 1 0 t x t y 1 ⎠ ⎞ ⎝ ⎛ s 0 0 0 s 0 0 0 1 ⎠ ⎞ ⎝ ⎛ cos θ sin θ 0 − sin θ cos θ 0 0 0 1 ⎠ ⎞ ⎝ ⎛ x 1 y 1 1 ⎠ ⎞ . ( 6 )
綜上，雖然我們只使用了一個簡單的 3x3 矩陣，但這已經足以描述我們日常生活中經常遇到的大部分簡單的運動了，其具體形式如圖3所示，包括平移、旋轉、縮放、仿射以及透視變換等等。
假設我們得到物體上的一些點（通常爲特徵點或顯著點）在兩幅平面圖像上對應座標，例如分別爲 ( x i 1 , y i 1 ) (x_{i1}, y_{i1}) ( x i 1 , y i 1 ) ( x i 2 , y i 2 ) (x_{i2}, y_{i2}) ( x i 2 , y i 2 )
由於參數優化過程中座標是已知量，而參數纔是未知數，將座標變換的式子重排可得( x 11 y 11 1 0 0 0 x 21 y 21 1 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 0 0 0 x 11 y 11 1 0 0 0 x 21 y 21 1 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ) ( a 11 a 12 a 13 a 21 a 22 a 23 ) = ( x 12 x 22 ⋮ y 12 y 22 ⋮ ) ⇔ A a = b . (7) \left( {\begin{array}{c}
{{x_{11}}}&{{y_{11}}}&1&0&0&0\\
{{x_{21}}}&{{y_{21}}}&1&0&0&0\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
0&0&0&{{x_{11}}}&{{y_{11}}}&1\\
0&0&0&{{x_{21}}}&{{y_{21}}}&1\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots
\end{array}} \right)\left( {\begin{array}{c}
{{a_{11}}}\\
{{a_{12}}}\\
{{a_{13}}}\\
{{a_{21}}}\\
{{a_{22}}}\\
{{a_{23}}}
\end{array}} \right) = \left( {\begin{array}{c}
{{x_{12}}}\\
{{x_{22}}}\\
\vdots \\
{{y_{12}}}\\
{{y_{22}}}\\
\vdots
\end{array}} \right) \Leftrightarrow {\bf{Aa}} = {\bf{b}}.\tag{7} ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ 0 0 ⋮ 0 0 ⋮ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ a 1 1 a 1 2 a 1 3 a 2 1 a 2 2 a 2 3 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 2 x 2 2 ⋮ y 1 2 y 2 2 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⇔ A a = b . ( 7 ) a = ( A T A ) − 1 A T b , (8) {\bf{a}} = {\left( {{{\bf{A}}^T}{\bf{A}}} \right)^{ - 1}}{{\bf{A}}^T}{\bf{b}},\tag{8} a = ( A T A ) − 1 A T b , ( 8 ) A T A a = A T b , (9) {{\bf{A}}^T}{\bf{Aa}} = {{\bf{A}}^T}{\bf{b}},\tag{9} A T A a = A T b , ( 9 ) A T A = ( ∑ i = 1 n x i 1 x i 1 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n x i 1 0 0 0 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n y i 1 y i 1 ∑ i = 1 n y i 1 0 0 0 ∑ i = 1 n x i 1 ∑ i = 1 n y i 1 ∑ i = 1 n 1 0 0 0 0 0 0 ∑ i = 1 n x i 1 x i 1 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n x i 1 0 0 0 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n y i 1 y i 1 ∑ i = 1 n y i 1 0 0 0 ∑ i = 1 n x i 1 ∑ i = 1 n y i 1 ∑ i = 1 n 1 ) , {{\bf{A}}^T}{\bf{A}} = \left( {\begin{array}{c}
{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}} }&0&0&0\\
{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&0&0&0\\
{\sum\limits_{i = 1}^n {{x_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&{\sum\limits_{i = 1}^n 1 }&0&0&0\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}} }\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&{\sum\limits_{i = 1}^n 1 }
\end{array}} \right), A T A = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ i = 1 ∑ n x i 1 x i 1 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n x i 1 0 0 0 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n y i 1 y i 1 i = 1 ∑ n y i 1 0 0 0 i = 1 ∑ n x i 1 i = 1 ∑ n y i 1 i = 1 ∑ n 1 0 0 0 0 0 0 i = 1 ∑ n x i 1 x i 1 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n x i 1 0 0 0 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n y i 1 y i 1 i = 1 ∑ n y i 1 0 0 0 i = 1 ∑ n x i 1 i = 1 ∑ n y i 1 i = 1 ∑ n 1 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ , A T b = ( ∑ i = 1 n x i 1 x i 2 ∑ i = 1 n y i 1 x i 2 ∑ i = 1 n x i 2 ∑ i = 1 n x i 1 y i 2 ∑ i = 1 n y i 1 y i 2 ∑ i = 1 n y i 2 ) T . {{\bf{A}}^T}{\bf{b}} = {\left( {\begin{array}{c}
{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{x_{i2}}} }&{\sum\limits_{i = 1}^n {{x_{i2}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i2}}} }
\end{array}} \right)^T}. A T b = ( i = 1 ∑ n x i 1 x i 2 i = 1 ∑ n y i 1 x i 2 i = 1 ∑ n x i 2 i = 1 ∑ n x i 1 y i 2 i = 1 ∑ n y i 1 y i 2 i = 1 ∑ n y i 2 ) T .
上面我們可以看到仿射變換的參數優化過程還是比較簡單的，那爲什麼透視變換的參數優化還要單獨去說呢？實際上，由於透視變換引入了一個 z z z ( x 11 y 11 1 0 0 0 − x 11 x 12 − y 11 x 12 − x 12 x 21 y 21 1 0 0 0 − x 21 x 22 − y 21 x 22 − x 22 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 0 0 0 x 11 y 11 1 − x 11 y 12 − y 11 y 12 − y 12 0 0 0 x 21 y 21 1 − x 21 y 22 − y 21 y 22 − y 22 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ) ( h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ) = ( 0 0 ⋮ 0 0 ⋮ ) . (10) \left( {\begin{array}{c}
{{x_{11}}}&{{y_{11}}}&1&0&0&0&{ - {x_{11}}{x_{12}}}&{ - {y_{11}}{x_{12}}}&{ - {x_{12}}}\\
{{x_{21}}}&{{y_{21}}}&1&0&0&0&{ - {x_{21}}{x_{22}}}&{ - {y_{21}}{x_{22}}}&{ - {x_{22}}}\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
0&0&0&{{x_{11}}}&{{y_{11}}}&1&{ - {x_{11}}{y_{12}}}&{ - {y_{11}}{y_{12}}}&{ - {y_{12}}}\\
0&0&0&{{x_{21}}}&{{y_{21}}}&1&{ - {x_{21}}{y_{22}}}&{ - {y_{21}}{y_{22}}}&{ - {y_{22}}}\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots
\end{array}} \right)\left( {\begin{array}{c}
{{h_{11}}}\\
{{h_{12}}}\\
{{h_{13}}}\\
{{h_{21}}}\\
{{h_{22}}}\\
{{h_{23}}}\\
{{h_{31}}}\\
{{h_{32}}}\\
{{h_{33}}}
\end{array}} \right) = \left( {\begin{array}{c}
0\\
0\\
\vdots \\
0\\
0\\
\vdots
\end{array}} \right).\tag{10} ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ 0 0 ⋮ 0 0 ⋮ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ − x 1 1 x 1 2 − x 2 1 x 2 2 ⋮ − x 1 1 y 1 2 − x 2 1 y 2 2 ⋮ − y 1 1 x 1 2 − y 2 1 x 2 2 ⋮ − y 1 1 y 1 2 − y 2 1 y 2 2 ⋮ − x 1 2 − x 2 2 ⋮ − y 1 2 − y 2 2 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ h 1 1 h 1 2 h 1 3 h 2 1 h 2 2 h 2 3 h 3 1 h 3 2 h 3 3 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ 0 0 ⋮ 0 0 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ . ( 1 0 ) A h = 0 {\bf{Ah=0}} A h = 0 h {\bf{h}} h k h {k\bf{h}} k h z z z h {\bf{h}} h A x = b {\bf{Ax=b}} A x = b A {\bf{A}} A n n n r r r R ( A , b ) = R ( A ) = r = n {R({\bf{A, b}})=R({\bf{A}})=r=n} R ( A , b ) = R ( A ) = r = n R ( A , b ) = R ( A ) = r < n {R({\bf{A, b}})=R({\bf{A}})=r<n} R ( A , b ) = R ( A ) = r < n A x = 0 {\bf{Ax=0}} A x = 0 n − r n-r n − r A x = b {\bf{Ax=b}} A x = b A x = 0 {\bf{Ax=0}} A x = 0 A x = b {\bf{Ax=b}} A x = b
在仿射變換矩陣的優化過程中，只要有三個點不共線，那麼式（7）中左側的矩陣都是列滿秩的，那麼從直覺上判斷式（10）中的矩陣 A {\bf{A}} A A {\bf{A}} A 0 {\bf{0}} 0 A {\bf{A}} A r = 8 r=8 r = 8 n − r = 1 n-r=1 n − r = 1 A {\bf{A}} A x 2 x_2 x 2 y 2 y_2 y 2 x 1 x_1 x 1 y 1 y_1 y 1 h ′ 11 x 1 + h ′ 12 y 1 + h ′ 13 − h ′ 31 x 1 x 2 − h ′ 32 y 1 x 2 − h ′ 33 x 2 = 0 , h ′ 21 x 1 + h ′ 22 y 1 + h ′ 23 − h ′ 31 x 1 y 2 − h ′ 32 y 1 y 2 − h ′ 33 y 2 = 0. (11) \begin{array}{l}
{{h'}_{11}}{x_1} + {{h'}_{12}}{y_1} + {{h'}_{13}} - {{h'}_{31}}{x_1}{x_2} - {{h'}_{32}}{y_1}{x_2} - {{h'}_{33}}{x_2} = 0,\\
{{h'}_{21}}{x_1} + {{h'}_{22}}{y_1} + {{h'}_{23}} - {{h'}_{31}}{x_1}{y_2} - {{h'}_{32}}{y_1}{y_2} - {{h'}_{33}}{y_2} = 0.
\end{array}\tag{11} h ′ 1 1 x 1 + h ′ 1 2 y 1 + h ′ 1 3 − h ′ 3 1 x 1 x 2 − h ′ 3 2 y 1 x 2 − h ′ 3 3 x 2 = 0 , h ′ 2 1 x 1 + h ′ 2 2 y 1 + h ′ 2 3 − h ′ 3 1 x 1 y 2 − h ′ 3 2 y 1 y 2 − h ′ 3 3 y 2 = 0 . ( 1 1 ) h ′ {\bf{h'}} h ′ A {\bf{A}} A h ′ {\bf{h'}} h ′ x 2 x_2 x 2 y 2 y_2 y 2 A {\bf{A}} A h 33 {h_{33}} h 3 3 B = ( a 1 a 2 ⋯ a 8 ) , x = ( h 11 h 12 ⋯ h 32 ) T , h 33 = 1 , B x = h 11 a 1 + h 12 a 2 + ⋯ + h 32 a 8 = − a 9 ⇒ B T B x = − B T a 9 . (12) \begin{array}{l}
{\bf{B}} = \left( {\begin{array}{c}
{{{\bf{a}}_1}}&{{{\bf{a}}_2}}& \cdots &{{{\bf{a}}_8}}
\end{array}} \right),{\rm{ }}{\bf{x}} = {\left( {\begin{array}{c}
{{h_{11}}}&{{h_{12}}}& \cdots &{{h_{32}}}
\end{array}} \right)^T},{\rm{ }}{h_{33}} = 1,\\
{\bf{Bx}} = {h_{11}}{{\bf{a}}_1} + {h_{12}}{{\bf{a}}_2} + \cdots + {h_{32}}{{\bf{a}}_8} = - {{\bf{a}}_9} \Rightarrow {{\bf{B}}^T}{\bf{Bx}} = - {{\bf{B}}^T}{{\bf{a}}_9}.
\end{array}\tag{12} B = ( a 1 a 2 ⋯ a 8 ) , x = ( h 1 1 h 1 2 ⋯ h 3 2 ) T , h 3 3 = 1 , B x = h 1 1 a 1 + h 1 2 a 2 + ⋯ + h 3 2 a 8 = − a 9 ⇒ B T B x = − B T a 9 . ( 1 2 ) B {\bf{B}} B R ( B , a 9 ) = R ( B ) = 8 {R({\bf{B, a}}_9)=R({\bf{B}})=8} R ( B , a 9 ) = R ( B ) = 8 h ′ {\bf{h'}} h ′
不過要注意的是，我們不能選取那些本來爲 0 的分量讓其等於 1，雖然一般來說 h 33 h_{33} h 3 3 h 33 {h_{33}} h 3 3 h 33 = 0 {h_{33}=0} h 3 3 = 0 A {\bf{A}} A R ( B ) = 7 R({\bf{B}})=7 R ( B ) = 7 R ( B , a 9 ) = 8 ≠ R ( B ) {R({\bf{B, a}}_9)=8≠R({\bf{B}})} R ( B , a 9 ) = 8 = R ( B ) A {\bf{A}} A ∣ ∣ h ∣ ∣ = 1 {||{\bf{h}}||=1} ∣ ∣ h ∣ ∣ = 1
前面我們雖然討論了很多，實際上都是基於所有匹配點都對應同一個單應性矩陣這一假設的，但在優化問題中，我們通常只能最小化匹配誤差，而不能讓其爲零。也就是說，矩陣 A {\bf{A}} A ∣ ∣ A h ∣ ∣ ||{\bf{Ah}}|| ∣ ∣ A h ∣ ∣ h = 0 \bf{h=0} h = 0 A h ≈ 0 {\bf{Ah}} \approx {\bf{0}} A h ≈ 0 k k k A ⋅ k h ≈ 0 {\bf{A}} \cdot k{\bf{h}} \approx {\bf{0}} A ⋅ k h ≈ 0 h 33 = 1 h_{33}=1 h 3 3 = 1 A {\bf{A}} A h 33 h_{33} h 3 3 k k k
我們知道，函數從不同方向接近極值點時其下降速度並不一定相等。以簡單的二次型（只包含 x i x j x_ix_j x i x j f ( x ) = f ( x , y ) = a x 2 + b y 2 (13) f({\bf{x}}) = f(x,y) = a{x^2} + b{y^2}\tag{13} f ( x ) = f ( x , y ) = a x 2 + b y 2 ( 1 3 ) a > b > 0 a > b > 0 a > b > 0
如果我們使用一系列垂直於 X O Y XOY X O Y y = k x {y=kx} y = k x x = 0 x = 0 x = 0 z = ∣ ∣ x ∣ ∣ ⋅ s i g n ( y ) z=||{\bf{x}}||·sign(y) z = ∣ ∣ x ∣ ∣ ⋅ s i g n ( y ) f ( z ) = c z 2 f(z)=cz^2 f ( z ) = c z 2 b ≤ c ≤ a b \le c \le a b ≤ c ≤ a f ( x ) {f(x)} f ( x ) a a a b b b
回到我們前面所討論的參數優化問題，我們需要最小化 ∣ ∣ A h ∣ ∣ ||{\bf{Ah}}|| ∣ ∣ A h ∣ ∣ f ( h ) = h T A T A h . (14) f({\bf{h}}) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}}.\tag{14} f ( h ) = h T A T A h . ( 1 4 ) f ( k h ) = k 2 f ( h ) {f(k{\bf{h}})=k^2f({\bf{h}})} f ( k h ) = k 2 f ( h ) k {k} k A \bf{A} A A T A {\bf{A}}^T{\bf{A}} A T A f ( k h ) ≥ 0 {f(k{\bf{h}}) \ge 0} f ( k h ) ≥ 0 A h ≈ 0 {\bf{Ah}} \approx {\bf{0}} A h ≈ 0 k {k} k A ⋅ k h ≈ 0 {\bf{A}} \cdot k{\bf{h}} \approx {\bf{0}} A ⋅ k h ≈ 0 k {k} k k k k k k k k k k ∣ ∣ h ∣ ∣ = 1 ||{\bf{h}}||=1 ∣ ∣ h ∣ ∣ = 1 A T A {\bf{A}}^T{\bf{A}} A T A m i n i m i z e f ( h ) = h T A T A h s u b j e c t t o h T h = 1 (15) \begin{array}{l}
{\rm{minimize }}f({\bf{h}}) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}}\\
{\rm{subject to }}{{\bf{h}}^T}{\bf{h}} = 1
\end{array}\tag{15} m i n i m i z e f ( h ) = h T A T A h s u b j e c t t o h T h = 1 ( 1 5 ) λ \lambda λ l ( h , λ ) = h T A T A h + λ ( 1 − h T h ) . (16) l\left( {{\bf{h}},\lambda } \right) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}} + \lambda \left( {1 - {{\bf{h}}^T}{\bf{h}}} \right).\tag{16} l ( h , λ ) = h T A T A h + λ ( 1 − h T h ) . ( 1 6 ) h ∗ \bf{h^*} h ∗ f ( h ) f(\bf{h}) f ( h ) λ ∗ \lambda^* λ ∗ ∇ l ( h ∗ , λ ∗ ) = ( D h l ( h ∗ , λ ∗ ) T D λ l ( h ∗ , λ ∗ ) T ) = ( 2 A T A h ∗ − 2 λ ∗ h ∗ 1 − h ∗ T h ∗ ) = 0. (17) \nabla l\left( {{{\bf{h}}^*},{\lambda ^*}} \right) = \left( {\begin{array}{c}
{{D_{\bf{h}}}l{{\left( {{{\bf{h}}^*},{\lambda ^*}} \right)}^T}}\\
{{D_\lambda }l{{\left( {{{\bf{h}}^*},{\lambda ^*}} \right)}^T}}
\end{array}} \right) = \left( {\begin{array}{c}
{2{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} - 2{\lambda ^*}{{\bf{h}}^*}}\\
{1 - {{\bf{h}}^{*T}}{{\bf{h}}^*}}
\end{array}} \right) = {\bf{0}}.\tag{17} ∇ l ( h ∗ , λ ∗ ) = ( D h l ( h ∗ , λ ∗ ) T D λ l ( h ∗ , λ ∗ ) T ) = ( 2 A T A h ∗ − 2 λ ∗ h ∗ 1 − h ∗ T h ∗ ) = 0 . ( 1 7 ) A T A h ∗ = λ ∗ h ∗ , h ∗ T h ∗ = 1. (18) \begin{array}{c}
{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} = {\lambda ^*}{{\bf{h}}^*},\\
{{\bf{h}}^{*T}}{{\bf{h}}^*} = 1.
\end{array}\tag{18} A T A h ∗ = λ ∗ h ∗ , h ∗ T h ∗ = 1 . ( 1 8 ) f ( h ∗ ) = h ∗ T A T A h ∗ = λ ∗ h ∗ T h ∗ = λ ∗ . (19) f({{\bf{h}}^*}) = {{\bf{h}}^{*T}}{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} = {\lambda ^*}{{\bf{h}}^{*T}}{{\bf{h}}^*} = {\lambda ^*}.\tag{19} f ( h ∗ ) = h ∗ T A T A h ∗ = λ ∗ h ∗ T h ∗ = λ ∗ . ( 1 9 ) h ∗ \bf{h^*} h ∗ f ( h ) f(\bf{h}) f ( h ) A T A {\bf{A}}^T{\bf{A}} A T A f ( h ∗ ) f(\bf{h^*}) f ( h ∗ )
其實，拉格朗日條件只是一個一階必要條件，也就是說滿足式（18）的特徵向量 h ∗ \bf{h^*} h ∗ A T A {\bf{A}}^T{\bf{A}} A T A y = ∑ i b i h i , ∑ i b i 2 = 1. (20) {\bf{y}} = \sum\limits_i {{b_i}{{\bf{h}}_i}} ,{\rm{ }}\sum\limits_i {b_i^2} = 1.\tag{20} y = i ∑ b i h i , i ∑ b i 2 = 1 . ( 2 0 ) y T A T A y = ∑ i b i h i T ⋅ ∑ i λ i b i h i = ∑ i λ i b i 2 . (21) {{\bf{y}}^T}{{\bf{A}}^T}{\bf{Ay}} = \sum\limits_i {{b_i}{\bf{h}}_i^T} \cdot \sum\limits_i {{\lambda _i}{b_i}{{\bf{h}}_i}} = \sum\limits_i {{\lambda _i}b_i^2} .\tag{21} y T A T A y = i ∑ b i h i T ⋅ i ∑ λ i b i h i = i ∑ λ i b i 2 . ( 2 1 ) h 1 T A T A h 1 = λ 1 ≤ y T A T A y ≤ λ 9 = h 9 T A T A h 9 . (22) {\bf{h}}_1^T{{\bf{A}}^T}{\bf{A}}{{\bf{h}}_1} = {\lambda _1} \le {{\bf{y}}^T}{{\bf{A}}^T}{\bf{Ay}} \le {\lambda _9} = {\bf{h}}_9^T{{\bf{A}}^T}{\bf{A}}{{\bf{h}}_9}.\tag{22} h 1 T A T A h 1 = λ 1 ≤ y T A T A y ≤ λ 9 = h 9 T A T A h 9 . ( 2 2 ) A \bf{A} A
綜上所述，透視變換的參數優化問題可以轉化爲求解式（18）的特徵向量的問題，而且最優的參數向量爲最小特徵值對應的特徵向量。由於特徵向量在某些分量上的值可能很小或者爲零，那麼在最初提到的直接讓 h 33 h_{33} h 3 3 h 33 ≈ 0 h_{33} \approx 0 h 3 3 ≈ 0 ∣ ∣ h ∣ ∣ ||\bf{h}|| ∣ ∣ h ∣ ∣ h 33 h_{33} h 3 3
例如我們有以下一張圖片，由於手機攝像頭的焦距比較短，所以有明顯的透視效應。我們想要更換屏幕上的壁紙，相當於將一張四平八方的圖片通過透視變換投影到屏幕包圍的區域。前面已經提到，要求解相應的透視變換參數，最少只需要 4 個匹配點對即可。所以，我們只需找到屏幕四個角對應的座標，然後加上我們想要更換的圖片的四個角的座標，構造矩陣 A T A {\bf{A}}^T{\bf{A}} A T A
import numpy as np
from PIL import Image
def solve_homo ( ptsrc, ptdst) :
npts = ptsrc. shape[ 0 ]
A = np. zeros( [ 2 * npts, 9 ] )
A[ : npts, [ 0 , 1 ] ] = np. copy( ptsrc)
A[ : npts, 2 ] = np. ones( npts)
A[ npts: , [ 3 , 4 ] ] = np. copy( ptsrc)
A[ npts: , 5 ] = np. ones( npts)
A[ : npts, 6 ] = - ptsrc[ : , 0 ] * ptdst[ : , 0 ]
A[ npts: , 6 ] = - ptsrc[ : , 0 ] * ptdst[ : , 1 ]
A[ : npts, 7 ] = - ptsrc[ : , 1 ] * ptdst[ : , 0 ]
A[ npts: , 7 ] = - ptsrc[ : , 1 ] * ptdst[ : , 1 ]
A[ : npts, 8 ] = - ptdst[ : , 0 ]
A[ npts: , 8 ] = - ptdst[ : , 1 ]
A = A. T. dot( A)
vals, vecs = np. linalg. eig( A)
homo_mat = vecs[ : , np. argmin( vals) ]
if np. max ( np. abs ( homo_mat) ) < 1e - 7 :
homo_mat[ - 1 ] = 1
if np. abs ( homo_mat[ - 1 ] ) > 1e - 7 :
homo_mat /= homo_mat[ - 1 ]
return homo_mat
if __name__ == '__main__' :
src = Image. open ( 'src.jpg' ) . convert( 'RGBA' )
dst = Image. open ( 'dst.jpg' ) . convert( 'RGBA' )
w, h = src. size
W, H = dst. size
ptsrc = np. array( [ [ 0 , 0 ] , [ w- 1 , 0 ] , [ 0 , h- 1 ] , [ w- 1 , h- 1 ] ] )
ptdst = np. array( [ [ 610 , 210 ] , [ 3322 , 388 ] , [ 378 , 1594 ] , [ 2988 , 2268 ] ] )
homo_mat = solve_homo( ptdst, ptsrc)
img = src. transform( ( W, H) , Image. PERSPECTIVE, homo_mat[ : - 1 ] , Image. BICUBIC)
dst. paste( img, img. split( ) [ - 1 ] )
dst = dst. convert( 'RGB' )
dst. save( 'new_wallpaper.jpg' )
圖1 透視示例
圖2 魚眼成像示例
圖3 常見的變換與運動模型
圖4 橢圓等高線
圖5 剖面二次曲線
