我们这里所说的透视，简单地说就是将三维物理空间中的点经过与光学中心的连线投影到某个面上，这个面可以是平面，也可以是曲面，如图 1 是一个简单的示例。平面投影透视的应用最为广泛，我们绝大多数的相机镜头都是基于这种方式设计的，不过这不代表镜头就一定是平面的。其特点是空间中的直线在平面上的投影依然是直线，这样在某种意义上我们可以认为是没有畸变的，当然近大远小作为一种物理性质始终不可避免。这种方式实际上就类似于小孔成像，为了方便我们可以把该投影平面当作是最终的成像平面。但这种方式的缺点也很明显，由于成像点到成像平面中心的距离与入射角是正切的关系，其有效视角通常比较小。
为了扩大镜头的视角，我们可以采用球面投影透视。而球面是一个曲面，我们很难设计一个球面的感光器件，所以，我们还需要一个从球面到平面的映射过程，这通常是基于折射的原理的来实现的。如果折射角小于入射角，那么我们就可以把更大的视角压缩到一张平面图像上了，就是鱼眼镜头的基本设计原理。如图 2 所示，空间中的一点经过球面投影透视投影到了 P P P P 1 P_1 P 1 P 2 P_2 P 2
一般来说，我们会以镜头的光学中心作为原点建立座标参考系，并且令成像平面为 z = 1 z=1 z = 1 X = ( X , Y , Z ) {\bf{X}}=(X,Y,Z) X = ( X , Y , Z ) x = ( x , y , 1 ) = 1 Z ( X , Y , Z ) = 1 Z X . (1) {\bf{x}} = \left( {x,y,1} \right) = \frac{1}{Z}\left( {X,Y,Z} \right) = \frac{1}{Z}{\bf{X}}. \tag{1} x = ( x , y , 1 ) = Z 1 ( X , Y , Z ) = Z 1 X . ( 1 ) H {\bf{H}} H ( X 2 Y 2 Z 2 ) = ( h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ) ( X 1 Y 1 Z 1 ) . (2) \left( {\begin{array}{c}
{{X_2}}\\
{{Y_2}}\\
{{Z_2}}
\end{array}} \right) = \left( {\begin{array}{c}
{{h_{11}}}&{{h_{12}}}&{{h_{13}}}\\
{{h_{21}}}&{{h_{22}}}&{{h_{23}}}\\
{{h_{31}}}&{{h_{32}}}&{{h_{33}}}
\end{array}} \right)\left( {\begin{array}{c}
{{X_1}}\\
{{Y_1}}\\
{{Z_1}}
\end{array}} \right). \tag{2} ⎝ ⎛ X 2 Y 2 Z 2 ⎠ ⎞ = ⎝ ⎛ h 1 1 h 2 1 h 3 1 h 1 2 h 2 2 h 3 2 h 1 3 h 2 3 h 3 3 ⎠ ⎞ ⎝ ⎛ X 1 Y 1 Z 1 ⎠ ⎞ . ( 2 ) x 2 = 1 Z 2 X 2 = 1 Z 2 H X 1 = Z 1 Z 2 H x 1 . (3) {{\bf{x}}_2} = \frac{1}{{{Z_2}}}{{\bf{X}}_2} = \frac{1}{{{Z_2}}}{\bf{H}}{{\bf{X}}_1} = \frac{{{Z_1}}}{{{Z_2}}}{\bf{H}}{{\bf{x}}_1}. \tag{3} x 2 = Z 2 1 X 2 = Z 2 1 H X 1 = Z 2 Z 1 H x 1 . ( 3 ) H \bf{H} H Z 1 {Z_1} Z 1 Z 1 = 1 {Z_1=1} Z 1 = 1 x 2 = h 11 x 1 + h 12 y 1 + h 13 h 31 x 1 + h 32 y 1 + h 33 , y 2 = h 21 x 1 + h 22 y 1 + h 23 h 31 x 1 + h 32 y 1 + h 33 . (4) {x_2} = \frac{{{h_{11}}{x_1} + {h_{12}}{y_1} + {h_{13}}}}{{{h_{31}}{x_1} + {h_{32}}{y_1} + {h_{33}}}},{\rm{ }}{y_2} = \frac{{{h_{21}}{x_1} + {h_{22}}{y_1} + {h_{23}}}}{{{h_{31}}{x_1} + {h_{32}}{y_1} + {h_{33}}}}.\tag{4} x 2 = h 3 1 x 1 + h 3 2 y 1 + h 3 3 h 1 1 x 1 + h 1 2 y 1 + h 1 3 , y 2 = h 3 1 x 1 + h 3 2 y 1 + h 3 3 h 2 1 x 1 + h 2 2 y 1 + h 2 3 . ( 4 ) Z 2 {Z_2} Z 2 Z 2 {Z_2} Z 2
考虑一种特殊的情况，假如令 h 31 = h 32 = 0 , h 33 = 1 h_{31}=h_{32}=0, h_{33}=1 h 3 1 = h 3 2 = 0 , h 3 3 = 1 A = ( a 11 a 12 a 13 a 21 a 22 a 23 0 0 1 ) . {\bf{A}}= \left( {\begin{array}{c}
{{a_{11}}}&{{a_{12}}}&{{a_{13}}}\\
{{a_{21}}}&{{a_{22}}}&{{a_{23}}}\\
0&0&1
\end{array}} \right). A = ⎝ ⎛ a 1 1 a 2 1 0 a 1 2 a 2 2 0 a 1 3 a 2 3 1 ⎠ ⎞ . Z 2 = 1 Z_2=1 Z 2 = 1 z z z z = 1 z=1 z = 1 z = 1 z=1 z = 1
从透视变换到仿射变换，实际反映了变换参数自由度的缩减。对于透视变换，其总共有 9 个可变化的参数，因此灵活性最高，通过变换可改变物体在平面上的基本形状。而仿射变换只剩下了 6 个可变参数，使得原本的非线性变换转换为线性变换，因此无法改变物体中线条的平行性，在一定程度上保留了物体的基本形状。同样，我们还可以继续缩减变换参数的自由度，例如只允许物体在平面上平移、旋转以及宽高等比例缩放，这时物体的形状除了大小以外没有任何改变，因此称为刚体（Rigid）或欧氏（Euclidean）变换，即( x 2 y 2 1 ) = ( s ⋅ cos θ − s ⋅ sin θ t x s ⋅ sin θ s ⋅ cos θ t y 0 0 1 ) ( x 1 y 1 1 ) . (5) \left( {\begin{array}{c}
{{x_2}}\\
{{y_2}}\\
1
\end{array}} \right) = \left( {\begin{array}{c}
{s \cdot \cos \theta }&{ - s \cdot \sin \theta }&{tx}\\
{s \cdot \sin \theta }&{s \cdot \cos \theta }&{ty}\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{{x_1}}\\
{{y_1}}\\
1
\end{array}} \right). \tag{5} ⎝ ⎛ x 2 y 2 1 ⎠ ⎞ = ⎝ ⎛ s ⋅ cos θ s ⋅ sin θ 0 − s ⋅ sin θ s ⋅ cos θ 0 t x t y 1 ⎠ ⎞ ⎝ ⎛ x 1 y 1 1 ⎠ ⎞ . ( 5 ) ( x 2 y 2 1 ) = ( 1 0 t x 0 1 t y 0 0 1 ) ( s 0 0 0 s 0 0 0 1 ) ( cos θ − sin θ 0 sin θ cos θ 0 0 0 1 ) ( x 1 y 1 1 ) . (6) \left( {\begin{array}{c}
{{x_2}}\\
{{y_2}}\\
1
\end{array}} \right) = \left( {\begin{array}{c}
1&0&{tx}\\
0&1&{ty}\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
s&0&0\\
0&s&0\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{\cos \theta }&{ - \sin \theta }&0\\
{\sin \theta }&{\cos \theta }&0\\
0&0&1
\end{array}} \right)\left( {\begin{array}{c}
{{x_1}}\\
{{y_1}}\\
1
\end{array}} \right).\tag{6} ⎝ ⎛ x 2 y 2 1 ⎠ ⎞ = ⎝ ⎛ 1 0 0 0 1 0 t x t y 1 ⎠ ⎞ ⎝ ⎛ s 0 0 0 s 0 0 0 1 ⎠ ⎞ ⎝ ⎛ cos θ sin θ 0 − sin θ cos θ 0 0 0 1 ⎠ ⎞ ⎝ ⎛ x 1 y 1 1 ⎠ ⎞ . ( 6 )
综上，虽然我们只使用了一个简单的 3x3 矩阵，但这已经足以描述我们日常生活中经常遇到的大部分简单的运动了，其具体形式如图3所示，包括平移、旋转、缩放、仿射以及透视变换等等。
假设我们得到物体上的一些点（通常为特征点或显著点）在两幅平面图像上对应座标，例如分别为 ( x i 1 , y i 1 ) (x_{i1}, y_{i1}) ( x i 1 , y i 1 ) ( x i 2 , y i 2 ) (x_{i2}, y_{i2}) ( x i 2 , y i 2 )
由于参数优化过程中座标是已知量，而参数才是未知数，将座标变换的式子重排可得( x 11 y 11 1 0 0 0 x 21 y 21 1 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 0 0 0 x 11 y 11 1 0 0 0 x 21 y 21 1 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ) ( a 11 a 12 a 13 a 21 a 22 a 23 ) = ( x 12 x 22 ⋮ y 12 y 22 ⋮ ) ⇔ A a = b . (7) \left( {\begin{array}{c}
{{x_{11}}}&{{y_{11}}}&1&0&0&0\\
{{x_{21}}}&{{y_{21}}}&1&0&0&0\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
0&0&0&{{x_{11}}}&{{y_{11}}}&1\\
0&0&0&{{x_{21}}}&{{y_{21}}}&1\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots
\end{array}} \right)\left( {\begin{array}{c}
{{a_{11}}}\\
{{a_{12}}}\\
{{a_{13}}}\\
{{a_{21}}}\\
{{a_{22}}}\\
{{a_{23}}}
\end{array}} \right) = \left( {\begin{array}{c}
{{x_{12}}}\\
{{x_{22}}}\\
\vdots \\
{{y_{12}}}\\
{{y_{22}}}\\
\vdots
\end{array}} \right) \Leftrightarrow {\bf{Aa}} = {\bf{b}}.\tag{7} ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ 0 0 ⋮ 0 0 ⋮ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ a 1 1 a 1 2 a 1 3 a 2 1 a 2 2 a 2 3 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 2 x 2 2 ⋮ y 1 2 y 2 2 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⇔ A a = b . ( 7 ) a = ( A T A ) − 1 A T b , (8) {\bf{a}} = {\left( {{{\bf{A}}^T}{\bf{A}}} \right)^{ - 1}}{{\bf{A}}^T}{\bf{b}},\tag{8} a = ( A T A ) − 1 A T b , ( 8 ) A T A a = A T b , (9) {{\bf{A}}^T}{\bf{Aa}} = {{\bf{A}}^T}{\bf{b}},\tag{9} A T A a = A T b , ( 9 ) A T A = ( ∑ i = 1 n x i 1 x i 1 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n x i 1 0 0 0 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n y i 1 y i 1 ∑ i = 1 n y i 1 0 0 0 ∑ i = 1 n x i 1 ∑ i = 1 n y i 1 ∑ i = 1 n 1 0 0 0 0 0 0 ∑ i = 1 n x i 1 x i 1 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n x i 1 0 0 0 ∑ i = 1 n x i 1 y i 1 ∑ i = 1 n y i 1 y i 1 ∑ i = 1 n y i 1 0 0 0 ∑ i = 1 n x i 1 ∑ i = 1 n y i 1 ∑ i = 1 n 1 ) , {{\bf{A}}^T}{\bf{A}} = \left( {\begin{array}{c}
{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}} }&0&0&0\\
{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&0&0&0\\
{\sum\limits_{i = 1}^n {{x_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&{\sum\limits_{i = 1}^n 1 }&0&0&0\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}} }\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }\\
0&0&0&{\sum\limits_{i = 1}^n {{x_{i1}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}} }&{\sum\limits_{i = 1}^n 1 }
\end{array}} \right), A T A = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ i = 1 ∑ n x i 1 x i 1 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n x i 1 0 0 0 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n y i 1 y i 1 i = 1 ∑ n y i 1 0 0 0 i = 1 ∑ n x i 1 i = 1 ∑ n y i 1 i = 1 ∑ n 1 0 0 0 0 0 0 i = 1 ∑ n x i 1 x i 1 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n x i 1 0 0 0 i = 1 ∑ n x i 1 y i 1 i = 1 ∑ n y i 1 y i 1 i = 1 ∑ n y i 1 0 0 0 i = 1 ∑ n x i 1 i = 1 ∑ n y i 1 i = 1 ∑ n 1 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ , A T b = ( ∑ i = 1 n x i 1 x i 2 ∑ i = 1 n y i 1 x i 2 ∑ i = 1 n x i 2 ∑ i = 1 n x i 1 y i 2 ∑ i = 1 n y i 1 y i 2 ∑ i = 1 n y i 2 ) T . {{\bf{A}}^T}{\bf{b}} = {\left( {\begin{array}{c}
{\sum\limits_{i = 1}^n {{x_{i1}}{x_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{x_{i2}}} }&{\sum\limits_{i = 1}^n {{x_{i2}}} }&{\sum\limits_{i = 1}^n {{x_{i1}}{y_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i1}}{y_{i2}}} }&{\sum\limits_{i = 1}^n {{y_{i2}}} }
\end{array}} \right)^T}. A T b = ( i = 1 ∑ n x i 1 x i 2 i = 1 ∑ n y i 1 x i 2 i = 1 ∑ n x i 2 i = 1 ∑ n x i 1 y i 2 i = 1 ∑ n y i 1 y i 2 i = 1 ∑ n y i 2 ) T .
上面我们可以看到仿射变换的参数优化过程还是比较简单的，那为什么透视变换的参数优化还要单独去说呢？实际上，由于透视变换引入了一个 z z z ( x 11 y 11 1 0 0 0 − x 11 x 12 − y 11 x 12 − x 12 x 21 y 21 1 0 0 0 − x 21 x 22 − y 21 x 22 − x 22 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 0 0 0 x 11 y 11 1 − x 11 y 12 − y 11 y 12 − y 12 0 0 0 x 21 y 21 1 − x 21 y 22 − y 21 y 22 − y 22 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ) ( h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ) = ( 0 0 ⋮ 0 0 ⋮ ) . (10) \left( {\begin{array}{c}
{{x_{11}}}&{{y_{11}}}&1&0&0&0&{ - {x_{11}}{x_{12}}}&{ - {y_{11}}{x_{12}}}&{ - {x_{12}}}\\
{{x_{21}}}&{{y_{21}}}&1&0&0&0&{ - {x_{21}}{x_{22}}}&{ - {y_{21}}{x_{22}}}&{ - {x_{22}}}\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
0&0&0&{{x_{11}}}&{{y_{11}}}&1&{ - {x_{11}}{y_{12}}}&{ - {y_{11}}{y_{12}}}&{ - {y_{12}}}\\
0&0&0&{{x_{21}}}&{{y_{21}}}&1&{ - {x_{21}}{y_{22}}}&{ - {y_{21}}{y_{22}}}&{ - {y_{22}}}\\
\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots
\end{array}} \right)\left( {\begin{array}{c}
{{h_{11}}}\\
{{h_{12}}}\\
{{h_{13}}}\\
{{h_{21}}}\\
{{h_{22}}}\\
{{h_{23}}}\\
{{h_{31}}}\\
{{h_{32}}}\\
{{h_{33}}}
\end{array}} \right) = \left( {\begin{array}{c}
0\\
0\\
\vdots \\
0\\
0\\
\vdots
\end{array}} \right).\tag{10} ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ 0 0 ⋮ 0 0 ⋮ x 1 1 x 2 1 ⋮ 0 0 ⋮ y 1 1 y 2 1 ⋮ 0 0 ⋮ 1 1 ⋮ − x 1 1 x 1 2 − x 2 1 x 2 2 ⋮ − x 1 1 y 1 2 − x 2 1 y 2 2 ⋮ − y 1 1 x 1 2 − y 2 1 x 2 2 ⋮ − y 1 1 y 1 2 − y 2 1 y 2 2 ⋮ − x 1 2 − x 2 2 ⋮ − y 1 2 − y 2 2 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ h 1 1 h 1 2 h 1 3 h 2 1 h 2 2 h 2 3 h 3 1 h 3 2 h 3 3 ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ = ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎛ 0 0 ⋮ 0 0 ⋮ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎞ . ( 1 0 ) A h = 0 {\bf{Ah=0}} A h = 0 h {\bf{h}} h k h {k\bf{h}} k h z z z h {\bf{h}} h A x = b {\bf{Ax=b}} A x = b A {\bf{A}} A n n n r r r R ( A , b ) = R ( A ) = r = n {R({\bf{A, b}})=R({\bf{A}})=r=n} R ( A , b ) = R ( A ) = r = n R ( A , b ) = R ( A ) = r < n {R({\bf{A, b}})=R({\bf{A}})=r<n} R ( A , b ) = R ( A ) = r < n A x = 0 {\bf{Ax=0}} A x = 0 n − r n-r n − r A x = b {\bf{Ax=b}} A x = b A x = 0 {\bf{Ax=0}} A x = 0 A x = b {\bf{Ax=b}} A x = b
在仿射变换矩阵的优化过程中，只要有三个点不共线，那么式（7）中左侧的矩阵都是列满秩的，那么从直觉上判断式（10）中的矩阵 A {\bf{A}} A A {\bf{A}} A 0 {\bf{0}} 0 A {\bf{A}} A r = 8 r=8 r = 8 n − r = 1 n-r=1 n − r = 1 A {\bf{A}} A x 2 x_2 x 2 y 2 y_2 y 2 x 1 x_1 x 1 y 1 y_1 y 1 h ′ 11 x 1 + h ′ 12 y 1 + h ′ 13 − h ′ 31 x 1 x 2 − h ′ 32 y 1 x 2 − h ′ 33 x 2 = 0 , h ′ 21 x 1 + h ′ 22 y 1 + h ′ 23 − h ′ 31 x 1 y 2 − h ′ 32 y 1 y 2 − h ′ 33 y 2 = 0. (11) \begin{array}{l}
{{h'}_{11}}{x_1} + {{h'}_{12}}{y_1} + {{h'}_{13}} - {{h'}_{31}}{x_1}{x_2} - {{h'}_{32}}{y_1}{x_2} - {{h'}_{33}}{x_2} = 0,\\
{{h'}_{21}}{x_1} + {{h'}_{22}}{y_1} + {{h'}_{23}} - {{h'}_{31}}{x_1}{y_2} - {{h'}_{32}}{y_1}{y_2} - {{h'}_{33}}{y_2} = 0.
\end{array}\tag{11} h ′ 1 1 x 1 + h ′ 1 2 y 1 + h ′ 1 3 − h ′ 3 1 x 1 x 2 − h ′ 3 2 y 1 x 2 − h ′ 3 3 x 2 = 0 , h ′ 2 1 x 1 + h ′ 2 2 y 1 + h ′ 2 3 − h ′ 3 1 x 1 y 2 − h ′ 3 2 y 1 y 2 − h ′ 3 3 y 2 = 0 . ( 1 1 ) h ′ {\bf{h'}} h ′ A {\bf{A}} A h ′ {\bf{h'}} h ′ x 2 x_2 x 2 y 2 y_2 y 2 A {\bf{A}} A h 33 {h_{33}} h 3 3 B = ( a 1 a 2 ⋯ a 8 ) , x = ( h 11 h 12 ⋯ h 32 ) T , h 33 = 1 , B x = h 11 a 1 + h 12 a 2 + ⋯ + h 32 a 8 = − a 9 ⇒ B T B x = − B T a 9 . (12) \begin{array}{l}
{\bf{B}} = \left( {\begin{array}{c}
{{{\bf{a}}_1}}&{{{\bf{a}}_2}}& \cdots &{{{\bf{a}}_8}}
\end{array}} \right),{\rm{ }}{\bf{x}} = {\left( {\begin{array}{c}
{{h_{11}}}&{{h_{12}}}& \cdots &{{h_{32}}}
\end{array}} \right)^T},{\rm{ }}{h_{33}} = 1,\\
{\bf{Bx}} = {h_{11}}{{\bf{a}}_1} + {h_{12}}{{\bf{a}}_2} + \cdots + {h_{32}}{{\bf{a}}_8} = - {{\bf{a}}_9} \Rightarrow {{\bf{B}}^T}{\bf{Bx}} = - {{\bf{B}}^T}{{\bf{a}}_9}.
\end{array}\tag{12} B = ( a 1 a 2 ⋯ a 8 ) , x = ( h 1 1 h 1 2 ⋯ h 3 2 ) T , h 3 3 = 1 , B x = h 1 1 a 1 + h 1 2 a 2 + ⋯ + h 3 2 a 8 = − a 9 ⇒ B T B x = − B T a 9 . ( 1 2 ) B {\bf{B}} B R ( B , a 9 ) = R ( B ) = 8 {R({\bf{B, a}}_9)=R({\bf{B}})=8} R ( B , a 9 ) = R ( B ) = 8 h ′ {\bf{h'}} h ′
不过要注意的是，我们不能选取那些本来为 0 的分量让其等于 1，虽然一般来说 h 33 h_{33} h 3 3 h 33 {h_{33}} h 3 3 h 33 = 0 {h_{33}=0} h 3 3 = 0 A {\bf{A}} A R ( B ) = 7 R({\bf{B}})=7 R ( B ) = 7 R ( B , a 9 ) = 8 ≠ R ( B ) {R({\bf{B, a}}_9)=8≠R({\bf{B}})} R ( B , a 9 ) = 8 = R ( B ) A {\bf{A}} A ∣ ∣ h ∣ ∣ = 1 {||{\bf{h}}||=1} ∣ ∣ h ∣ ∣ = 1
前面我们虽然讨论了很多，实际上都是基于所有匹配点都对应同一个单应性矩阵这一假设的，但在优化问题中，我们通常只能最小化匹配误差，而不能让其为零。也就是说，矩阵 A {\bf{A}} A ∣ ∣ A h ∣ ∣ ||{\bf{Ah}}|| ∣ ∣ A h ∣ ∣ h = 0 \bf{h=0} h = 0 A h ≈ 0 {\bf{Ah}} \approx {\bf{0}} A h ≈ 0 k k k A ⋅ k h ≈ 0 {\bf{A}} \cdot k{\bf{h}} \approx {\bf{0}} A ⋅ k h ≈ 0 h 33 = 1 h_{33}=1 h 3 3 = 1 A {\bf{A}} A h 33 h_{33} h 3 3 k k k
我们知道，函数从不同方向接近极值点时其下降速度并不一定相等。以简单的二次型（只包含 x i x j x_ix_j x i x j f ( x ) = f ( x , y ) = a x 2 + b y 2 (13) f({\bf{x}}) = f(x,y) = a{x^2} + b{y^2}\tag{13} f ( x ) = f ( x , y ) = a x 2 + b y 2 ( 1 3 ) a > b > 0 a > b > 0 a > b > 0
如果我们使用一系列垂直于 X O Y XOY X O Y y = k x {y=kx} y = k x x = 0 x = 0 x = 0 z = ∣ ∣ x ∣ ∣ ⋅ s i g n ( y ) z=||{\bf{x}}||·sign(y) z = ∣ ∣ x ∣ ∣ ⋅ s i g n ( y ) f ( z ) = c z 2 f(z)=cz^2 f ( z ) = c z 2 b ≤ c ≤ a b \le c \le a b ≤ c ≤ a f ( x ) {f(x)} f ( x ) a a a b b b
回到我们前面所讨论的参数优化问题，我们需要最小化 ∣ ∣ A h ∣ ∣ ||{\bf{Ah}}|| ∣ ∣ A h ∣ ∣ f ( h ) = h T A T A h . (14) f({\bf{h}}) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}}.\tag{14} f ( h ) = h T A T A h . ( 1 4 ) f ( k h ) = k 2 f ( h ) {f(k{\bf{h}})=k^2f({\bf{h}})} f ( k h ) = k 2 f ( h ) k {k} k A \bf{A} A A T A {\bf{A}}^T{\bf{A}} A T A f ( k h ) ≥ 0 {f(k{\bf{h}}) \ge 0} f ( k h ) ≥ 0 A h ≈ 0 {\bf{Ah}} \approx {\bf{0}} A h ≈ 0 k {k} k A ⋅ k h ≈ 0 {\bf{A}} \cdot k{\bf{h}} \approx {\bf{0}} A ⋅ k h ≈ 0 k {k} k k k k k k k k k k ∣ ∣ h ∣ ∣ = 1 ||{\bf{h}}||=1 ∣ ∣ h ∣ ∣ = 1 A T A {\bf{A}}^T{\bf{A}} A T A m i n i m i z e f ( h ) = h T A T A h s u b j e c t t o h T h = 1 (15) \begin{array}{l}
{\rm{minimize }}f({\bf{h}}) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}}\\
{\rm{subject to }}{{\bf{h}}^T}{\bf{h}} = 1
\end{array}\tag{15} m i n i m i z e f ( h ) = h T A T A h s u b j e c t t o h T h = 1 ( 1 5 ) λ \lambda λ l ( h , λ ) = h T A T A h + λ ( 1 − h T h ) . (16) l\left( {{\bf{h}},\lambda } \right) = {{\bf{h}}^T}{{\bf{A}}^T}{\bf{Ah}} + \lambda \left( {1 - {{\bf{h}}^T}{\bf{h}}} \right).\tag{16} l ( h , λ ) = h T A T A h + λ ( 1 − h T h ) . ( 1 6 ) h ∗ \bf{h^*} h ∗ f ( h ) f(\bf{h}) f ( h ) λ ∗ \lambda^* λ ∗ ∇ l ( h ∗ , λ ∗ ) = ( D h l ( h ∗ , λ ∗ ) T D λ l ( h ∗ , λ ∗ ) T ) = ( 2 A T A h ∗ − 2 λ ∗ h ∗ 1 − h ∗ T h ∗ ) = 0. (17) \nabla l\left( {{{\bf{h}}^*},{\lambda ^*}} \right) = \left( {\begin{array}{c}
{{D_{\bf{h}}}l{{\left( {{{\bf{h}}^*},{\lambda ^*}} \right)}^T}}\\
{{D_\lambda }l{{\left( {{{\bf{h}}^*},{\lambda ^*}} \right)}^T}}
\end{array}} \right) = \left( {\begin{array}{c}
{2{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} - 2{\lambda ^*}{{\bf{h}}^*}}\\
{1 - {{\bf{h}}^{*T}}{{\bf{h}}^*}}
\end{array}} \right) = {\bf{0}}.\tag{17} ∇ l ( h ∗ , λ ∗ ) = ( D h l ( h ∗ , λ ∗ ) T D λ l ( h ∗ , λ ∗ ) T ) = ( 2 A T A h ∗ − 2 λ ∗ h ∗ 1 − h ∗ T h ∗ ) = 0 . ( 1 7 ) A T A h ∗ = λ ∗ h ∗ , h ∗ T h ∗ = 1. (18) \begin{array}{c}
{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} = {\lambda ^*}{{\bf{h}}^*},\\
{{\bf{h}}^{*T}}{{\bf{h}}^*} = 1.
\end{array}\tag{18} A T A h ∗ = λ ∗ h ∗ , h ∗ T h ∗ = 1 . ( 1 8 ) f ( h ∗ ) = h ∗ T A T A h ∗ = λ ∗ h ∗ T h ∗ = λ ∗ . (19) f({{\bf{h}}^*}) = {{\bf{h}}^{*T}}{{\bf{A}}^T}{\bf{A}}{{\bf{h}}^*} = {\lambda ^*}{{\bf{h}}^{*T}}{{\bf{h}}^*} = {\lambda ^*}.\tag{19} f ( h ∗ ) = h ∗ T A T A h ∗ = λ ∗ h ∗ T h ∗ = λ ∗ . ( 1 9 ) h ∗ \bf{h^*} h ∗ f ( h ) f(\bf{h}) f ( h ) A T A {\bf{A}}^T{\bf{A}} A T A f ( h ∗ ) f(\bf{h^*}) f ( h ∗ )
其实，拉格朗日条件只是一个一阶必要条件，也就是说满足式（18）的特征向量 h ∗ \bf{h^*} h ∗ A T A {\bf{A}}^T{\bf{A}} A T A y = ∑ i b i h i , ∑ i b i 2 = 1. (20) {\bf{y}} = \sum\limits_i {{b_i}{{\bf{h}}_i}} ,{\rm{ }}\sum\limits_i {b_i^2} = 1.\tag{20} y = i ∑ b i h i , i ∑ b i 2 = 1 . ( 2 0 ) y T A T A y = ∑ i b i h i T ⋅ ∑ i λ i b i h i = ∑ i λ i b i 2 . (21) {{\bf{y}}^T}{{\bf{A}}^T}{\bf{Ay}} = \sum\limits_i {{b_i}{\bf{h}}_i^T} \cdot \sum\limits_i {{\lambda _i}{b_i}{{\bf{h}}_i}} = \sum\limits_i {{\lambda _i}b_i^2} .\tag{21} y T A T A y = i ∑ b i h i T ⋅ i ∑ λ i b i h i = i ∑ λ i b i 2 . ( 2 1 ) h 1 T A T A h 1 = λ 1 ≤ y T A T A y ≤ λ 9 = h 9 T A T A h 9 . (22) {\bf{h}}_1^T{{\bf{A}}^T}{\bf{A}}{{\bf{h}}_1} = {\lambda _1} \le {{\bf{y}}^T}{{\bf{A}}^T}{\bf{Ay}} \le {\lambda _9} = {\bf{h}}_9^T{{\bf{A}}^T}{\bf{A}}{{\bf{h}}_9}.\tag{22} h 1 T A T A h 1 = λ 1 ≤ y T A T A y ≤ λ 9 = h 9 T A T A h 9 . ( 2 2 ) A \bf{A} A
综上所述，透视变换的参数优化问题可以转化为求解式（18）的特征向量的问题，而且最优的参数向量为最小特征值对应的特征向量。由于特征向量在某些分量上的值可能很小或者为零，那么在最初提到的直接让 h 33 h_{33} h 3 3 h 33 ≈ 0 h_{33} \approx 0 h 3 3 ≈ 0 ∣ ∣ h ∣ ∣ ||\bf{h}|| ∣ ∣ h ∣ ∣ h 33 h_{33} h 3 3
例如我们有以下一张图片，由于手机摄像头的焦距比较短，所以有明显的透视效应。我们想要更换屏幕上的壁纸，相当于将一张四平八方的图片通过透视变换投影到屏幕包围的区域。前面已经提到，要求解相应的透视变换参数，最少只需要 4 个匹配点对即可。所以，我们只需找到屏幕四个角对应的座标，然后加上我们想要更换的图片的四个角的座标，构造矩阵 A T A {\bf{A}}^T{\bf{A}} A T A
import numpy as np
from PIL import Image
def solve_homo ( ptsrc, ptdst) :
npts = ptsrc. shape[ 0 ]
A = np. zeros( [ 2 * npts, 9 ] )
A[ : npts, [ 0 , 1 ] ] = np. copy( ptsrc)
A[ : npts, 2 ] = np. ones( npts)
A[ npts: , [ 3 , 4 ] ] = np. copy( ptsrc)
A[ npts: , 5 ] = np. ones( npts)
A[ : npts, 6 ] = - ptsrc[ : , 0 ] * ptdst[ : , 0 ]
A[ npts: , 6 ] = - ptsrc[ : , 0 ] * ptdst[ : , 1 ]
A[ : npts, 7 ] = - ptsrc[ : , 1 ] * ptdst[ : , 0 ]
A[ npts: , 7 ] = - ptsrc[ : , 1 ] * ptdst[ : , 1 ]
A[ : npts, 8 ] = - ptdst[ : , 0 ]
A[ npts: , 8 ] = - ptdst[ : , 1 ]
A = A. T. dot( A)
vals, vecs = np. linalg. eig( A)
homo_mat = vecs[ : , np. argmin( vals) ]
if np. max ( np. abs ( homo_mat) ) < 1e - 7 :
homo_mat[ - 1 ] = 1
if np. abs ( homo_mat[ - 1 ] ) > 1e - 7 :
homo_mat /= homo_mat[ - 1 ]
return homo_mat
if __name__ == '__main__' :
src = Image. open ( 'src.jpg' ) . convert( 'RGBA' )
dst = Image. open ( 'dst.jpg' ) . convert( 'RGBA' )
w, h = src. size
W, H = dst. size
ptsrc = np. array( [ [ 0 , 0 ] , [ w- 1 , 0 ] , [ 0 , h- 1 ] , [ w- 1 , h- 1 ] ] )
ptdst = np. array( [ [ 610 , 210 ] , [ 3322 , 388 ] , [ 378 , 1594 ] , [ 2988 , 2268 ] ] )
homo_mat = solve_homo( ptdst, ptsrc)
img = src. transform( ( W, H) , Image. PERSPECTIVE, homo_mat[ : - 1 ] , Image. BICUBIC)
dst. paste( img, img. split( ) [ - 1 ] )
dst = dst. convert( 'RGB' )
dst. save( 'new_wallpaper.jpg' )
图1 透视示例
图2 鱼眼成像示例
图3 常见的变换与运动模型
图4 椭圆等高线
图5 剖面二次曲线
