python加权平均融合矩阵运算(Image Stitching 1)

原創

2020-02-21 19:26

考虑到python通过for循环实现加权平均融合效率比较低，本文采用矩阵运算的形式实现加权平均融合。其中加权平均融合的公式如下：

$f(x,y)=\begin{Bmatrix} f_{1}(x,y) \; \; \; (x,y)\in f_{1}\\ w_{1}(x,y)f_{1}(x,y) + w_{2}(x,y)f_{2}(x,y) \; \; \; (x,y) \in (f_{1}\cap f_{2}) \\ f_{2}(x,y) \;\;\; (x,y)\in f_{2})\\ \end{matrix}$

是融合图像， $f_{1},f_{2}$ 是需要拼接的两幅图像， $w_{1},w_{2}$ 是渐入渐出法中的权重，权重的计算公式如下：

$w_{1}=\frac{x_{r} - x_{i}}{x_{r} - x_{l}} \;\;\; w_{2}= 1-w_{1} = \frac{x_{i}-x_{l}}{x_{r} - x_{l}}$

也就是上面的两个公式，挺简单的，相对于C++而言，用python实现加权平均融合的矩阵运算更加容易理解。直接上结果图，以下是加权平均融合之前与融合之后的对比。

加权平均融合前对比融合后，拼接的折痕消失了，这就是加权平均融合的作用所在。

具体上python代码：

    def removal_seam(self, img_trans, img_targ, transform_corners, threshold=20):
        # img_trans warpPerspective image
        # img_targ target image
        # transform_corners the 4 corners of warpPerspective image

        # corners_orig = np.array([[0, 0, 1],
        #                         [0, img.shape[0], 1],
        #                         [img.shape[1], 0, 1],
        #                         [img.shape[1], img.shape[0], 1]])
        # obtain 4 corners from T transform  

        pano = copy.deepcopy(img_trans)
        pano[0:img_targ.shape[0], 0:img_targ.shape[1]] = img_targ

        x_right = img_targ.shape[1]
        x_left = int(min(transform_corners[0, 0], transform_corners[0, 1]))
        rows = pano.shape[0]
        # calculate weight matrix
        alphas = np.array([x_right - np.arange(x_left, x_right)] * rows) / (x_right - x_left)
        alpha_matrix = np.ones((alphas.shape[0], alphas.shape[1], 3))
        alpha_matrix[:, :, 0] = alphas
        alpha_matrix[:, :, 1] = alphas
        alpha_matrix[:, :, 2] = alphas
        # common area one image no pixels
        alpha_matrix[img_trans[0:rows, x_left:x_right, :] <= threshold] = 1

        img_targ = pano[:, 0:img_targ.shape[1]]
        pano[0:rows, x_left:x_right] = img_targ[0:rows, x_left:x_right] * alpha_matrix \
                                       + img_trans[0:rows, x_left:x_right] * (1 - alpha_matrix)
        
        retrun pano

为了更好理解removal_seam函数，有必要绘制一个简单的示意图(关于img_trans, img_targ)。img_trans的变换需要通过特征匹配计算变换矩阵T，特征提取可以选择为SIFT、SURF、ORB等。