LDR圖像融合HDR: MertensTMO方法

參考文獻：

"Exposure Fusion" by Tom Mertens, Jan Kautz, Frank Van Reeth in Proceedings of Pacific Graphics 2007

 

（1）高斯金字塔

高斯金字塔是最基本的圖像塔。首先將原圖像作爲最底層圖像G0（高斯金字塔的第0層），利用高斯核（5*5）對其進行卷積，然後對卷積後的圖像進行下采樣（去除偶數行和列）得到上一層圖像G1，將此圖像作爲輸入，重複卷積和下采樣操作得到更上一層圖像，反覆迭代多次，形成一個金字塔形的圖像數據結構，即高斯金字塔。

高斯金字塔的當前層圖像就是對其前一層圖像首先進行高斯低通濾波，然後再進行隔行和隔列的降2採樣而生成的。前一層圖像大小依次爲當前層圖像大小的4倍。

Opencv中使用pyrdown函數可獲得高斯金字塔。

function p = pyrGaussGen(img)
[r, c, ~] = size(img);
levels = floor(log2(min(r, c)));
list = [];
for i=1:levels
    %Detail layer
    ts   = struct('detail', img);
    list = [list, ts];  
    %Next level
    img = pyrGaussGenAux(img);
end
%Base layer
p = struct('list', list, 'base', img);
end

function imgOut = pyrGaussGenAux(img)
%5x5 Gaussian Kernel
kernel = [1, 4, 6, 4, 1];
mtx = kernel' * kernel;
mtx = mtx / sum(mtx(:));
%Convolution
imgB = imfilter(img, mtx, 'replicate');
%Downsampling
[r, c] = size(img);
imgOut = imgB(1:2:r, 1:2:c); %imresize(imgB, 0.5, 'bilinear');
end

（2）拉普拉斯金字塔

在高斯金字塔的運算過程中，圖像經過卷積和下采樣操作會丟失部分高頻細節信息。爲描述這些高頻信息，人們定義了拉普拉斯金字塔(Laplacian Pyramid， LP)。用高斯金字塔的每一層圖像減去其上一層圖像的上採樣並高斯卷積之後的預測圖像，得到一系列的差值圖像即爲 LP 分解圖像。

function p = pyrLapGen(img)
[r, c, ~] = size(img);
levels = floor(log2(min(r, c)));
list = [];
for i=1:levels
    %Calculating detail and base layers
    [tL0, tB0] = pyrLapGenAux(img);
    img = tL0;
    %Detail layer
    ts   = struct('detail', tB0);
    list = [list, ts];  
end
%Base layer
p = struct('list', list, 'base', tL0);
end

function [L0, B0] = pyrLapGenAux(img)
%5x5 Gaussian kernel
kernel = [1, 4, 6, 4, 1];
mtx = kernel' * kernel;
mtx = mtx / sum(mtx(:));
%Convolution
imgB = imfilter(img, mtx, 'replicate');
%Downsampling
[r, c] = size(img);
L0 =  imgB(1:2:r, 1:2:c); %imresize(imgB, 0.5, 'bilinear');
%Upsampling
imgE = imresize(L0, [r, c], 'bilinear');
%Difference between the two levels
B0 = img - imgE;
end

function imgOut = MertensTMO(imageStack, weights)
%default parameters if they are not present
if(~exist('weights', 'var'))
    weights = ones(1, 3);
end
wE = weights(1);
wS = weights(2);
wC = weights(3);

%number of images in the stack
[r, c, col, n] = size(imageStack);

%compute the weights for each image
total  = zeros(r, c);
weight = ones(r, c, n);
for i=1:n
    if(wE > 0.0)
        weightE = MertensWellExposedness(imageStack(:,:,:,i));
        weight(:,:,i) = weight(:,:,i) .* weightE.^wE;
    end
    if(wC > 0.0)
        if(size(imageStack(:,:,:,i), 3) > 1)
            L = mean(imageStack(:,:,:,i), 3);
        else
            L = imageStack(:,:,:,i);
        end
        weightC = MertensContrast(L);
        weight(:,:,i) = weight(:,:,i) .* (weightC.^wC);
    end
    if(wS > 0.0)
        weightS = MertensSaturation(imageStack(:,:,:,i));
        weight(:,:,i) = weight(:,:,i) .* (weightS.^wS);
    end
    weight(:,:,i) = weight(:,:,i) + 1e-12;
    total = total + weight(:,:,i);
end
for i=1:n %weights normalization
    weight(:,:,i) = weight(:,:,i) ./ total;
end

%empty pyramid
pyrAcc = [];
for i=1:n
    %Laplacian pyramid: image
    pyrImg = pyrImg3(imageStack(:,:,:,i), @pyrLapGen);
    %Gaussian pyramid: weight   
    pyrW   = pyrGaussGen(weight(:,:,i));
    %image times weight
    pyrImgW = pyrLstS2OP(pyrImg, pyrW, @pyrMul);
    if(i == 1)
        pyrAcc = pyrImgW;
    else %accumulate
        pyrAcc = pyrLst2OP(pyrAcc, pyrImgW, @pyrAdd);
    end
end

%reconstruction
imgOut = zeros(r, c, col);
for i=1:col
    imgOut(:,:,i) = pyrVal(pyrAcc(i));
end

%clamp to values in [0,1]
min_i = min(imgOut(:));
max_i = max(imgOut(:));
imgOut = ClampImg((imgOut - min_i) / (max_i - min_i),  0.0, 1.0);
end

飽和度權重：

function Ws = MertensSaturation(img)
[r,c,col] = size(img);
if(col==1)
    Ws = ones(r, c);
else
    mu = zeros(r,c);
    for i=1:col
        mu = mu + img(:,:,i);
    end
    mu = mu/col;
    sumC = zeros(r,c);
    for i=1:col
        sumC = sumC + (img(:,:,i)-mu).^2;
    end
    Ws = sqrt( sumC/col );
end
end

對比度權重：

function Wc = MertensContrast(L)
H = [0 1 0; 1 -4 1; 0 1 0];
Wc = abs(imfilter(L, H, 'replicate'));
End

優曝光權重：

function We = MertensWellExposedness(img, we_mean, we_sigma)
%mean for the Well-exposedness weights.
if(~exist('we_mean', 'var'))
    we_mean = 0.5; %as in the original paper
end
%sigma for the Well-exposedness weights.
if(~exist('we_sigma', 'var'))
    we_sigma = 0.2; %as in the original paper
end
sigma2 = 2.0 * we_sigma^2;
[r, c, col] = size(img);
We = ones(r, c);
for i=1:col
    We = We .* exp(-(img(:,:,i) - we_mean).^2 / sigma2);
end
end

金字塔對應層相乘：

function pOut = pyrMul(pA, pB)
%checking lenght of the pyramids
nA = length(pA.list);
nB = length(pB.list);
%multiplying base levels
pOut.base = pA.base .* pB.base;
pOut.list = pA.list;
%multiplying the detail of each level
for i=1:nA
    pOut.list(i).detail = pA.list(i).detail .* pB.list(i).detail;
end
end

金字塔圖像重構融合：

function img = pyrVal(pyramid)
list = pyramid.list;
base = pyramid.base;
 
n = length(list);
img=[];
for i=1:n
    ind = n - i + 1;
    [r, c] = size(list(ind).detail);
    if(i == 1)        
        base = imresize(base, [r, c], 'bilinear');
        img  = base + list(ind).detail;
    else
        img = imresize(img, [r, c], 'bilinear');
        img = img + list(ind).detail;        
    end
end
end