四參數擬合之LM(有約束)

前言

之前幾篇文章寫的都是四參數的無約束的優化方法，問題是，四參數本身是有約束的。這裏，通過變化法，將一個有約束的問題轉換爲無約束的問題，再使用LMF法求得最優解。

目標變換

原目標
$arg \ \underset{w}{min}\sum_{i=1}^{m}||y_i-f(w;x_i)||^2+\lambda||\Delta w ||^2 \\f(w;x_i) = D+\frac{A-D}{1+(x_i/C)^B} \\s.t. \ \ \ w>0$
令$A_1 = \sqrt{A},B_1 = \sqrt{B},C_1 = \sqrt{C},D_1 = \sqrt{D}$，則有

$arg \ \underset{w_1}{min}\sum_{i=1}^{m}||y_i-f(w_1;x_i)||^2+\lambda||\Delta w_1 ||^2 \\f(w_1;x_i) = D_1^2+\frac{A_1^2-D_1^2}{1+(x_i/C_1^2)^{B_1^2}} \\w_1 = [A_1,B_1,C_1,D_1]$
這樣就轉爲無約束問題。由於之前無約束的主要問題在於，參數A總是取負值，這裏圖簡便，只針對參數A做處理，其餘不變。
實際目標
$\\f(w_1;x_i) = D+\frac{A_1^2-D}{1+(x_i/C)^{B}} \\w_1 = [A_1,B,C,D]$
目標函數的地形圖，固定A，C，連續參數B，D，梯度還是有的，有一些地方比較平滑。

固定B，D，調整參數A，C

matlab Code

%Levenberg-Marquardt

clear;
load census;
x1 = cdate ;
y1 = pop ;
m = length(x1);
%parameters
eps = 0.001;
eta1 = 0.01;
eta2 = 0.75;
gama1 = 0.5;
gama2 = 2;
lamda = 1;
%init a,b,c,d
d = max(y1)+1;
a = min(y1)-0.1;
y2 = log((y1-d)./(a-y1));
x2 = log(x1);
[curve2,gof2] = fit(x2,y2, 'poly1');
b = -curve2.p1;
c = exp(curve2.p2/b);
%LMF
w=[a,b,c,d]'; 
[res,R,fit] = evaluateFit(y1,x1,w);
mse = 0.5*sum((y1-fit).^2); 
r = y1-fit;
lastRes = 100000;
for k = 1:1:5000
    JacobiMatrix = getJaccobiMatrix(x1,a,b,c,d);
    HessenMatrix = (JacobiMatrix)'*(JacobiMatrix)+lamda*eye(4);
    delta_w = inv(HessenMatrix)*JacobiMatrix'*r;
    w_new = w+delta_w;
    [res,R,fit_new] = evaluateFit(y1,x1,w_new);
    mse_new =0.5*sum( (y1-fit_new).^2);
    
    q = (mse - mse_new)/(r'*JacobiMatrix*delta_w+0.5*delta_w'*HessenMatrix*delta_w);
    de = abs(norm(JacobiMatrix'*r));
     if q<eta1
        lamda = lamda * gama2;
        continue;
    elseif  q>eta2
         lamda = lamda * gama1;
     end
       %coverage
    if de<eps
        break;
    end
    %update a b c d
    a = w_new(1);
    b = w_new(2);
    c = w_new(3);
    d= w_new(4);
    w=w_new; 
    %update compution result
    fit = fit_new;
    mse = mse_new;
    r  = y1-fit;
    R  = sqrt(R);
    de = abs(lastRes-res);
    %coverage
    if de<eps
        return;
    end
end
function [JacobiMatrix] = getJaccobiMatrix(x1,a,b,c,d)
   JacobiMatrix = ones(length(x1),4);
   for i = 1:1:length(x1)
        JacobiMatrix(i,1) = calc_pA(x1(i),a,b,c,d);
        JacobiMatrix(i,2) = calc_pB(x1(i),a,b,c,d);
        JacobiMatrix(i,3) = calc_pC(x1(i),a,b,c,d);
        JacobiMatrix(i,4) = calc_pD(x1(i),a,b,c,d);
    end
end
function val =Lapprox(s,fx,g,G)
        val = fx+g'*s+0.5*s'*G*s;
end
function [res,R2,fit] = evaluateFit(y,x,w)
fit = getFittingValue(x,w);
res =  norm(y-fit)/sqrt(length(fit));
yu =  mean(y);
R2 = 1 - norm(y-fit)^2/norm(y - yu)^2;
end
function fit = getFittingValue(x,w)
len = length(x);
fit = ones(len,1);
for i = 1:1:len
    fit(i)  = hypothesis(x(i),w);
end
end
function val  = hypothesis(x,w)
a = w(1);b= w(2);c= w(3);d= w(4);
val = d+(a^2-d)/(1+(x/c)^b);
end
function val = calc_pA(x,A,B,C,D)
val = (2*A)/((x/C)^B + 1);%1/((x/C)^B + 1);
end
function val = calc_pB(x,A,B,C,D)
val = (log(x/C)*(x/C)^B*(- A^2 + D))/((x/C)^B + 1)^2;%-(log(x/C)*(A - D)*(x/C)^B)/((x/C)^B + 1)^2;
end
function val = calc_pC(x,A,B,C,D)
val =-(B*x*(x/C)^(B - 1)*(- A^2 + D))/(C^2*((x/C)^B + 1)^2);% (B*x*(A - D)*(x/C)^(B - 1))/(C^2*((x/C)^B + 1)^2);
end
function val = calc_pD(x,A,B,C,D)
val = 1 - 1/((x/C)^B + 1);
end

運算結果

L4P運算結果

基本一致，但還是有一些出入