原文鏈接：https://blog.csdn.net/jbb0523/article/details/52079687

題目：壓縮感知重構算法之迭代硬閾值(Iterative Hard Thresholding,IHT)

本篇來介紹IHT重構算法。一般在壓縮感知參考文獻中，提到IHT時一般引用的都是文獻【1】，但IHT實際上是在文獻【2】中提出的。IHT並不是一種凸優化算法，它類似於OMP，是一種迭代算法，但它是由一個優化問題推導得到的。文獻【1】和文獻【2】的作者相同，署名單位爲英國愛丁堡大學(University ofEdinburgh)，第一作者的個人主頁見參考文獻【3】，從個人主頁來看，作者現在已到英國南安普敦大學(University of Southampton)，作者發表的論文均可以從其個人主頁中下載。

文獻【1】的貢獻是當把IHT應用於壓縮感知重構問題時進行了一個理論分析：

1、迭代硬閾值(IHT)的提出

值得一提的是，IHT在文獻【2】中提出時並不叫Iterative Hard Thresholding，而是M-Sparse Algorithm，如下圖所示：

該算法是爲了求解M-稀疏問題(M-sparse problem)式(3.1)而提出的，經過一番推導得到了迭代公式式(3.2)，其中H_M(·)的含義參見式(3.3)。

這裏面最關鍵的問題是：式(3.2)這個迭代公式是如何推導得到的呢？

以下Step1~Step4推導過程可以參見本文的補充說明：迭代硬閾值(IHT)的補充說明，若要透徹地理解IHT，需要知道Majorization-Minimization優化框架和硬閾值(Hard Thresholding)函數。

2、Step1:替代目標函數

首先，將式(3.1)的目標函數用替代目標函數(surrogate objective fucntion)式(3.5)替換：

這裏中的M應該指的是M-sparse，S應該指的是Surrogate。這裏要求：

爲什麼式目標函數式(3.1)可以用式(3.5) 替代呢？這得往回看一下了……

實際上，文獻【2】分別針對兩個優化問題進行了討論，本篇主要是文獻中的第二個優化問題，由於兩個問題有一定的相似性，所以文中在推導第二個問題時進行了一些簡化，下面簡單回顧一些必要的有關第一個問題的內容，第一個優化問題是：

將目標函數定義爲：

爲了推導迭代公式（詳見式(2.2)和式(2.3)）式(1.5)用如下替代目標函數代替：

這裏注意波浪下劃線中提到的“[29]”(參見文獻【4】)，surrogate objective function的思想來自這篇文件。然後注意對Φ的約束（第一個紅框），之後以會有這個約束，個人認爲是爲了使式(2.5)後半部分大於等於零，即爲了使

大於等於零（當y=z時這部分等於零）。由此自然就有了式(2.5)與式(1.5)兩個目標函數的關係（第二個紅框），這也很容易理解，將y=z代入式(2.5)自然可得這個關係。

到此應該明白式(2.5)爲什麼可以替代式(1.5)了吧……

而我們用式(3.5)替代目標函數

的道理是一模一樣的。

補充一點：有關對||Φ||2<1的約束文獻【2】中有一處提到了如下描述：

3、Step2:替代目標函數變形

接下來，式(3.5)進行了變形：

這個式子是怎麼來的呢？我們對式(3.5)進行一下推導：

這裏，後面三項2範數的平方是與y無關的項，因此可視爲常量，若對參數y求最優化時這三項並不影響優化結果，可略去，因此就有了變形的結果，符號“∝”表示成正比例。

4、Step3:極值點的獲得

接下來文獻【2】直接給出了極值點：

注意文中提到了“landweder”，搜索一下可知經常出現的是“landweder迭代”，這個暫且不提。那麼極值點是如何推導得到的呢？其實就是一個簡單的配方，中學生就會的：

令，則

當，取得最小值

5、Step4:迭代公式的獲得

極值點得到了，替代目標函數的極小值也得到了：

那麼如何得到迭代公式式(3.2)呢？這時要注意，推導過程中有一個約束條件一直沒管，即式(3.1)中的約束條件：

也就是向量y的稀疏度不大於M。綜合起來說，替代函數的最小值是

那麼怎麼使這個最小值在向量y的稀疏度不大於M的約束下最小呢，顯然是保留最大的M項（因爲是平方，所以要取絕對值absolute value），剩餘的置零（注意這裏有個負號，所以要保留最大的M項）。

至此，我們就得到了迭代公式式(3.2)。

6、IHT算法的MATLAB代碼

這裏一共給出三個版本的IHT實現：

第一個版本：

在作者的主頁有官方版IHT算法MATLAB代碼，但有些複雜，這裏給出一個簡化版的IHT代碼，方便理解：


function [ y ] = IHT_Basic( x,Phi,M,mu,epsilon,loopmax )  
%IHT_Basic Summary of this function goes here  
%Version: 1.0 written by jbb0523 @2016-07-30  
%Reference:Blumensath T, Davies M E. Iterative Thresholding for Sparse Approximations[J]. 
%Journal of Fourier Analysis & Applications, 2008, 14(5):629-654. 
%(Available at: http://link.springer.com/article/10.1007%2Fs00041-008-9035-z)
%   Detailed explanation goes here  
    if nargin < 6  
        loopmax = 3000;  
    end  
    if nargin < 5    
        epsilon = 1e-3;    
    end   
    if nargin < 4    
        mu = 1;    
    end   
    [x_rows,x_columns] = size(x);    
    if x_rows<x_columns    
        x = x';%x should be a column vector    
    end  
    n = size(Phi,2);  
    y = zeros(n,1);%Initialize y=0  
    loop = 0;  
    while(norm(x-Phi*y)>epsilon && loop < loopmax)  
        y = y + Phi'*(x-Phi*y)*mu;%update y  
        %the following two lines of code realize functionality of H_M(.)  
        %1st: permute absolute value of y in descending order  
        [ysorted inds] = sort(abs(y), 'descend');  
        %2nd: set all but M largest coordinates to zeros  
        y(inds(M+1:n)) = 0;  
        loop = loop + 1;  
    end  
end

第二個版本：（作者給出的官方版本）

文件：hard_l0_Mterm.m(\sparsify_0_5\HardLab)

鏈接：http://www.personal.soton.ac.uk/tb1m08/sparsify/sparsify_0_5.zip


function [s, err_mse, iter_time]=hard_l0_Mterm(x,A,m,M,varargin)
% hard_l0_Mterm: Hard thresholding algorithm that keeps exactly M elements 
% in each iteration. 
%
% This algorithm has certain performance guarantees as described in [1],
% [2] and [3].
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Usage
%
%   [s, err_mse, iter_time]=hard_l0_Mterm(x,P,m,M,'option_name','option_value')
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Input
%
%   Mandatory:
%               x   Observation vector to be decomposed
%               P   Either:
%                       1) An nxm matrix (n must be dimension of x)
%                       2) A function handle (type "help function_format" 
%                          for more information)
%                          Also requires specification of P_trans option.
%                       3) An object handle (type "help object_format" for 
%                          more information)
%               m   length of s 
%               M   non-zero elements to keep in each iteration
%
%   Possible additional options:
%   (specify as many as you want using 'option_name','option_value' pairs)
%   See below for explanation of options:
%__________________________________________________________________________
%   option_name    |     available option_values                | default
%--------------------------------------------------------------------------
%   stopTol        | number (see below)                         | 1e-16
%   P_trans        | function_handle (see below)                | 
%   maxIter        | positive integer (see below)               | n^2
%   verbose        | true, false                                | false
%   start_val      | vector of length m                         | zeros
%   step_size      | number                                     | 0 (auto)
%
%   stopping criteria used : (OldRMS-NewRMS)/RMS(x) < stopTol
%
%   stopTol: Value for stopping criterion.
%
%   P_trans: If P is a function handle, then P_trans has to be specified and 
%            must be a function handle. 
%
%   maxIter: Maximum number of allowed iterations.
%
%   verbose: Logical value to allow algorithm progress to be displayed.
%
%   start_val: Allows algorithms to start from partial solution.
%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Outputs
%
%    s              Solution vector 
%    err_mse        Vector containing mse of approximation error for each 
%                   iteration
%    iter_time      Vector containing computation times for each iteration
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Description
%
%   Implements the M-sparse algorithm described in [1], [2] and [3].
%   This algorithm takes a gradient step and then thresholds to only retain
%   M non-zero elements. It allows the step-size to be calculated
%   automatically as described in [3] and is therefore now independent from 
%   a rescaling of P.
%   
%   
% References
%   [1]  T. Blumensath and M.E. Davies, "Iterative Thresholding for Sparse 
%        Approximations", submitted, 2007
%   [2]  T. Blumensath and M. Davies; "Iterative Hard Thresholding for 
%        Compressed Sensing" to appear Applied and Computational Harmonic 
%        Analysis 
%   [3] T. Blumensath and M. Davies; "A modified Iterative Hard 
%        Thresholding algorithm with guaranteed performance and stability" 
%        in preparation (title may change) 
% See Also
%   hard_l0_reg
%
% Copyright (c) 2007 Thomas Blumensath
%
% The University of Edinburgh
% Email: [email protected]
% Comments and bug reports welcome
%
% This file is part of sparsity Version 0.4
% Created: April 2007
% Modified January 2009
%
% Part of this toolbox was developed with the support of EPSRC Grant
% D000246/1
%
% Please read COPYRIGHT.m for terms and conditions.
 
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                    Default values and initialisation
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
 
[n1 n2]=size(x);
if n2 == 1
    n=n1;
elseif n1 == 1
    x=x';
    n=n2;
else
   error('x must be a vector.');
end
    
sigsize     = x'*x/n;
oldERR      = sigsize;
err_mse     = [];
iter_time   = [];
STOPTOL     = 1e-16;
MAXITER     = n^2;
verbose     = false;
initial_given=0;
s_initial   = zeros(m,1);
MU          = 0;
 
if verbose
   display('Initialising...') 
end
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                           Output variables
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
switch nargout 
    case 3
        comp_err=true;
        comp_time=true;
    case 2 
        comp_err=true;
        comp_time=false;
    case 1
        comp_err=false;
        comp_time=false;
    case 0
        error('Please assign output variable.')        
    otherwise
        error('Too many output arguments specified')
end
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                       Look through options
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
% Put option into nice format
Options={};
OS=nargin-4;
c=1;
for i=1:OS
    if isa(varargin{i},'cell')
        CellSize=length(varargin{i});
        ThisCell=varargin{i};
        for j=1:CellSize
            Options{c}=ThisCell{j};
            c=c+1;
        end
    else
        Options{c}=varargin{i};
        c=c+1;
    end
end
OS=length(Options);
if rem(OS,2)
   error('Something is wrong with argument name and argument value pairs.') 
end
for i=1:2:OS
   switch Options{i}
        case {'stopTol'}
            if isa(Options{i+1},'numeric') ; STOPTOL     = Options{i+1};   
            else error('stopTol must be number. Exiting.'); end
        case {'P_trans'} 
            if isa(Options{i+1},'function_handle'); Pt = Options{i+1};   
            else error('P_trans must be function _handle. Exiting.'); end
        case {'maxIter'}
            if isa(Options{i+1},'numeric'); MAXITER     = Options{i+1};             
            else error('maxIter must be a number. Exiting.'); end
        case {'verbose'}
            if isa(Options{i+1},'logical'); verbose     = Options{i+1};   
            else error('verbose must be a logical. Exiting.'); end 
        case {'start_val'}
            if isa(Options{i+1},'numeric') && length(Options{i+1}) == m ;
                s_initial     = Options{i+1};  
                initial_given=1;
            else error('start_val must be a vector of length m. Exiting.'); end
        case {'step_size'}
            if isa(Options{i+1},'numeric') && (Options{i+1}) > 0 ;
                MU     = Options{i+1};   
            else error('Stepsize must be between a positive number. Exiting.'); end
        otherwise
            error('Unrecognised option. Exiting.') 
   end
end
 
if nargout >=2
    err_mse = zeros(MAXITER,1);
end
if nargout ==3
    iter_time = zeros(MAXITER,1);
end
 
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                        Make P and Pt functions
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
if          isa(A,'float')      P =@(z) A*z;  Pt =@(z) A'*z;
elseif      isobject(A)         P =@(z) A*z;  Pt =@(z) A'*z;
elseif      isa(A,'function_handle') 
    try
        if          isa(Pt,'function_handle'); P=A;
        else        error('If P is a function handle, Pt also needs to be a function handle. Exiting.'); end
    catch error('If P is a function handle, Pt needs to be specified. Exiting.'); end
else        error('P is of unsupported type. Use matrix, function_handle or object. Exiting.'); end
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                        Do we start from zero or not?
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
 
if initial_given ==1;
    
    if length(find(s_initial)) > M
        display('Initial vector has more than M non-zero elements. Keeping only M largest.')
    
    end
    s                   =   s_initial;
    [ssort sortind]     =   sort(abs(s),'descend');
    s(sortind(M+1:end)) =   0;
    Ps                  =   P(s);
    Residual            =   x-Ps;
    oldERR      = Residual'*Residual/n;
else
    s_initial   = zeros(m,1);
    Residual    = x;
    s           = s_initial;
    Ps          = zeros(n,1);
    oldERR      = sigsize;
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                 Random Check to see if dictionary norm is below 1 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        
        x_test=randn(m,1);
        x_test=x_test/norm(x_test);
        nP=norm(P(x_test));
        if abs(MU*nP)>1;
            display('WARNING! Algorithm likely to become unstable.')
            display('Use smaller step-size or || P ||_2 < 1.')
        end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                        Main algorithm
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if verbose
   display('Main iterations...') 
end
tic
t=0;
done = 0;
iter=1;
while ~done
    
    if MU == 0
        %Calculate optimal step size and do line search
        olds                =   s;
        oldPs               =   Ps;
        IND                 =   s~=0;
        d                   =   Pt(Residual);
        % If the current vector is zero, we take the largest elements in d
        if sum(IND)==0
            [dsort sortdind]    =   sort(abs(d),'descend');
            IND(sortdind(1:M))  =   1;    
         end  
        id                  =   (IND.*d);
        Pd                  =   P(id);
        mu                  =   id'*id/(Pd'*Pd);
        s                   =   olds + mu * d;
        [ssort sortind]     =   sort(abs(s),'descend');
        s(sortind(M+1:end)) =   0;
        Ps                  =   P(s);
        
        % Calculate step-size requirement 
        omega               =   (norm(s-olds)/norm(Ps-oldPs))^2;
        % As long as the support changes and mu > omega, we decrease mu
        while mu > (0.99)*omega && sum(xor(IND,s~=0))~=0 && sum(IND)~=0
%             display(['decreasing mu'])
                    
                    % We use a simple line search, halving mu in each step
                    mu                  =   mu/2;
                    s                   =   olds + mu * d;
                    [ssort sortind]     =   sort(abs(s),'descend');
                    s(sortind(M+1:end)) =   0;
                    Ps                  =   P(s);
                    % Calculate step-size requirement 
                    omega               =   (norm(s-olds)/norm(Ps-oldPs))^2;
        end
        
    else
        % Use fixed step size
        s                   =   s + MU * Pt(Residual);
        [ssort sortind]     =   sort(abs(s),'descend');
        s(sortind(M+1:end)) =   0;
        Ps                  =   P(s);
        
    end
        Residual            =   x-Ps;
        
     ERR=Residual'*Residual/n;
     if comp_err
         err_mse(iter)=ERR;
     end
     
     if comp_time
         iter_time(iter)=toc;
     end
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                        Are we done yet?
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     
         if comp_err && iter >=2
             if ((err_mse(iter-1)-err_mse(iter))/sigsize<STOPTOL);
                 if verbose
                    display(['Stopping. Approximation error changed less than ' num2str(STOPTOL)])
                 end
                done = 1; 
             elseif verbose && toc-t>10
                display(sprintf('Iteration %i. --- %i mse change',iter ,(err_mse(iter-1)-err_mse(iter))/sigsize)) 
                t=toc;
             end
         else
             if ((oldERR - ERR)/sigsize < STOPTOL) && iter >=2;
                 if verbose
                    display(['Stopping. Approximation error changed less than ' num2str(STOPTOL)])
                 end
                done = 1; 
             elseif verbose && toc-t>10
                display(sprintf('Iteration %i. --- %i mse change',iter ,(oldERR - ERR)/sigsize)) 
                t=toc;
             end
         end
         
    % Also stop if residual gets too small or maxIter reached
     if comp_err
         if err_mse(iter)<1e-16
             display('Stopping. Exact signal representation found!')
             done=1;
         end
     elseif iter>1 
         if ERR<1e-16
             display('Stopping. Exact signal representation found!')
             done=1;
         end
     end
 
     if iter >= MAXITER
         display('Stopping. Maximum number of iterations reached!')
         done = 1; 
     end
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                    If not done, take another round
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   
     if ~done
        iter=iter+1; 
        oldERR=ERR;        
     end
end
 
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                  Only return as many elements as iterations
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
if nargout >=2
    err_mse = err_mse(1:iter);
end
if nargout ==3
    iter_time = iter_time(1:iter);
end
if verbose
   display('Done') 
end

第三個版本：

文件：Demo_CS_IHT.m（部分）

鏈接：http://www.pudn.com/downloads518/sourcecode/math/detail2151378.html


function hat_x=cs_iht(y,T_Mat,m)
% y=T_Mat*x, T_Mat is n-by-m
% y - measurements
% T_Mat - combination of random matrix and sparse representation basis
% m - size of the original signal
% the sparsity is length(y)/4
 
hat_x_tp=zeros(m,1);         % initialization with the size of original 
s=floor(length(y)/4);        % sparsity
u=0.5;                       % impact factor
 
% T_Mat=T_Mat/sqrt(sum(sum(T_Mat.^2))); % normalizae the whole matrix
 
for times=1:s
    
    x_increase=T_Mat'*(y-T_Mat*hat_x_tp);
    
    hat_x=hat_x_tp+u*x_increase;
    
    [val,pos]=sort((hat_x),'descend');  % why? worse performance with abs()
    
    hat_x(pos(s+1:end))=0;   % thresholding, keeping the larges s elements
 
    hat_x_tp=hat_x;          % update
 
end

7、單次重構代碼

%壓縮感知重構算法測試


clear all;close all;clc;      
M = 64;%觀測值個數      
N = 256;%信號x的長度      
K = 10;%信號x的稀疏度      
Index_K = randperm(N);      
x = zeros(N,1);      
x(Index_K(1:K)) = 5*randn(K,1);%x爲K稀疏的，且位置是隨機的      
Psi = eye(N);%x本身是稀疏的，定義稀疏矩陣爲單位陣x=Psi*theta      
Phi = randn(M,N);%測量矩陣爲高斯矩陣  
Phi = orth(Phi')';    
A = Phi * Psi;%傳感矩陣    
% sigma = 0.005;    
% e = sigma*randn(M,1);  
% y = Phi * x + e;%得到觀測向量y      
y = Phi * x;%得到觀測向量y    
%% 恢復重構信號x      
tic      
theta = IHT_Basic(y,A,K); 
% theta = cs_iht(y,A,size(A,2));
% theta = hard_l0_Mterm(y,A,size(A,2),round(1.5*K),'verbose',true);
x_r = Psi * theta;% x=Psi * theta      
toc      
%% 繪圖      
figure;      
plot(x_r,'k.-');%繪出x的恢復信號      
hold on;      
plot(x,'r');%繪出原信號x      
hold off;      
legend('Recovery','Original')      
fprintf('\n恢復殘差：');      
norm(x_r-x)%恢復殘差

這裏就不給出重構結果了，給出仿真結論：本人編的IHT基本版能夠正常工作，但偶爾會重構失敗；第二個版本hard_l0_Mterm.m重構效果很好；第三個版本Demo_CS_IHT.m重構效果很差，估計是作者疑問（why? worse performance with abs()），沒有加abs取絕對值的原因吧……

8、結束語

8.1有關算法的名字

值得注意的是，在文獻【2】中將式(2.2)稱爲iterative hard-thresholding algorithm，而將式(3.2)稱爲M-sparse algorithm，在文獻【1】中又將式(3.2)稱爲Iterative Hard Thresholding algorithm (IHTs)，一般簡稱IHT的較多，多餘的s指的是s稀疏。可見算法的名稱是也是一不斷完善的過程啊……

8.2與GraDeS算法的關係

如果你學習過GraDeS算法（參見http://blog.csdn.net/jbb0523/article/details/52059296），然後再學習本算法，是不是有一種似曾相似的感覺？

沒錯，這兩個算法的迭代公式幾乎是一樣的，尤其是文獻【1】中的式(12)（如上圖第二個紅框）進一步拓展了該算法的定義。這個就跟CoSaMP與SP兩個算法一樣，在GraDeS的提出文獻【5】中開始部分還提到了IHT，但後面就沒提了，不知道作者是怎麼看待這個問題的。如果非說二者有區別，那就是GraDeS的參數γ=1+δ_2s，且δ_2s<1/3。

所以，有想法得趕緊寫成論文發表出來，否則被搶了先機那就……

8.3重構效果問題

另外，在GraDeS算法中提到該算法的重構效果不好，這裏注意文獻【2】中的一段話：

也就是說，IHT作者也意識到了該種算法的問題，並提出了兩種應用策略(two strategies for asuccessful application of the methods)。

8.4Landweber迭代

在網上搜索“Landweber迭代”時找到了一段程序^[6]：


function [x,k]=Landweber(A,b,x0)
alfa=1/sum(diag(A*A'));
k=1;
L=200;
x=x0;
while k<L
    x1=x;
    x=x+alfa*A'*(b-A*x);
    if norm(b-A*x)/norm(b)<0.005
        break;
    elseif norm(x1-x)/norm(x)<0.001
        break;
    end
    k=k+1;
end

注意該程序的迭代部分“x=x+alfa*A'*(b-A*x);”，除了多了一些alfa係數外，這跟IHT不是基本一樣麼？或者說與GraDeS有什麼區別？

有關LandWeber迭代可參見文獻：“Landweber L. An iteration formula for Fredholm integral equations of the first kind[J]. American journal of mathematics, 1951, 73(3): 615-624.”，此處不再多述。

8.5改進算法

作者後來又提出了兩個關於IHT的改進算法，分別是RIHT(Normalized IHT)^[7]和AIHT(Accelerated IHT)^[8]。

提出RIHT主要是由於IHT有一些缺點^[7]：

新算法RIHT將會有如下優點：

之所以作者提供的軟件包（第二個版本IHT）重構效果更好是由於最新版的hard_l0_Mterm.m (\sparsify_0_5\HardLab)程序中已經更新成了RIHT。

RIHT的算法流程如下：

將IHT改進爲AIHT後會有如下優點^[8]：

值得注意的是，AIHT應該是一類算法的總稱（雖然作者只闡述了兩種實現策略），這個類似於FFT是所有DFT快速算法的總稱：

8.6稀疏度對IHT的影響

自己可以試一下，IHT輸入參數中的稀疏度並不是很關鍵，若實際稀疏度爲K，則稀疏度這個輸入參數只要不小於K就可以了，重構效果都挺不錯的，比如第三個版本的IHT程序，作者直接將稀疏度定義爲信號y長度的四分之一。

8.7作者去向？

細心的人會發現，文獻【8】的暑名單位爲劍橋大學(University of Oxford)，並不是作者主頁所在的南安普敦大學(University of Southampton)，在文獻【8】的最後南提到：

Previous position?難道作者跳到Oxford了？

9、參考文獻

【1】Blumensath T, Davies M E.Iterative hard thresholding for compressed sensing[J]. Applied & Computational HarmonicAnalysis, 2008, 27(3):265-274. (Available at:http://www.sciencedirect.com/science/article/pii/S1063520309000384)

【2】Blumensath T, Davies M E.Iterative Thresholding for Sparse Approximations[J]. Journal of Fourier Analysis & Applications,2008, 14(5):629-654. (Available at:http://link.springer.com/article/10.1007%2Fs00041-008-9035-z)

【3】Homepageof Blumensath T :http://www.personal.soton.ac.uk/tb1m08/index.html

【4】Lange, K., Hunter, D.R., Yang, I.. OptimizationTransfer Using Surrogate Objective Functions[J]. Journal of Computational &Graphical Statistics, 2000, 9(1):1-20. (Available at: http://sites.stat.psu.edu/~dhunter/papers/ot.pdf)

【5】GargR, Khandekar R. Gradient descent with sparsification: an iterative algorithmfor sparse recovery with restricted isometry property[C]//Proceedings of the26th Annual InternationalConference on Machine Learning. ACM, 2009: 337-344

【6】shasying2. landweber迭代方法.http://download.csdn.net/detail/shasying2/5092828

【7】Blumensath T, Davies M E.Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance[J]. IEEE Journal of Selected Topics in Signal Processing, 2010,4(2):298-309.

【8】Blumensath T. Accelerated iterative hard thresholding[J]. Signal Processing, 2012, 92(3):752-756.

迭代硬閾值MATLAB代碼

1、迭代硬閾值(IHT)的提出

2、Step1:替代目標函數

3、Step2:替代目標函數變形

4、Step3:極值點的獲得

5、Step4:迭代公式的獲得

6、IHT算法的MATLAB代碼

7、單次重構代碼

8、結束語

8.1有關算法的名字

8.2與GraDeS算法的關係

8.3重構效果問題

8.4Landweber迭代

8.5改進算法

8.6稀疏度對IHT的影響

8.7作者去向？

9、參考文獻

linux安裝cuda和cudnn

測試人員都是畫畫大神，讓我看看誰還不會用代碼圖？

Object.values()對象遍歷

我拍了拍Redis，被移出了羣聊···

網絡現代化通向雲原生應用的高速公路

面試官：說說你對序列化的理解

我宣佈，這是我找到的史上AI最全論文體系！

硬閾值(Hard Thresholding)函數解讀

DenseNet解讀 DenseNet解讀

EventSR: From Asynchronous Events to Image Reconstruction, Restoration, andSuper-Resolution

圖像超分辨的幾種方法

從SRCNN到EDSR，總結深度學習端到端超分辨率方法發展歷程從SRCNN到EDSR，總結深度學習端到端超分辨率方法發展歷程

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結