Gradient Projection for Sparse Reconstruction
Mário Figueiredo, Robert D. Nowak, Stephen J. Wright
Institutode Telecomunicações Electricaland Computer Engineering ComputerSciences Dept.
InstitutoSuperior Técnico Universityof Wisconsin-Madison Universityof Wisconsin-Madison
Lisboa,PORTUGAL Madison,WI, USA Madison,WI, USA
Many problems in signal processing andstatistical inference are based on finding a sparse solution to an undeterminedlinear system of equations.
BasisPursuit, the Least Absolute Shrinkage and Selection Operator (LASSO), wavelet-baseddeconvolution, and CompressedSensing are just a few well-known examples.
Computationally, the problem can be formulatedin different ways, most of them being convex optimization problems. Weconsidered a formulation in which a penalty term involving the scaled l1-normof the signal is added to a least-squares term, a problem that can bereformulated as a convex quadratic program with bound constraints. This problemhas a potentially extremely large number of variables (though only a smallfraction of them are away from their bounds at the solution) and the data thatdefines it can often not be stored explicitly. We found that a solver ofgradient projection type, using special line search and termination techniques,gave faster solutions on our test problems than other techniques that had beenproposed previously, including interior-point techniques. A debiasing step based on theconjugate-gradient algorithm improves the results further.
Finalversion, September 12, 2007.
To appear in the IEEE Journal of SelectedTopics in Signal Processing: Special Issue on Convex Optimization Methods forSignal Processing).
NEW! Updated version (January 19, 2009) of theMATLAB code is available here: GPSR_6.0
The figure below shows a test case of a signalwith 4096 elements only 160 of which are not zero, and which is beingreconstructed from projection on 1024 unit-norm random vectors in4096-dimensional space; this is, of course, a highly under-determinedproblem. The true signal is shown at thetop, while the reconstructions obtained from the l1-regularizedformulation are shown in the second and third plots. Note that the locations ofthe spikes are reconstructed with high accuracy; their magnitudes areattenuated, but these can be corrected by applying our conjugate-gradientdebiasing approach. The lower part of the figure shows the minimum normsolution, which is not sparse and which bears little relation to the truesignal.
The figures below shows a comparison, in termsof computational speed, of GPSR versus three state-of-the-art solvers for thesame problem:
· thel1-magic code, available here (from CalTech);
· theSparseLab code, available here (from Stanford);
· thenew l1_ls code (March 2007),available here (fromStanford);
· thebound-optimization method (or iterative shrinkage/thresholding – IST),originally developed for wavelet-based deconvolution, described here.
The plots shows that our GPSR method is fasterand scales more favorably
(w.r.t.n, the length of the unknown signal) than the competing techniques.
See the paper for details about theexperiments.
Funding Acknowledgment:
· Thiswork was partially supported by the USA National Science Foundation (NSF),under grants CCF-0430504 and CNS-0540147.
· Thiswork was partially supported by the Portuguese Fundação para a Ciência e Tecnologia (FCT), under projectPOSC/EEA-CPS/61271/2004.