Optimal Estimation of L1-Regularization Prior from a Regularized Empirical Bayesian Risk Standpoint

Inverse Problem and Imaging 2012

Hui Huang1*     Eldad Haber2     Lior Horesh3   

  1VisuCA/SIAT   2The University of British Columbia    

3Business Analytics and Mathematical Sciences IBM T J Watson Research Center      

Figure 1: True MRI images are on the top; corrupted data are in the middle; our reconstructions with L1D regularization are at the bottom.

We address the problem of prior matrix estimation for the solution of `1-regularized ill-posed inverse problems. From a Bayesian viewpoint, we show that such a matrix can be regarded as an in uence matrix in a multivariate `1-Laplace density function. Assuming a training set is given, the prior matrix design problem is cast as a maximum likelihood term with an additional sparsity-inducing term. This formulation results in an unconstrained yet nonconvex optimization problem. Memory requirements as well as computation of the nonlinear, nonsmooth sub-gradient equations are prohibitive for large-scale problems. Thus, we introduce an iterative algorithm to design ecient priors for such large problems. We further demonstrate that the solutions of ill-posed inverse problems by incorporation of `1-regularization using the learned prior matrix perform generally better than commonly used regularization techniques where the prior matrix is chosen a-priori.


(a) True images                                                             (b) Training images                                                         (c) Corrupted images

Figure 2: True, training and corrupted input data sets for the following deconvolution.

(a) Reconstruction with DWT regularization         (b) Reconstruction with adaptive regularization          (c) Reconstruction with L1D regularization

Figure 3: Comparing reconstructions on car images with dierent regularization schemes.

Thanks go to Mark Schmidt for his Matlab code minFunc, which has been modi ed to solve our unconstrained optimization problem. We also thank the authors of [16] for making their codes available online. The car database is downloaded from the Center for Biological and Computational Learning (CBCL) at MIT. This study was partially supported by NSFC (61103166), Guangdong Science and Technology Program (2011B050200007), Shenzhen Science and Innovation Program (CXB201104220029A) and a Design in Inversion Open Collaborative Research (OCR) program between IBM TJ Watson Research Center, the University of British Colombia and MITACS.
@ARTICLE{Optimal Estimation 2012
title = {Optimal estimation of L1 regularization prior from a regularized empirical bayesian risk},
author = {Hui Huang and Eldad Haber and Lior Horesh},
journal = {Inverse Problems and Imaging},
volume = {6},
issue = {3},
pages = {447-464},
year = {2012},

Copyright © 1998-2018 Visual Computing Research Center