garrote variational garrote.
DESCRIPTION
Variational garrote implementation of sparse regression
REFERENCE
http://arxiv.org/abs/1109.0486
EXAMPLE
n=10; % input dimension
p=100; % number of samples
pt = 2*n;
ns = round(.05*n); % sparsity level
betax=1; % inverse noise in the input
betah=1; % inverse noise response variance
snonzero=randperm(n);
snonzero(ns+1:end) = [];
w=sparse(1,n);
w(snonzero)=1;
sigma=sqrt(1/betah); % noise response
sigmax=sqrt(1/betax); % noise input
x=sigmax*randn(n,p);
x=x-mean(x,2)*ones(1,p);
dx=sqrt(1/p*sum(x.^2,2));
x=x./(dx*ones(1,p));
y=w*x+sigma*randn(1,p);
y=y-mean(y);
xt=sigmax*randn(n,pt);
xt=xt-mean(xt,2)*ones(1,pt);
yt=w*xt+sigma*randn(1,pt);
yt=yt-mean(yt);
m = dml.garrote('valset',20);
m = m.train(x',y');
yp = m.test(xt');
plot([yt; yp]');
DEVELOPERS
Bert Kappen (b.kappen@science.ru.nl)
Vicenc Gomez (v.gomez@science.ru.nl)
| beta_max |
increases gamma values until beta=beta_max (1e3) |
| dmmin |
convergence threshold for mean field error (1e-12) |
| indims |
dimensions of the input data (excluding the trial dim and time dim in time series data) |
| max_sum_m |
increases gamma values until sum(m)=max_sum_m (n/2) |
| maxiter |
maximum number of iterations for optimization for fixed gamma (1e4) |
| method |
method for optimization 'dual' or 'regression' for fixed gamma ('dual') |
| n_gamma |
number of gamma values to scan (50) |
| res |
learnt parameters |
| restart |
when false, starts at the previously learned parameters; needed for online learning and grid search |
| valset |
part of the training set used for validation (0.1*p) |
| verbose |
whether or not to generate diagnostic output |