View
3
Download
0
Category
Preview:
Citation preview
RANDOM TOPICSstochastic gradient descent
&Monte Carlo
MASSIVE MODEL FITTING
minimize1
2kAx� bk2 =
X
i
1
2(aix� bi)
2
least squares
minimize1
2kwk2 + h(LDw) =
1
2kwk2 +
X
i
h(lidiw)
SVM
low-rank factorization
Big!(over 100K)
minimize f(x) =1
n
nX
i=1
fi(x)
minimize1
2kD �XY k2 =
X
ij
1
2(dij � xiyj)
2
THE BIG IDEA
True gradient
⌦ ⇢ [1, N ]
Idea: choose a subset of the data
usually just one sample
minimize f(x) =1
n
nX
i=1
fi(x)
rf =1
n
nX
i=1
rfi(x)
rf ⇡ g⌦ =1
|⌦|X
i2⌦
rfi(x)
INFINITE SAMPLE VS FINITE SAMPLE
minimize f(x) = Es[fs(x)] =
Z
sfs(x)p(s)ds
minimize f(x) =1
n
nX
i=1
fi(x)
We can solve finite sample problem to high accuracy….
…but true accuracy is limited by sample size
finite sample
infinite sample
SGD
big errorsolution improves
compute gradientselect data
update
gk =1
M
MX
i=1
rf(x, di)gk ⇡ rf(x, d8)gk ⇡ rf(x, d12)
small errorsolutions gets worse
xk+1 = xk � ⌧kgk
SGDcompute gradientselect
dataupdate
gk ⇡ rf(x, d8) xk+1 = xk � ⌧kgk
Error must decrease as we approach solution
limk!1
⌧k = 0
classical solution
O(1/pk)
shrink stepsize slow convergence
Variance reductionCorrect error in gradient approximations
EXAMPLE: WITHOUT DECREASING STEPSIZE
minimize1
2kAx� bk2
inexact gradientwhy does this happen?
what's happening?
DECREASING STEP SIZEbig stepsize
small stepsize
⌧k =a
b+ k
used for strongly convex problems
xk+1 = xk � ⌧krfk(x)
(almost) equivalent tochoose larger sample
why?
AVERAGING
averaging
xk+1 = xk � ⌧krfk(x)
⌧k =apk + b
xk+1 =1
k + 1
Xxi
“ergodic” averaging
used for weakly convex problems
x
does this limit convergence rate?
why is this bad?
AVERAGING
xk+1 =1
k + 1
Xxi
ergodic averaging
averaging
x
xk+1 =k
k + 1xk +
1
k + 1xk+1
compute without storage
xk+1 =k
k + ⌘xk +
⌘
k + ⌘xk+1
short memory version
⌘ � 1
tradeoff variance for bias
KNOWN RATES: WEAKLY CONVEX
TheoremSuppose f is convex, krf(x)k G, and that the diameterof dom(f) is less than D. If we use the stepsize
⌧k =cpk,
then
SeeShamir and Zhang, ICML ’13
Rakhlin, Shamir, Sridharan, CoRR ‘11
E[f(xk)� f?] ✓D2
c+ cG
◆2 + log(k)p
k
KNOWN RATES: STRONGLY CONVEX
Theorem
Shamir and Zhang, ICML ’13
E[f(xk)� f?] 58(1 + ⌘/k)
✓⌘(⌘ + 1) +
(⌘ + .5)3(1 + log k)
k
◆G2
mk
xk+1 =k
k + ⌘xk +
⌘
k + ⌘xk+1
Suppose f is strongly convex with parameter m, and thatkrf(x)k G. If you use stepsize ⌧k = 1/mk, and the limitedmemory averaging
with ⌘ � 1, then
EXAMPLE: SVMPEGASOS: Primal Estimated sub-GrAdient SOlver for SVM
rh(x) =
(�1, x < 1
0, otherwise
note: this is a “subgradient” descent method
minimize1
2kwk2 + Ch(Aw)
PEGOSOSPEGASOS: Primal Estimated sub-GrAdient SOlver for SVM
⌧k =1
�k
minimizeX
i
�
2kwk2 + h(aiw)
While “not converged”:
not used in practicewk+1 =1
k + 1
k+1X
i=1
wk
If aTkwk < 1 : wk+1 = wk � ⌧k(�wk � aTi )
If aTkwk � 1 : wk+1 = wk � ⌧k�wk
PEGOSOS
gradientmethod
stochasticmethods
finite sample method
PEGOSOS
gradientmethod
stochasticmethods
classification error (CV)
You hope to converge before SGD gets slow!
ALLREDUCEbut if you don’t…
use SGD as warm start for iterative method
Agarwal. “Effective Terascale Linear Learning.” ‘12
SGDcompute gradientselect
dataupdate
gk ⇡ rf(x, d8) xk+1 = xk � ⌧kgk
Error must decrease as we approach solution
variance reduction solutionmake gradient more accurate
preserve fast convergence
SGD+VARIANCE REDUCTIONcompute gradient update
gk ⇡ rf(x, d8) xk+1 = xk � ⌧kgk
Error must decrease as we approach solution
variance reduction solutionmake gradient more accurate
preserve fast convergence
� error8
select data
VR APPROACHES
Central VRA VR approach targeting distributed ML
SAGADefazio, Bach, Lacoste-Julian, 2014
SVRGJohnson, Zhang, 2013
SAGLe Roux, Schmidt, Bach, 2013
many more…
“Efficient Distributed SGD with Variance Reduction,” ICDM 2016
shameless self-promotion
The original: requires full gradient computations
Avoid full gradient computations
SVRG
rf1(x1m)
rf2(x2m)
rfn(xnm)
...
rfn�1(xn�1m )
First epoch gradient tableau
rf3(x3m)
rf1(x1m)
rf2(x2m)
rfn(xnm)
...
rfn�1(xn�1m )} gm =
1
n
nX
i=1
rfi(xim)
Average gradientover last epoch rf3(x
3m)
gradient tableau
SVRG
rf1(x1m)
rf2(x2m)
rfn(xnm)
...
rfn�1(xn�1m )
gm =1
n
nX
i=1
rfi(xim)
Average gradientover last epoch
rf3(x3m)
(rf3(x3m)� gm)�
corrected gradient}error
rf3(x3m+1)
new gradient
gradient tableau
SVRG
SVRG GETS BACK DETERMINISTIC RATES
Theorem
Johnson and Zhang, 2013
Suppose of objective terms are m stronglyconvex with L Lipschitz gradients. If thelearning rate is small enough, then
<latexit sha1_base64="HTGAliInHfDxWKYbVWtacL71/ok=">AAACj3icbVHLbtNAFB2bVzGvFJbdXBFXYoEiOyzKCkWwoVIXRZC2UhxF48m1PXQe1sx1IET5Br6PT+AD2DNJs6AtVxrp6Dx0r86UrZKesuxXFN+5e+/+g72HyaPHT54+6+0/P/O2cwLHwirrLkruUUmDY5Kk8KJ1yHWp8Ly8/LDRzxfovLTmCy1bnGpeG1lJwSlQs97PolKdbxRWBMnnrm2tR7AV2PIrCpILBEKnPXCHkOoUPDlrarUsigSENQv8Dt8kNZCepHAiWy8aST+gdnwu0ZAfABxXQA1CCCjkzkhTg+OEID14zZUCNLarm9cbl0lmvX42yLYDt0G+A322m9NZ73cxt6LTYZtQ3PtJnrU0XXFHUihcJ0XnseXiktc4CdBwjX662ha3hsPAzKGyLjxDsGX/Tay49n6py+DUnBp/U9uQ/9MmHVVvpytp2o7QiKtFVaeALGx+AebShXrVMgAunAy3gmi44yK07ZPk2h4jBVZBWodu8ptN3AZnw0H+ZjD8NOyP3u9a2mMH7CV7xXJ2xEbsIztlYybYn+ggSqPDeD8+it/FoytrHO0yL9i1iY//AlY7xdk=</latexit>
Ef(wk)� f? ck[f(w0)� f?]<latexit sha1_base64="FkMPZaQmJTfxjKaog3LujK0S4+U=">AAACNHicbVBNS8NAEN3U7/pV9ehlsQj1YElqQY+iCB4rWBWatGy2k3bpZhN2N0oJ/SP+EM9e9R8I3sSDF3+DmzYHW30w8Hhvhpl5fsyZ0rb9ZhXm5hcWl5ZXiqtr6xubpa3tGxUlkkKTRjySdz5RwJmApmaaw10sgYQ+h1t/cJ75t/cgFYvEtR7G4IWkJ1jAKNFG6pTqbkh03/fTixEOKg/twcEhDtqu0kRilwOm7QFuZYZ9cJjrHu6UynbVHgP/JU5OyihHo1P6crsRTUIQmnKiVMuxY+2lRGpGOYyKbqIgJnRAetAyVJAQlJeOvxvhfaN0cRBJU0Ljsfp7IiWhUsPQN53ZL2rWy8T/vFaigxMvZSJONAg6WRQkHOsIZ1HhLpNANR8aQqhk5lZM+0QSqk2gxeLUHsEoBMYamWyc2ST+kpta1Tmq1q7q5dOzPKVltIv2UAU56BidokvUQE1E0SN6Ri/o1Xqy3q0P63PSWrDymR00Bev7BxyTqZY=</latexit>
for some c < 1.<latexit sha1_base64="7i+9y8tRinww0pMsrhRs+pZ9DCo=">AAACD3icbVC9TgJBGNzDP8QfTi1tNoKJFbnDQgsLoo0lJvKTACF7y3ewYW/3srtnQggPYW2rz2BnbH0EH8G3cA+uEHCqycx8+SYTxJxp43nfTm5jc2t7J79b2Ns/OCy6R8dNLRNFoUEll6odEA2cCWgYZji0YwUkCji0gvFd6reeQGkmxaOZxNCLyFCwkFFirNR3i6FUWMsIcJne+JVy3y15FW8OvE78jJRQhnrf/ekOJE0iEIZyonXH92LTmxJlGOUwK3QTDTGhYzKEjqWCRKB703nxGT63ygCnFUIpDJ6rfy+mJNJ6EgU2GREz0qteKv7ndRITXvemTMSJAUEXj8KEYyNxugIeMAXU8IklhCpmu2I6IopQY7cqFJb+CEYhtNbMbuOvLrFOmtWKf1mpPlRLtdtspTw6RWfoAvnoCtXQPaqjBqIoQS/oFb05z8678+F8LqI5J7s5QUtwvn4BB86bCg==</latexit>
MONTO CARLO METHODS
Monte Carlo casino
Methods that involve randomly sampling a distribution
Monaco?
BAYESIAN LEARNING
ingredients…prior
likelihood
estimate parameters in a model
probability distribution of unknown parameters
data = M(parameters)estimate
thesemeasure
these
probability of datagiven (unknown) parameters Thomas Bayes
BAYES RULE
P (A|B) =P (B|A)P (A)
P (B)
P (parameters|data) = P (data|parameters)P (parameters)
P (data)
likelihood prior
P (parameters|data) / P (data|parameters)P (parameters)
we really care about this
“posterior distribution”
EXAMPLE: LOGISTIC REGRESSION
P (parameters|data) / P (data|parameters)P (parameters)
prior
P (y,X|w) =Y
i
exp(yi · xTi w)
1 + exp(yi · xTi w)
<latexit sha1_base64="lWgkerH9AERziYYhNxTP4CqGnHg=">AAAIGXicfVXNbtw2EFaSNpu6f0567EWoE8BuXGPlBEgvBZK6SNuDUbeIHQOmLVBcasUsfxSSyu6W1pPkGfICufVW9JpTgfZdOtTuNruU1jxIQ37fzHBmyGFWcmZsv//3tes3PvjwZu/WRxsff/LpZ59v3r5zYlSlCT0miit9mmFDOZP02DLL6WmpKRYZp8+z0YHHn7+i2jAln9lpSc8FHkqWM4ItLKWbPx5tT3dPL8c78XcxKrUapAzlGhOH6KTcnsKMDJSNJym7eDbeqV1yvxtIN7f6e/1mxG0hmQtb0XwcpbdvvkUDRSpBpSUcG3OW9Et77rC2jHBab6DK0BKTER5Sh4UxU5HV8T2BbWFCzC92YpMmQfXGxr2l1bPK5t+eOybLylJJQBGwvOKxVbFPUTxgmhLLpyBgohnsJyYFhqRYSOSKKZcpNbI4M+ACSTomSggsBw4RY+qz5NyhjA6ZdAQqZOqtBFEAZ5N6VUHZguoxM7R2yNKJdfH7ldA4ZwKOBthwSLNhYbHWalyHnHxh6ec8Dr1xsC3/9+QncctLgbk30RyGpHb7oQ2NvX8sh02xVqx7hHchxgJSZGriYrSLdk2VvYBMQ+L9rNnCKp/JASj4uhLM3Wm4BSZf1e7CfZOEgGU+OFx1YUrZpbA8q6X9ntHA7dhHpfc7ut8yPhIN0OG1BJsXkACswzxSzM08yixzv4WaIJRL+EELLxiTvlIgpO4ghazkdhqyxFJMoiskSMtyXva7EuM5o5CUjjpoZoVl2FDgdo189a4shAko3XaAcokkzjiOJ5ctUOBJ7aazZrVgBSTKuZJw8e56KU3uBvBBwa46hII8WYKfhHBul9CnrVR5v7PS5i4J7iBU0rd1+lIqf3VoSVQloQc5+rJq2neN4m0Et3cx3/H6P1Bop5oegs1fSqqxVfprcCEgTYeHJ/VaApNMsN9hOwsJgdXBej6eLPhz6Wo+1A46TvNdR8F62FQL/mjXS1cRmVwQ2XqLptBMwomd/ztpzj974LX5rmFYOIEQ6ewHTQpeuyR829rCyf5e8mBv/9eHW4+/n797t6Ivo6+i7SiJHkWPo5+io+g4ItGb6F30T/Rv73Xvj96fvb9m1OvX5jpfRCuj9+4/+/8CsA==</latexit>
P (yi = 1|xi) =exp(xT
i w)
1 + exp(xTi w)
<latexit sha1_base64="30xsw632vjL+Lqp1+IPVHH/WWSI=">AAAIBHicfVXNbtw2EFbSNJu4f05zzEWIE8BuXGO1CZBcAiRxULQHo24ROwZMW6C41IpZ/igkFe+G1jXP0Iforeilh1zbV+jbdCjvNruU1jpIQ37ffMOZIams5MzYfv/fK1c/u/b59d6Nm2tffPnV19+s3/r20KhKE3pAFFf6KMOGcibpgWWW06NSUywyTl9n412Pv35HtWFKvrLTkp4IPJIsZwRbmErXB/ub05Q9Tc4nKduKn8Yo15g4RCflJsycvjrbql3yYGmcrm/0d/rNE7eNZGZsRLNnP711/S80VKQSVFrCsTHHSb+0Jw5rywin9RqqDC0xGeMRdVgYMxVZHd8X2BYmxPxkJzZpalGvrd1fmD2ubP7kxDFZVpZKAo6A5RWPrYp9NeIh05RYPgUDE81gPTEpMJTAQs2WpFym1NjizEAIJOkZUUJgOXSIGFMfJycOZXTEpCPQDFNvJIgCeDGolx2ULag+Y4bWDlk6sS7+NBOKcyZgF4CGQ5qNCou1Vmd1yMnnSj/lcRiNg7b8P5IfxK0oBeZeoml9UrtBqKGxj4/lqGnWkrpHeBdiLCBFpiYuRtto21TZG6g0FN6PmiUs85kcgoPvK8HcHYVLYPJd7U7d90kIWOaTw1UXppRdSMuzWt6fGA3czn1c+rjjBy3xsWiAjqglaJ5CAbAO60gxN7Mss8z9GnqCUS7guy28YEz6ToGRut0UqpLbacgSCzmJrpSgLIt1GXQVxnPGISkdd9DMEsuwkcDtHvnuXdoIE1C6dYByjiTOOI4n5y1Q4EntpogMlY3nrIBEOVcSDt49b6XJvQDeLdhlm1CQ5wvw8xDO7QL6Q6tUPu5Fa3OXBGcQOulvcPpWKn90aElUJeEOcvRt1dzUNYo3EZze+XjL+7+kcJ1qugeaP5dUY6v0dxBCQJn29g7rlQQmmWDvYTlzC4HqcDUfT+b8mXU5H3oHN07zXkXBetR0C75o21uXEZmcE9lqRVNoJmHHzr6dNIdKrSBq817BsLADIdOLD1xS8LdLwn9b2zgc7CQPdwa/PNp49mL237sR3YnuRptREj2OnkU/RvvRQUSi36KP0d/RP70Pvd97f/T+vKBevTLzuR0tPb2P/wHrBPmj</latexit>
EXAMPLE: LOGISTIC REGRESSION
P (parameters|data) / P (data|parameters)P (parameters)
prior
logP (w) = �|w|Example:
P (w|y,X) =
Y
i
exp(yi · xTi w)
1 + exp(yi · xTi w)
!P (w)
<latexit sha1_base64="004+ZLNp5XNu9cMH76QXus7GxG4=">AAAIK3icfVXdchs1FN4WqEv4S+GSmx3Sztg0ZLyhM+WGmZYwDFxkCEyThomSHa2s9QrrZytpaxtlH4kX4IaH4AqG297CK3C0tqmtXUcXu0f6vvMd6Rz9ZCVnxg6Hf966/cabb93p3X17551333v/g917H54ZVWlCT4niSp9n2FDOJD21zHJ6XmqKRcbp82xy5PHnL6k2TMlndl7SS4HHkuWMYAtD6e5PJ/3p9Xz/fBB/iTjNbT9GpVajlKFcY+IQnZX9OfTISNl4lrKrZ9NB7ZKH3QDSbFzYQQyig3R3b3gwbFrcNpKlsRct20l6785vaKRIJai0hGNjLpJhaS8d1pYRTusdVBlaYjLBY+qwMGYusjp+ILAtTIj5wU5s1qSs3tl5sDZ6Udn8i0vHZFlZKgk4ApZXPLYq9kmLR0xTYvkcDEw0g/nEpMCQIAup3ZBymVITizMDIZCkU6KEwHLkEDGmvkguHcromElHoGam3ksQBXDRqTcdlC2onjJDa4csnVkXvx4JxTkTsFlAwy0qgLVW0zrk5Cul7/I4jMZBW/4fyXfiVpQCcy/RbIykdoehhsY+Ppbjplgb6h7hXYixgBSZmrkY7aN9U2U/Q6Yh8b7XTGGTz+QIHHxdCebuPJwCky9rd+U+S0LAMr84XHVhStm1ZXlWy/s1o4Hba5+UPu7kYUt8IhqgI2oJmleQAKzDPFLMzXKVWeZ+DD3BKNfwoxZeMCZ9pcBI3VEKWcntPGSJtTWJriVBWtbzctiVGM+ZhKR00kEzGyzDxgK3a+Srd2MhTEDp1gHKNZI44zieXbdAgWe1my8urhUrIFHOlYSDd99baXI/gI8KdtMmFOTpGvw0hHO7hn7TSpWPuyht7pLgDEIl/UVPX0jljw4tiaok3EGOvqiaC71GcR/B6V31B97/awrXqabHoPl9STW2Sn8KIQSk6fj4rN5KYJIJ9gtMZ2UhUB1t5+PZir+0buZD7eDGab7bKFiPm2rBH+176yYikysi265oCs0k7Njlv5Pm/BMIUZvvFoaFHQgrXfzgkoLXLgnftrZxdniQfH5w+MOjvSdfLd+9u9HH0SdRP0qix9GT6NvoJDqNSPR79Cr6J/q392vvj95fvb8X1Nu3lj4fRRut9+o/62oJbg==</latexit>
logP (w|y,X) = logP (w) +X
i
� log(1 + exp(�yi · xTi w))
<latexit sha1_base64="3Y80qniIrti7yfwt00Tirz2HpIc=">AAAIFXicfVXdjhs1FJ620JTlbwuX3FhsKyXsjzILEr1BaglCcLFiQd3tSuvdkcfxZEz8M7U9TYJ3noNn4BWQuEPc9pa+DceThCYzyc7FzLG/z9/xOcc+kxaCW9fvv7lz9947797vPHhv5/0PPvzo492Hn5xbXRrKzqgW2lykxDLBFTtz3Al2URhGZCrYi3Q8CPiLV8xYrtVzNyvYlSQjxTNOiYOpZHeAhR6h0+7kZnZw0UPfoOW4h/YRtqVMODqsJ7vxPmbTons4SzimQ+3QNOHXzye9XrK71z/q1w9qG/HC2IsWz2ny8P6feKhpKZlyVBBrL+N+4a48MY5TwaodXFpWEDomI+aJtHYm0wo9lsTltomFyY3YtE5OtbPzeGX2snTZkyvPVVE6pigsBCwrBXIahfSgITeMOjEDg1DDYT+I5sQQ6iCJa1I+1XrsSGrBBVZsQrWURA09ptZWl/GVxykbceUpVMdWezFmAM4H1foC7XJmJtyyymPHps6jtzNNccElHAvQ8NjwUe6IMXpSNTnZUunHDDW9CdBW/3sKA9TykhMRJDKI28eVP25qGBL8EzWqi7WmHhCxCbEOkDzVU4/wAT6wZforZBoSH0b1Ftb5XA1hQagrJcJfNLfA1avKX/vDuAk4HoIj5SZMa7cSVmC1Vr9l1HA79nER/I73W+JjWQMbvBageQ0JIKaZR0aEXUSZpv6X5kowihV80MJzzlWoFBiJHySQlczNmiy5EpPcFBKkZTUvx5sSEzjjJikZb6DZNZblI0naNQrVu7UQtkHZrAOUG6xIKgia3rRASaaVn82b1ZLVIDEhtIKL9yhYSfyoAQ9yftshlPTZCvysCWduBf2+largd17azMeNOwiVDC2dvVQ6XB1WUF0q6EGevSzr1l1h1MVwe5fjXlj/HYN2atgJaP5UMEOcNl+ACwlpOjk5r7YSuOKS/wbbWVoYVIfb+WS65C+s2/lQO+g49XsbhZhRXS344oNg3Ubkaknk2xVtbriCE7v4bqR5XBgNXuv3FoaDEwiRzj/QpOBvFzf/bW3j/Pgo/vLo+Oev9p5+u/jvPYg+iz6PulEcfR09jX6ITqOziEZ/RK+jf6M3nd87f3X+7vwzp969s1jzabT2dF7/B9Db/e8=</latexit>
BAYESIANS OFFER NO SOLUTIONS
P (w|D, l)
solution space
optimizationsolution
Monte Carlo methods randomly sample this solution space
BayesLagrange
WHY MONTE CARLO?You need to know more than just a max/minimizer
optimizationsolution
“error bars”
Var(w) = E[(w � µ)(w � µ)T ] =
ZwwTP (w) dw � µµT
E(w) =
ZwP (w) dw
EXAMPLE: POKER BOTSYou can’t solve your problem by other (faster) methods
maximize logP (w) we don’t know derivative
poker bots
model parameters describe player behavior
2.5M different community cards
10 players = 59K raise/fold/call perms per round
WHY MONTE CARLO?You’re incredibly lazymaximize logP (w)
I could differentiate this,I just don’t wanna
argmax logP (w) ⇡ maxk
{P (wk)}
MARKOV CHAINS
Irreducible: can visit any state starting at any state
Aperiodic: does not get trapped in deterministic cycles
MC has steady state if
What is an MC?
METROPOLIS HASTINGSingredients
q(y|x)
proposal distribution
p(x)
posterior distribution
MH AlgorithmStart with x0
For k = 1, 2, 3, . . .Choose candidate y from q(y|xk)Compute acceptance probability
↵ = min
⇢1,
p(y)q(xk|y)p(xk)q(y|xk)
�
Set xk+1 = y with probability ↵otherwise xk+1 = xk
CONVERGENCE
Theorem
Suppose the support of q contains the support of p. Thenthe MH sampler has a stationary distribution, and the
distribution is equal to p.
Irreducible: support of q must contain support of p
Aperiodic: there must be positive rejection probability states
METROPOLIS ALGORITHMingredients
q(y|x)
proposal distribution
p(x)
posterior distribution
Assume proposal is symmetricq(xk|y) = q(y|xk)
↵ = min
⇢1,
p(y)q(xk|y)p(xk)q(y|xk)
�↵ = min
⇢1,
p(y)
p(xk)
�
Metropolis Hastings Metropolis
EXAMPLE: GMMhistogram of Metropolis iterates
Andrieu, Freitas, Doucet, Jordan ‘03
PROPERTIES OF MH
We don’t need a normalization constant for posteriorWe can run many chains in parallel (cluster/GPU)We don’t need any derivatives
Pro’s
P (parameters|data) = P (data|parameters)P (parameters)
P (data)
P (parameters|data) / P (data|parameters)P (parameters)
PROPERTIES OF MH
“Mixing time” depends on proposal distribution• too wide = constant rejections = slow mixing• too narrow = short movements=slow mixing
Con’s
Samples only meaningful at stationary distribution • “Burn in” samples must be discarded• Many samples needed because of correlations
Posterior
proposal
This is bad…
xk<latexit sha1_base64="8tY7aDdJ2IAPpvSJDyWI10/Am7g=">AAAH03icfVVLb9w2EJbTNuu4rzg55iLUMeC2jrFyAqRHOw6K9mDUfdgxYNoLikutWPEhk5S9G1qXotdc2//Sf9J/06G8W+9SWusgDfl98w1nhqTSkjNj+/1/Vx589PEnD3urj9Y+/ezzL758vP7kxKhKE3pMFFf6NMWGcibpsWWW09NSUyxSTt+lxYHH311RbZiSv9lJSc8FHkmWMYItTP06vigGjzf6O/3midtGMjU2oulzNFh/+A8aKlIJKi3h2JizpF/ac4e1ZYTTeg1VhpaYFHhEHRbGTERax5sC29yEmJ/sxMZNavXa2ubc7Flls+/OHZNlZakk4AhYVvHYqtgnFw+ZpsTyCRiYaAbriUmONSYWSrAg5VKlCotTAyGQpNdECYHl0CFiTH2WnDuU0hGTjkBtTb2RIArg7aBedFA2p/qaGVo7ZOnYuvhuJhTnTEBTQcMhzUa5xVqr6zrkZDOlH7M4jMZBW/4fyQ/iVpQccy+RQd4uqd1uqKGxj4/lqGnWgrpHeBdiLCB5qsYuRtto21Tp71BpKLwfNUtY5DM5BAffV4K5Ow2XwORV7S7ciyQELPPJ4aoLU8rOpeVZLe87RgO3cy9KH7f4tiVeiAboiFqC5gUUAOuwjhRzM80yTd0voScY5Rx+0MJzxqTvFBgDdzCAqmR2ErLEXE6iKyUoy3xddrsK4zlFSBoUHTSzwDJsJHC7R7579zbCBJRuHaDcIIlTjuPxTQsUeFy7CSJDZeMZKyBRzpWEg/fcW4PkeQAf5Oy+TSjI/hy8H8KZnUO/b5XKx71tbeaS4AxCJ/2FTC+l8keHlkRVEu4gRy+r5uKtUbyF4PTOxl97/7cUrlNND0Hzp5JqbJX+BkIIKNPh4Um9lMAkE+w9LGdmIVAdLufj8Yw/te7nQ+/gxmneyyhYj5puwRdte+s+IpMzIluuaHLNJOzY6beT5lCpFURt3ksYFnYgZHr7gUsK/nZJ+G9rGye7O8nLnd2fX23svZn+91ajZ9FX0VaURK+jveiH6Cg6jkg0ij5Ef0V/9457rvdH789b6oOVqc/TaOHpffgP+CLnSg==</latexit>
SIMULATED ANNEALINGmaximize p(x)
Easy choice: run MCMC, thenmax
kp(xk)
Better choice: sample the distribution
p1
Tk (xk) “temperature”
Why is this better?Where does the name come from?
COOLING SCHEDULE
Andrieu, Freitas, Doucet, Jordan ‘03
hot warm
cool cold
CONVERGENCE
Granville, "Simulated annealing: A proof of convergence”
TheoremSuppose the MCMC mixes fast enough that epsilon-dense sampling occurs in finite time starting at every temperature. For an annealing schedule with temperature
simulated annealing converges to a global optima with probability 1.
Tk =1
C log(k + T0)
SA solves non-convex problem, even NP-complete problems,as time goes to infinity.
WHEN TO USE SIMULATED ANNEALING
flow chart
Should I use simulated annealing?
no.
No practical way to choose temperature scheduleToo fast = stuck in local minimum (risky)
Too slow = no different from MCMCAct of desperation!
GIBBS SAMPLERWant to sample
P (x1, x2, x3)
on stage k, pick some coordinate j
q(y|xk) =
(p(yj |xk
jc), if yjc = xkjc
0, otherwise
↵ =p(y)q(xk|y)p(xk)q(y|xk)
=p(y)p(xk
j |xkjc)
p(xk)p(yj |yjc)
=p(yjc)
p(xkjc)
= 1
P (B) =P (A and B)
P (A|B)
p(y) = p(yj and yjc)<latexit sha1_base64="KTec+pwpBDIxNpH0RsoPrN7Tsvc=">AAACI3icbVDLSgMxFM3Ud31VXboJFkE3ZaYKuhFENy4r2Fpoa8mkd9poJjMkd8RhmA/wQ1y71W9wJ25c+AH+hWntwqoHQg7nnMtNjh9LYdB1353C1PTM7Nz8QnFxaXlltbS23jBRojnUeSQj3fSZASkU1FGghGasgYW+hEv/5nToX96CNiJSF5jG0AlZX4lAcIZW6pbK8U66S4+ovbrXtI1whxllqkdzmnaz6yue71KbcivuCPQv8cakTMaodUuf7V7EkxAUcsmMaXlujJ2MaRRcQl5sJwZixm9YH1qWKhaC6WSjz+R02yo9GkTaHoV0pP6cyFhoTBr6NhkyHJjf3lD8z2slGBx2MqHiBEHx70VBIilGdNgM7QkNHGVqCeNa2LdSPmCacbT9FYsTe5TgEFgrt914v5v4SxrVirdXqZ7vl49Pxi3Nk02yRXaIRw7IMTkjNVInnNyTR/JEnp0H58V5dd6+owVnPLNBJuB8fAH9lqL0</latexit>
GIBBS SAMPLERWant to sample
P (x1, x2, x3)
iterates
x2 ⇠ P (x1|x12, x
13, x
14, . . . , x
1n)
x3 ⇠ P (x2|x21, x
23, x
24, . . . , x
2n)
x4 ⇠ P (x3|x31, x
32, x
34, . . . , x
3n)
· · ·
APPLICATION: SAMPLING GRAPHICAL MODELS
Restricted Boltzmann Machine (RBM)
E(v, h) = �aT v � bTh� vTWh
P (v, h) =1
Ze�E(v,h) “partition
function”
weights
binaryrandomvariables
0/1
APPLICATION: SAMPLING GRAPHICAL MODELS
Restricted Boltzmann Machine (RBM)
E(v, h) = �aT v � bTh� vTWh P (v, h) =1
Ze�E(v,h)
remove normalization
aggregate constant
cancel constants
sigmoid function
P (vi = 1|h) = P (vi = 1|h)P (vi = 0|h) + P (vi = 1|h)
=exp(�ai +
Pj wijhj + C)
exp(C) + exp(�ai +P
j wijhj + C)
=exp(�ai +
Pj wijhj)
1 + exp(�ai +P
j wijhj)
= �(�ai +X
i
wijhj)
BLOCK GIBBS FOR RBM
P (vi = 1|h) = �(�ai +X
j
wijhj)
hiddenvisible
freeze hiddenrandomly sample visible
stage 1
freeze visiblerandomly sample hidden
stage 2
P (hj = 1|v) = �(�bj +X
i
wijvi)
DEEP BELIEF NETSvisible
hidden
DBN = layered RBM
Each layer only dependson layer beneath it
(feed forward)
Probability for each hidden node is sigmoid function
EXAMPLE: MNIST
Train 3-layer DBN with 200 hidden units
pre-training: layer-by-layer training
training: train all weights simultaneously
Gan, Henao, Carlson, Carin ‘15
trained on 60K MNIST digits
training done using Gibbs sampler
Gibbs sampler used to explore final solution
EXAMPLE: MNISTtraining data Learned features
observations sampled from deep belief network using Gibbs sampler
WHAT’S WRONG WITH GIBBS?
susceptible to strong correlationsalso a problem for MH: bad proposal distribution
SLICE SAMPLERProblems with MH:
hard to choose proposal distributiondoes not exploit analytical information about problem
long mixing timeSlice sampler
q(x, u) =
(1, if 0 u p(x)
0, otherwise
1 1
x
p(x)
x
u
“LIFTED INTEGRAL”
1 1
x
p(x)
x
u
p(x) =
Zq(x, u) du
Zf(x)p(x) =
Z Zf(x)q(x, u) du dx
sample this
Problems with MH:hard to choose proposal distribution
does not exploit analytical information about problemlong mixing timeSlice sampler
SLICE SAMPLERZ
f(x)p(x) =
Z Zf(x)q(x, u) du dx
do Gibbs on thisFreeze x :
Choose u uniformly from [0, p(x)]Freeze u :
Choose x uniformly from {x|p(x) � u}need analytical formula
for this set
p(x)u
MODEL FITTING
q(x, u1, u2, . . . , un) =
(1, if pi(x) ui 8i0, otherwise
Zq(x, u1, u2, . . . , un) du =
Z p1(x)
u1=0
Z p2(x)
u2=0· · ·
Z pn(x)
un=0du =
nY
k=1
pi(x)
log p(x) =nX
i=1
log pi(x)p(x) =nY
i=1
pi(x)
Zf(x)p(x) dx =
Z
x
Z
uf(x)q(x, u) dx du
support is a box
MODEL FITTINGZ
f(x)p(x) dx =
Z
x
Z
uf(x)q(x, u) dx du
Freeze x :For all i, choose ui uniformly from [0, pi(x)]
Freeze u :Choose x uniformly from
Ti{x|pi(x) � ui}
q(x, u1, u2, . . . , un) =
(1, if pi(x) ui 8i0, otherwise
SLICE SAMPLER PROPERTIESpros
• does not require proposal distribution• mixes super fast• simple implementation
cons• analytical formula for super-level sets• hard in multiple dimensions
fixes / generalizations• step-out methods• random hyper-rectangles
STEP-OUT METHODSp(x)
u
p(x)u
p(x)u
•pick small interval
•double width size until slice is contained
•sample uniformly from slice using rejection sampling - throw away selections outside of slice
HIGH DIMS: COORDINATE METHOD
Zf(x)p(x) =
Z Zf(x)q(x, u) du dx
do coordinate Gibbs on this
Freeze x :Choose u uniformly from [0, p(x)]
Freeze u :For i = 1 · · ·n
Choose xi uniformly from{xi|p(x1, · · · , xi · · · , xn) � u}
can use stepping -out methodRadford Neal: “Slice Sampling” ‘03
HIGH DIMS: HYPER-RECTANGLE
Radford Neal: “Slice Sampling” ‘03
•choose “large” rectangle
•randomly/uniformly place rectangle around current location
•draw random proposal point from rectangle
•if proposal lies outside slice, shrink box so proposal is on boundary
performs much better for poorly-conditioned objectives
COMPARISON
Gradient/Splitting
SGD
MCMC
rate
e�ck
1/k
1/pk
when to use
you value reliability and precision(moderate speed, high accuracy)
you value speed over accuracy(high speed, moderate accuracy)
you value simplicity (no gradient)or need statistical inference
(slow and inaccurate)
DO MATLAB EXERCISEMCMC
Recommended