ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: December 30, 2017 Homework: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). https://piazza.com/caltech/winter2018/acm216/home Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (5 problem sets, one every two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: Florian T. Schäfer: o Office hour:  Wednesday 4:30-5:30pm o Location: Annenberg 106 o Phone: 626 365 7384 o Email: florian.schaefer@caltech.edu Robin Brown: o Office hour:  Monday 8-9pm o Location: Annenberg 107 o Phone: 301 5266178 o Email: rabrown@caltech.edu Anna Winnicki: o Office hour:  Tuesday 3-4pm o Location: Annenberg o Phone: 808-343-7200 o Email: awinnick@caltech.edu Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/ Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud).