ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: December 30, 2017 Homework: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). https://piazza.com/caltech/winter2018/acm216/home Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (5 problem sets, one every two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: • Florian T. Schäfer: o Office hour: Wednesday 4:30-5:30pm o Location: Annenberg 106 o Email: florian.schaefer@caltech.edu • Robin Brown: o Office hour: Monday 8-9pm o Location: Annenberg 107 o Email: rabrown@caltech.edu • Anna Winnicki: o Office hour: Tuesday 3-4pm o Location: Annenberg o Email: awinnick@caltech.edu Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. • Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/ • Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. • Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud). • Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).