ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: January 6, 2020 Homework: Posted on Piazza Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). https://piazza.com/caltech/winter2020/acm216/home Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (4 problem sets, one every two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: Anirudh Rangaswamy: o Office hour: Monday 7-8pm o Location: Annenberg 106 o Email: anirudh@caltech.edu Zhang, Xiaotian (Jim): o Office hour: Tuesday 3-4pm o Location: Annenberg 243 o Email: jim@caltech.edu Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/ Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud). Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).