ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: January 6, 2020 Homework: Posted on Piazza Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). https://piazza.com/caltech/winter2020/acm216/home Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (4 problem sets, one every two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: • Anirudh Rangaswamy: o Office hour: Monday 7-8pm o Location: Annenberg 106 o Email: anirudh@caltech.edu • Zhang, Xiaotian (Jim): o Office hour: Tuesday 3-4pm o Location: Annenberg 243 o Email: jim@caltech.edu Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. • Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/ • Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. • Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud). • Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).