ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: January 2, 2019 Homework: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). https://piazza.com/caltech/winter2019/acm216/home Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (5 problem sets, one every two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: • Andrew Bai: o Office hour: Wednesday 4-5pm o Location: o Email: abai@caltech.edu • Corina Panda: o Office hour: Monday 3-4pm o Location: Annenberg 106 o Email: cpanda@caltech.edu Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. • Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/ • Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. • Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud). • Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).