ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.

Last Update: July 18, 2017 Homework: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Homework 1: Handed out on Jan 4, due on Jan 16 Homework 2: Handed out on Jan 16, due on Jan 25 Homework 3: Handed out on Jan 25, due on Feb 8 Homework 4: Handed out on Feb 8, due on  Feb 20 Homework 5: Handed out on Feb 20, due on Mar 6 Lecture Notes : Sent via email to registered students Prerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreement Piazza: for all class-related discussions (in particular for Q/A). Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg. Grading: Homework (4 problem sets, one two weeks): 100% Instructor: Houman Owhadi Office hour: Tues/Thu 10:30am-11:00 am, Steele House, 201. TAs: Schedule: Classes are scheduled from 9am to 10:25am on Tuesdays and Thursdays in 105  Annenberg. Grading: Homework (4 problem sets): 100% Syllabus: Markov Chains. Computer simulation of Markov Chains. Irreducible and aperiodic Markov Chains. Stationary Distributions. Reversible Markov Chains. Markov Chain Monte Carlo. Fast Convergence of MCMC algortithms. Approximate counting. The Propp-Wilson algorithm. Sandwiching. Simulated annealing. Convergence rates. Continuous time Markov Chains Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions. Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications. Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud). Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).