ACM 216: Markov Chains, Discrete Stochastic Processes and Applications.
Last Update: July 18, 2017Homework: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list).•Homework 1: Handed out on Jan 4, due on Jan 16•Homework 2: Handed out on Jan 16, due on Jan 25•Homework 3: Handed out on Jan 25, due on Feb 8•Homework 4: Handed out on Feb 8, due on Feb 20•Homework 5: Handed out on Feb 20, due on Mar 6Lecture Notes : Sent via email to registered studentsPrerequisite: ACM/EE 116 or CMS/ACM/EE 117 or instructor agreementPiazza: for all class-related discussions (in particular for Q/A).Schedule: Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 105 Annenberg.Grading: Homework (4 problem sets, one two weeks): 100%Instructor: Houman OwhadiOffice hour: Tues/Thu 10:30am-11:00 am, Steele House, 201.TAs:Schedule: Classes are scheduled from 9am to 10:25am on Tuesdays and Thursdays in 105 Annenberg.Grading: Homework (4 problem sets): 100%Syllabus:Markov Chains.Computer simulation of Markov Chains.Irreducible and aperiodic Markov Chains.Stationary Distributions.Reversible Markov Chains.Markov Chain Monte Carlo.Fast Convergence of MCMC algortithms.Approximate counting.The Propp-Wilson algorithm.Sandwiching.Simulated annealing.Convergence rates.Continuous time Markov ChainsTextbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions.•Markov Chains and Stochastic Stability (S. P. Meyn and R. L. Tweedie). Well written and comprehensive. Can be downloaded from http://probability.ca/MT/•Finite Markov Chains and Algorithmic Applications (Olle Hagstrom). This thin and inexpensive book is a nice and up-to-date introduction to Markov Chain, algorithms and applications.•Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (P. Bremaud).•Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).