Title: Markov Decision Processes
Nevin Kapur

Abstract: Markov Decision Processes (MDPs), also known as stochastic
dynamic programs or stochastic control programs are used to model
sequential decision-making in the presence of uncertainty.  The model
is equipped with "states," "actions," "rewards," and "transition
probabilities."  The choice of an action at each discrete time unit
generates a reward and a transition to the next state according to the
transition probabilities.  A "policy" is a prescription of actions
from each state at each decision-making epoch.  We seek optimal
policies, defined as those that maximize the total expected reward
over the decision-making timeline.

In this self-contained introductory talk, I will motivate and set up
MDPs, identify conditions under which optimal policies exist, and
present various algorithms (drawing on fixed-point ideas, dynamic
programming, and linear programming) for determining optimal policies.