Revisiting Log-Linear Learning

                 	Jason Marden
   Center for the Mathematics of Information
       California Institute of Technology

Abstract:  This talk focuses on the theory of learning in games.  We
consider the learning algorithm log-linear learning introduced by Blume in
1993. In potential games, log-linear learning provides guarantees on the
percentage of time that the joint action profile will be at a potential
maximizer. The traditional analysis of log-linear learning has centered
around explicitly computing the stationary distribution. This analysis
relied on a highly structured setting: i) players' utility functions
constitute an exact potential game, ii) players update their strategies one
at a time, which we refer to as asynchrony, iii) at any stage, a player can
select any action in the action set, which we refer to as completeness, and
iv) each player is endowed with the ability to assess the utility he would
have received for any alternative action provided that the actions of all
other players remain fixed. Since the appeal of log-linear learning is not
solely the explicit form of the stationary distribution, we seek to address
to what degree one can relax the structural assumptions while maintaining
that only potential function maximizers are the stochastically stable action
profiles.  The first part of this talk reviews the theory of resistance
trees for regular perturbed Markov decision processes introduced by Young in
1993.  This theory provides a methodology for evaluating the stochastically
stable states in any regular perturbed Markov decision process.  The second
part of this talk focuses on utilizing this theory to develop new
distributed learning algorithms with similar guarantees as log-linear
learning but with less stringent structural requirements.