Beyond Convexity: Submodularity in Machine Learning

                 	  Andreas Krause
   Center for the Mathematics of Information
       California Institute of Technology

ABSTRACT:  Convex optimization has become a main workhorse for many machine
learning algorithms during the past ten years. When minimizing a convex loss
function for, e.g., training a Support Vector Machine, we can rest assured
to efficiently find an optimal solution, even for large problems. In recent
years, another fundamental problem structure, which has similar beneficial
properties, has emerged as very useful in a variety of machine learning
applications:
Submodularity is an intuitive diminishing returns property, stating that
adding an element to a smaller set helps more than adding it to a larger
set. Similarly to convexity, submodularity allows one to efficiently find
provably (near-)optimal solutions. I will give an introduction to the
concept of submodularity, discuss algorithms for minimizing and maximizing
submodular functions and - as the main focus - illustrate their usefulness
in solving difficult machine learning problems, such as active learning and
sparse experimental design, structure learning, clustering, influence
maximization and ranking.

This talk is based on a tutorial that was presented at ICML 2008.