Title:  Statistical Learning III: A convex approach to semi-supervised and
unsupervised learning

                 	Ben Recht
   Center for the Mathematics of Information
       California Institute of Technology

ABSTRACT:  In this third talk, I will focus on my own research in 
statistical learning --building models that incorporate prior 
information to deal with incompletely specified data-sets.  Recovering 
latent states with short description length from high-dimensional 
observations is a ubiquitous problem in data analysis and mining.  When 
enough examples of the mapping from measurements to states are given, 
nonlinear regression techniques can learn any smooth mapping using a 
sufficiently general family of functions.  But these traditional methods 
often require too many input-output examples to be of practical use.  
Generative models, on the other hand, are powerful tools that allow one 
to search for the most likely latent state from measurements alone.  
Unfortunately, except in special cases, these methods typically require 
non-convex local search plagued by local minima.
 

I will present an optimization-driven approach to semi-supervised and 
unsupervised learning that augments standard methods in statistical 
regression with priors on the unlabelled data.  By formulating this 
learning problem as a concise set of goals and constraints, tools from 
convex optimization may be readily applied to fit approximate models.  
The resulting approximations are convex and globally optimal solutions 
can be computed efficiently along with diagnostic bounds on the quality 
of their solution.  Depending on the specifics of the prior, different 
estimation algorithms can be derived, and, with surprisingly few 
examples, relationships between various types of data can be discovered.
 

To illustrate the power of these methods, I will present a diverse set 
of applications in fields such as data segmentation, RFID tracking, and 
video processing.  In each of these examples, I will present empirical 
results demonstrating the performance of the new algorithms and draw 
connections to existing techniques in machine learning.