CMS/ACM/EE 117 Probability and Random Processes

Last Update: October 31, 2017 Homeworks: Posted on Piazza Lecture Notes: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Prerequisite: A first course in probability theory and some elementary linear algebra. Piazza: For all class-related discussions (in particular for Q/A). https://piazza.com/caltech/fall2017/cmsacmee117/home Instructor: Houman Owhadi Office hour: Tue/Thu 10:30 am-11 am, Steele House 201. TA: • Max Budninskiy : o Office hour: Tuesday 5-6pm o Location: Annenberg 314 o Email: max.budninskiy@gmail.com • Serin Hong : o Office hour: Wednesday 3-4pm o Location: Annenberg 106 o Email: shong2@caltech.edu Schedule Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 213 Annenberg. Grading: Homework (8 problem sets, one every week): 100% Syllabus: • Probability spaces and $\sigma $-algebras. • Conditioning and Independence. • Bayes formula. • Continuous and Discrete Random Variables. o Expected value, variance, moment generating functions, distribution functions, density functions, characteristic functions, Chebyshev's inequality. o The Bernoulli Random Variable. The Binomial Random Variable. The Geometric random variable. Negative Binomial random variable. The Poisson random variable. Uniform random variable. Exponential random variable. Gamma random variable. m-Erlang random variable. Gaussian/Normal random variable. Cauchy random variable. • Branching (Galton-Watson) Processes. • Poisson (Point) Processes. • Limit theorems o Weak and strong convergence, convergence in Probability. Borel Cantelli lemma. Application to convergence in probability. Strong Law of Large Numbers. Glivenko-Cantelli. o Gaussian variables and the central limit theorem. Central Limit Theorem. Z-scores/values. Confidence intervals/bounds. o Hoeffding's and McDiarmid's concentration inequalities. • Revisiting Statistics o Testing the fit of a distribution to data. o Chi-Square random variable with $k$ degrees of freedom. o Chi-Square Test. o Sample mean and variance. o Student-t distribution with n degrees of freedom. o Elements of game theory. Von Neumann's minimax theorem. Connection with decision theory. • Gaussian vectors. o Gaussian vectors in R d . o Central limit Theorem in R d . o Gaussian vectors in an Euclindean space. • Conditional Expectation. o Conditional Expectation with respect to an event. o Conditional Expectation with respect to random variable. o Conditional Expectation with respect to sigma-algebra. o Conditional Expectation as a Least Square Projection. o Gaussian spaces, Gaussian processes and Gaussian conditioning. o Brownian Motion as a Gaussian Process. • Gaussian measures. o Existence and construction of Gaussian Measures. o Brownian Measure from a Gaussian Measure. o Construction of the B.M. on $[0,1]$ from a Gaussian Measure. o Wiener Integral with Respect to a Brownian Motion. o Sample Path Properties of the Brownian Motion. o Stationary processes. • Gaussian fields (on Banach spaces). • Elements of martingale theory (definition, examples, martingale convergence theorem) • Gaussian regression with Gaussian fields. • Numerical approximation, kernel methods and Gaussian learning. Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions • Probability and Random processes (G. R. Grimmett and D. R. Stirzaker). The most comprehensive (does not contain everything you will see in CMS/ACM 117 but if you need a textbook you should get that one). • CM/ACM 117 is a graduate level class. The following books could be helpful as remedial books. o A second course in probability theory (Sheldon M. Ross and Erol A. Pekoz) o Chapter 2 of Lawrence C. Evans lecture notes, available at http://math.berkeley.edu/~evans/SDE.course.pdf, for the first classes. o Introduction to probability models (Sheldon M. Ross). o Probability and random processes for electrical and computer engineers (John A. Gunber) o Probability and random processes for electrical engineering (Alberto Leon-Garcia). o Introduction to probability (Dimitri P. Bertsekas). Elementary but helpful if you are struggling with basic concepts. o Introduction to probability (Charles M. Grinstead and J. Laurie Snell). Elementary but helpful if you are struggling with basic concepts.