CMS/ACM/EE 117 Probability and Random Processes

Last Update: October 31, 2017 Homeworks: Posted on Piazza Lecture Notes: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list). Prerequisite: A first course in probability theory and some elementary linear algebra. Piazza: For all class-related discussions (in particular for Q/A). https://piazza.com/caltech/fall2017/cmsacmee117/home Instructor: Houman Owhadi Office hour: Tue/Thu 10:30 am-11 am, Steele House 201. TA: Max Budninskiy : o Office hour:  Tuesday 5-6pm o Location: Annenberg 314 o Phone: 626 233 2657 o Email: max.budninskiy@gmail.com Serin Hong : o Office hour:  Wednesday 3-4pm o Location: Annenberg 106 o Phone: 626 818 7351 o Email: shong2@caltech.edu Schedule Classes are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 213 Annenberg. Grading: Homework (8 problem sets, one every week): 100% Syllabus: Probability spaces and $\sigma $-algebras. Conditioning and Independence. Bayes formula. Continuous and Discrete Random Variables. o Expected value, variance, moment generating functions, distribution functions, density functions, characteristic functions, Chebyshev's inequality. o The Bernoulli Random Variable. The Binomial Random Variable. The Geometric random variable. Negative Binomial random variable. The Poisson random variable. Uniform random variable. Exponential random variable. Gamma random variable. m-Erlang random variable. Gaussian/Normal random variable. Cauchy random variable. Branching (Galton-Watson) Processes. Poisson (Point) Processes. Limit theorems o Weak and strong convergence, convergence in Probability. Borel Cantelli lemma. Application to convergence in probability. Strong Law of Large Numbers. Glivenko-Cantelli. o Gaussian variables and the central limit theorem. Central Limit Theorem. Z-scores/values. Confidence intervals/bounds. o Hoeffding's and McDiarmid's concentration inequalities. Revisiting Statistics o  Testing the fit of a distribution to data. o Chi-Square random variable with $k$ degrees of freedom. o Chi-Square Test. o Sample mean and variance. o Student-t distribution with n degrees of freedom. o Elements of game theory. Von Neumann's minimax theorem. Connection with decision theory. Gaussian vectors. o Gaussian vectors in R d . o Central limit Theorem in R d . o Gaussian vectors in an Euclindean space. Conditional Expectation. o  Conditional Expectation with respect to an event. o Conditional Expectation with respect to random variable. o Conditional Expectation with respect to sigma-algebra. o Conditional Expectation as a Least Square Projection. o Gaussian spaces, Gaussian processes and Gaussian conditioning. o Brownian Motion as a Gaussian Process. Gaussian measures. o Existence and construction of Gaussian Measures. o Brownian Measure from a Gaussian Measure. o Construction of the B.M. on $[0,1]$ from a Gaussian Measure. o Wiener Integral with Respect to a Brownian Motion. o Sample Path Properties of the Brownian Motion. o Stationary processes. Gaussian fields (on Banach spaces). Elements of martingale theory (definition, examples, martingale convergence theorem) Gaussian regression with Gaussian fields. Numerical approximation, kernel methods and Gaussian learning. Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions  Probability and Random processes (G. R. Grimmett and D. R. Stirzaker). The most comprehensive (does not contain everything you will see in CMS/ACM 117 but if you need a textbook you should get that one). CM/ACM 117  is a graduate level class. The following books could be helpful as remedial books. o  A second course in probability theory (Sheldon M. Ross and Erol A. Pekoz) o Chapter 2 of Lawrence C. Evans lecture notes, available at http://math.berkeley.edu/~evans/SDE.course.pdf, for the first classes. o Introduction to probability models (Sheldon M. Ross). o Probability and random processes for electrical and computer engineers (John A. Gunber) o Probability and random processes for electrical engineering (Alberto Leon-Garcia). o  Introduction to probability (Dimitri P. Bertsekas). Elementary but helpful if you are struggling with basic concepts. o Introduction to probability (Charles M. Grinstead and J. Laurie Snell). Elementary but helpful if you are struggling with basic concepts.