Last Update: October 31, 2017Homeworks: Posted on PiazzaLecture Notes: Sent via email to registered students (drop the instructor an email if you are not registered yet to be added to the email list).Prerequisite: A first course in probability theory and some elementary linear algebra.Piazza: For all class-related discussions (in particular for Q/A).https://piazza.com/caltech/fall2017/cmsacmee117/homeInstructor: Houman OwhadiOffice hour: Tue/Thu 10:30 am-11 am, Steele House 201.TA:•Max Budninskiy :oOffice hour: Tuesday 5-6pmoLocation: Annenberg 314oPhone: 626 233 2657oEmail: firstname.lastname@example.org•Serin Hong :oOffice hour: Wednesday 3-4pmoLocation: Annenberg 106oPhone: 626 818 7351oEmail: email@example.comScheduleClasses are scheduled from 9:00am to 10:25am on Tuesdays and Thursdays in 213 Annenberg.Grading:Homework (8 problem sets, one every week): 100%Syllabus:•Probability spaces and $\sigma $-algebras.•Conditioning and Independence.•Bayes formula.•Continuous and Discrete Random Variables.oExpected value, variance, moment generating functions, distribution functions, density functions, characteristic functions, Chebyshev's inequality.oThe Bernoulli Random Variable. The Binomial Random Variable. The Geometric random variable. Negative Binomial random variable. The Poisson random variable. Uniform random variable. Exponential random variable. Gamma random variable. m-Erlang random variable. Gaussian/Normal random variable. Cauchy random variable.•Branching (Galton-Watson) Processes.•Poisson (Point) Processes.•Limit theoremsoWeak and strong convergence, convergence in Probability. Borel Cantelli lemma. Application to convergence in probability. Strong Law of Large Numbers. Glivenko-Cantelli. oGaussian variables and the central limit theorem. Central Limit Theorem. Z-scores/values. Confidence intervals/bounds. oHoeffding's and McDiarmid's concentration inequalities.•Revisiting Statisticso Testing the fit of a distribution to data.oChi-Square random variable with $k$ degrees of freedom.oChi-Square Test.oSample mean and variance.oStudent-t distribution with n degrees of freedom.oElements of game theory. Von Neumann's minimax theorem. Connection with decision theory.•Gaussian vectors.oGaussian vectors in Rd.oCentral limit Theorem in Rd.oGaussian vectors in an Euclindean space.•Conditional Expectation.o Conditional Expectation with respect to an event.oConditional Expectation with respect to random variable.oConditional Expectation with respect to sigma-algebra.oConditional Expectation as a Least Square Projection.oGaussian spaces, Gaussian processes and Gaussian conditioning.oBrownian Motion as a Gaussian Process.•Gaussian measures.oExistence and construction of Gaussian Measures.oBrownian Measure from a Gaussian Measure.oConstruction of the B.M. on $[0,1]$ from a Gaussian Measure.oWiener Integral with Respect to a Brownian Motion.oSample Path Properties of the Brownian Motion.oStationary processes.•Gaussian fields (on Banach spaces).•Elements of martingale theory (definition, examples, martingale convergence theorem)•Gaussian regression with Gaussian fields.•Numerical approximation, kernel methods and Gaussian learning.Textbooks: The lectures will not follow closely any of those textbooks (I will distribute my lecture notes), they are given here only as suggestions• Probability and Random processes (G. R. Grimmett and D. R. Stirzaker).The most comprehensive (does not contain everything you will see in CMS/ACM 117 but if you need a textbook you should get that one).•CM/ACM 117 is a graduate level class. The following books could be helpful as remedial books.o A second course in probability theory (Sheldon M. Ross and Erol A. Pekoz) oChapter 2 of Lawrence C. Evans lecture notes, available at http://math.berkeley.edu/~evans/SDE.course.pdf, for the first classes.oIntroduction to probability models (Sheldon M. Ross).oProbability and random processes for electrical and computer engineers (John A. Gunber)oProbability and random processes for electrical engineering (Alberto Leon-Garcia).o Introduction to probability (Dimitri P. Bertsekas). Elementary but helpful if you are struggling with basic concepts.oIntroduction to probability (Charles M. Grinstead and J. Laurie Snell). Elementary but helpful if you are struggling with basic concepts.