cs189 lecture 1

8/9/2019 cs189 lecture 1

http://slidepdf.com/reader/full/cs189-lecture-1 1/113

CS189/CS289AIntroduction to Machine Learning

Lecture 1: Overview

Alexei Efros and Peter Bartlett

January 20, 2015

8/9/2019 cs189 lecture 1

Organizational Issues

8/9/2019 cs189 lecture 1

Instructors: Peter Bartlett and Alyosha Efros.

8/9/2019 cs189 lecture 1

GSIs: Peter Gao, Yun Park, Faraz Tavakoli, Kevin Tee, Pat Virtue,Christopher Xie, Daniel Xu, Yuchen Zhang.

8/9/2019 cs189 lecture 1

Discussion sections: You choose. If the room is full, please go toanother one. (If necessary, we may offer some specialtysections—watch website for announcements.)

8/9/2019 cs189 lecture 1

Office hours: see web site.

8/9/2019 cs189 lecture 1

Office hours: see web site.

http://www-inst.eecs.berkeley.edu/∼cs189

bCourses (+ piazza, kaggle), office hours, syllabus, assignments,readings, lecture slides, announcements.

8/9/2019 cs189 lecture 1

Assessment:CS189 Homework 40%

Implementation and application of methods. (Kaggle)

Mathematical/reinforcement of concepts.Seven total.Late policy: 5 slip days total. That’s it.

Midterm 20%(Thursday, March 19, in the lecture slot.)Final Exam 40%

8/9/2019 cs189 lecture 1

Assessment:

CS289A Plus a project:

Homework 40%Midterm 20%Final Exam 20%Final Project 20%(due Friday, May 1. Proposal due Friday, April 3.)

8/9/2019 cs189 lecture 1

(Real) Prerequisites:Math53 (vector calculus); Math54 (linear algebra); CS70 (discretemath, probability); CS188 (more probability, decision theory).

8/9/2019 cs189 lecture 1

No screens in lectures. (To see why, google “laptops in class.”)

8/9/2019 cs189 lecture 1

Ethics:

8/9/2019 cs189 lecture 1

Ethics:Discussion of homework problems with other students is encouraged.

8/9/2019 cs189 lecture 1

Ethics:Discussion of homework problems with other students is encouraged.All homeworks must be written individually (including programmingcomponents).

8/9/2019 cs189 lecture 1

Ethics:Discussion of homework problems with other students is encouraged.All homeworks must be written individually (including programmingcomponents).Please read the department policy on academic dishonesty. We will be

actively checking for plagiarism.

8/9/2019 cs189 lecture 1

Ethics:Discussion of homework problems with other students is encouraged.All homeworks must be written individually (including programmingcomponents).Please read the department policy on academic dishonesty. We will be

actively checking for plagiarism.Questions: Use piazza. Public and private.

8/9/2019 cs189 lecture 1

CS189 I d i M hi L i

8/9/2019 cs189 lecture 1

CS189: Introduction to Machine Learning

CS189 I d i M hi L i

8/9/2019 cs189 lecture 1

Machine Learning

Systems that learn to solveinformation processing problems.

CS189 I t d ti t M hi L i

8/9/2019 cs189 lecture 1

Machine Learning

LearnUse experience to improve performance:data, queries, interaction, experiments

Statistical issues are central.

CS189 I t d ti t M hi L i

8/9/2019 cs189 lecture 1

Machine Learning

LearnUse experience to improve performance:data, queries, interaction, experiments

Statistical issues are central.

Systems

Computational issues are also central.

Algorithms, optimization.

An Overview of Machine Learning

8/9/2019 cs189 lecture 1

1 Problems

8/9/2019 cs189 lecture 1

1 Problems

2 Methods3

8/9/2019 cs189 lecture 1

1 Problems

2 Methods3 Concepts

8/9/2019 cs189 lecture 1

1 Problems

2 Methods3 Concepts

Classification Problems (Homework)

8/9/2019 cs189 lecture 1

ESL9/37

8/9/2019 cs189 lecture 1

Classification

8/9/2019 cs189 lecture 1

Classification

8/9/2019 cs189 lecture 1

Classification

microsoft.com

Classification

8/9/2019 cs189 lecture 1

Classification

apple.com

ESL13/37

Classification

8/9/2019 cs189 lecture 1

Classification

8/9/2019 cs189 lecture 1

Classification

8/9/2019 cs189 lecture 1

ESL16/37

Regression

8/9/2019 cs189 lecture 1

ESL17/37

Regression

8/9/2019 cs189 lecture 1

ESL18/37

Regression

8/9/2019 cs189 lecture 1

Regression

8/9/2019 cs189 lecture 1

Regression

8/9/2019 cs189 lecture 1

Density Estimation

8/9/2019 cs189 lecture 1

Density Estimation

8/9/2019 cs189 lecture 1

Dimensionality Reduction

8/9/2019 cs189 lecture 1

Clustering

8/9/2019 cs189 lecture 1

ESL27/37

Clustering

8/9/2019 cs189 lecture 1

Clustering

8/9/2019 cs189 lecture 1

Clustering

8/9/2019 cs189 lecture 1

Machine Learning Problems

8/9/2019 cs189 lecture 1

Classification

8/9/2019 cs189 lecture 1

Classification

Regression

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Dimensionality reduction

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

Ranking

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

Ranking

Collaborative filtering

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

Ranking

Sequential decisionproblems:

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

Ranking

bandits

8/9/2019 cs189 lecture 1

Classification

Regression

Density estimation

Clustering

Ranking

banditscontextual bandits

8/9/2019 cs189 lecture 1

Classification

RegressionDensity estimation

Clustering

Ranking

banditscontextual banditsdynamic pricing

8/9/2019 cs189 lecture 1

Classification

RegressionDensity estimation

Clustering

Ranking

banditscontextual banditsdynamic pricingreinforcement learning

8/9/2019 cs189 lecture 1

1 Problems

2 Methods

3 Concepts

Methods

8/9/2019 cs189 lecture 1

Linear classifiers: Perceptron

Support vector machines

Gaussian class conditionals

Logistic regression

Naive Bayes

Linear discriminant analysisLinear regression

Decision trees, regression trees

Ensemble methods

Neural networksNearest neighbor

Principal components analysis

k-means clustering

Methods

8/9/2019 cs189 lecture 1

Logistic regression

Naive Bayes

Ensemble methods

k-means clustering

1 Classification2 Regression

Methods

8/9/2019 cs189 lecture 1

Logistic regression

Naive Bayes

Ensemble methods

k-means clustering

1 Probabilistic

modeling.

2 Prediction; not basedon a model.

8/9/2019 cs189 lecture 1

1 Problems

2 Methods

3 Concepts

Concepts

P di i b bili i d li

8/9/2019 cs189 lecture 1

1 Prediction versus probabilistic modeling.

Concepts

P di i b bili i d li

8/9/2019 cs189 lecture 1

1 Prediction versus probabilistic modeling.2 Probabilistic modeling:

Concepts

1 P di ti b bili ti d li

8/9/2019 cs189 lecture 1

Generative versus discriminative models.

Concepts

1 Prediction ers s probabilistic modeling

8/9/2019 cs189 lecture 1

Generative versus discriminative models.Maximum likelihood estimation.

Concepts

1 Prediction versus probabilistic modeling

8/9/2019 cs189 lecture 1

Generative versus discriminative models.Maximum likelihood estimation.Bayesian inference.

Concepts

8/9/2019 cs189 lecture 1

3 Optimization.

Concepts

8/9/2019 cs189 lecture 1

3 Optimization.

Convexity.

Concepts

8/9/2019 cs189 lecture 1

3 Optimization.

Convexity.

(Stochastic) gradient methods.

Concepts

8/9/2019 cs189 lecture 1

Prediction versus probabilistic modeling.2 Probabilistic modeling:

3 Optimization.

Convexity.

(Stochastic) gradient methods.Newton’s method.

Concepts

8/9/2019 cs189 lecture 1

3 Optimization.

Convexity.

4 Controlling complexity:

Concepts

8/9/2019 cs189 lecture 1

3 Optimization.

Convexity.

Bias-variance/approximation-estimation trade-off.

Concepts

8/9/2019 cs189 lecture 1

p g2 Probabilistic modeling:

3 Optimization.

Convexity.

Bias-variance/approximation-estimation trade-off.Regularization

Concepts

8/9/2019 cs189 lecture 1

p g2 Probabilistic modeling:

3 Optimization.

Convexity.

Priors

8/9/2019 cs189 lecture 1

Concepts

8/9/2019 cs189 lecture 1

2 Probabilistic modeling:

3 Optimization.

Convexity.

Priors5 Practical issues:

Train/validate/test. Over-fitting.

Concepts

8/9/2019 cs189 lecture 1

2 Probabilistic modeling:

3 Optimization.

Convexity.

Priors5 Practical issues:

Train/validate/test. Over-fitting.Resampling methods.

Overview (Part I: Bartlett)

Linear classification

8/9/2019 cs189 lecture 1

Statistical learning background

8/9/2019 cs189 lecture 1

Decision theory

8/9/2019 cs189 lecture 1

Decision theoryGenerative and discriminative models

8/9/2019 cs189 lecture 1

Decision theoryGenerative and discriminative modelsControlling complexity.

8/9/2019 cs189 lecture 1

Decision theoryGenerative and discriminative modelsControlling complexity.Resampling, cross-validation.

8/9/2019 cs189 lecture 1

Decision theoryGenerative and discriminative modelsControlling complexity.Resampling, cross-validation.The multivariate normal distribution.

8/9/2019 cs189 lecture 1

Linear regression

8/9/2019 cs189 lecture 1

Linear regression

Optimization

8/9/2019 cs189 lecture 1

Linear regression

Optimization

Linear Classification revisited

8/9/2019 cs189 lecture 1

Linear regression

Optimization

Logistic regression

8/9/2019 cs189 lecture 1

Linear regression

Optimization

Logistic regression

Linear Discriminant Analysis

8/9/2019 cs189 lecture 1

Linear regression

Optimization

Logistic regression

Linear Discriminant AnalysisSupport vector machines

8/9/2019 cs189 lecture 1

Linear regression

Optimization

Logistic regression

Linear Discriminant AnalysisSupport vector machines

Statistical learning theory

Overview (Part II: Efros)1 Memory-based/Instance-based learning

8/9/2019 cs189 lecture 1

k-nearest-neighbor

8/9/2019 cs189 lecture 1

k-nearest-neighborProperties of high dimensional spaces

8/9/2019 cs189 lecture 1

Properties of high-dimensional spaces

8/9/2019 cs189 lecture 1

k-nearest-neighborProperties of high-dimensional spaces

8/9/2019 cs189 lecture 1

Properties of high-dimensional spaces

distance learningEfficient indexing and retrieval methods

8/9/2019 cs189 lecture 1

Properties of high dimensional spaces

2 Decision treesClassification and regression trees

8/9/2019 cs189 lecture 1

Properties of high dimensional spaces

2 Decision treesClassification and regression treesRandom Forests

8/9/2019 cs189 lecture 1

3 Boosting

8/9/2019 cs189 lecture 1

3 Boosting4 Neural networks / Deep Learning

8/9/2019 cs189 lecture 1

Multilayer perceptronsVariations such as convolutional nets

8/9/2019 cs189 lecture 1

Multilayer perceptronsVariations such as convolutional netsExamples and applications

8/9/2019 cs189 lecture 1

Multilayer perceptronsVariations such as convolutional netsExamples and applications

Unsupervised methods

8/9/2019 cs189 lecture 1

3 Boosting4 Neural networks / Deep LearningMultilayer perceptronsVariations such as convolutional netsExamples and applications

Unsupervised methodsClustering

8/9/2019 cs189 lecture 1

Boosting4 Neural networks / Deep LearningMultilayer perceptronsVariations such as convolutional netsExamples and applications

Unsupervised methodsClusteringDensity estimation

8/9/2019 cs189 lecture 1

distance learningEfficient indexing and retrieval methods2 Decision trees

Classification and regression treesRandom Forests

Unsupervised methodsClusteringDensity estimationDimensionality reduction

8/9/2019 cs189 lecture 1

distance learningEfficient indexing and retrieval methods2 Decision trees

Classification and regression treesRandom Forests

Unsupervised methodsClusteringDensity estimationDimensionality reductionApplications: Collaborative filtering, etc.

cs189 lecture 1

Documents

CS189A - Capstonechris/teaching/cs189/doc/cs... · 2011. 3. 9. · – Type of association among different components of a module – Cohesion assesses why the components are grouped

Click Models for Web Search - Lecture 1 · Lecture 1 Lecture 2 Lecture 4 Practical 2 Lecture 5 Lecture 3 Lecture 2 Practical 1 AC{IM{MdR Click Models for Web Search 10. ... # shown

CS189A - Capstone - cs.ucsb.educhris/teaching/cs189/doc/cs189-05.pdf · CS189A - Capstone Christopher Kruegel Department of Computer Science ... Project Management “The Mythical

Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1

LECTURE 1 INTRODUCTION Shoubra... · 2016-10-13 · •PART 1 • Lecture 1: Introduction • Lecture 2: Entering Data • Lecture 3: Using Excel Formula • Lecture 4: Working with

Lecture 1 W.C Carter 1 Lecture 1: Introduction and Course

Lecture 11Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 vv

Lecture 1 | Winter 2019 term Lecture 1 | Winter 2019 term ... · Lecture 1 | Winter 2019 term Lecture 1 | Winter 2019 term Lecture 1 | Winter 2019 term Sequential learning, sequential

Lecture 3 1 - PEEEBpeeeb.dk/book1/chapter3/lecture3.pdf · Lecture 3 1 Lecture 3 2 Lecture 3 3 Lecture 3 4 Lecture 3 5 Lecture 3 6 Lecture 3 7 Lecture 3 8 Lecture 3 9 Lecture 3 10

2.04A Lecture Notes – Lecture 1

Lecture 1 the lecture

Lecture Notes Lecture 1

Lecture 1 - math.ucsd.edubdriver/103B_S09/Lecture... · 1 Lecture 1 1.1 De nition of Rings and Examples

iBreatheSports Inc. - UCSBchris/teaching/cs189/doc/cs... · 2011-02-04 · MooTools MooTools is a compact, modular, Object-Oriented JavaScript framework designed for the intermediate

Ch. 1. Movement Descriptionmmc.rmee.upc.edu/.../Slides/GRAU2016-2017_FC/Ch4_v19_FC.pdfLecture 1 Lecture 2 Lecture 3 Lecture 4 Lecture 5 Lecture 6 Lecture 7 3 Overview (cont’d) Stress

Anatomy, Lecture 1, Introduction (lecture notes)

Lecture 14Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 vv

Lecture 1: Automation in Environmental Engineering Lecture 1

Lecture Slides-Lecture 1

Lecture 8Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 Lecture 1 vv