21
Review CS540 Introduction to Artificial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu and Yingyu Liang July 2, 2019

Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

CS540 Introduction to Artificial IntelligenceLecture 14

Young WuBased on lecture slides by Jerry Zhu and Yingyu Liang

July 2, 2019

Page 2: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

PerceptronReview

Perceptron update rule.

Perceptron termination condition.

Page 3: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Logistic RegressionReview

Logistic update rule.

Logistic cost function.

Convexity.

Hessian, Laplacian, eigenvalue.

Page 4: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Neural NetworkReview

Activation.

Backpropogation.

L1 and L2 regularization.

Cross validation.

Multi class classification.

Page 5: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Multi Layer Neural Network ExampleReview

Page 6: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

LTU Activation ExampleReview

Page 7: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Support Vector MachineReview

Hard margin support vector.

Soft margin maximization.

Subgradient descent.

Kernel trick.

Page 8: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Support Vector Margin ExampleReview

Page 9: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Feature Vector to Kernel ExampleReview

Page 10: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Decision TreeReview

Entropy.

Information gain.

Bagging and boosting.

Page 11: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Decision Tree ExampleReview

Page 12: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

K Nearest NeighborReview

Distance functions.

Page 13: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

K Nearest Neighbor Cross Validation ExampleReview

Page 14: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Convolutional Neural NetworkReview

Convolution.

Pooling.

Trained weights.

Page 15: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Convolutional Weights Count ExampleReview

Page 16: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Computer VisionReview

Histogram of Gradients Features.

Scale Invariant Feature Transform.

Block normalization.

Dominant orientation.

Harr Features.

Page 17: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Histogram of Gradient ExampleReview

Page 18: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Natural Language ProcessingReview

Bigram and trigram model.

Transition matrix.

Random word generation.

Bayes rule.

Page 19: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Document Bayes Rule ExampleReview

Page 20: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Bayesian NetworkReview

Conditional probability table.

Maximum likelihood estimation.

Training vs inference.

Chow Liu algorithm.

Page 21: Young Wu July 2, 2019pages.cs.wisc.edu/~yw/CS540/CS540_Lecture_14_P.pdfReview CS540 Introduction to Arti cial Intelligence Lecture 14 Young Wu Based on lecture slides by Jerry Zhu

Review

Common Cause Network ExampleReview