Embed Size (px)
Citation preview
Neural Network Design and Application Fall 2015CPTS 434 & 534
Instructor: John MillerOffice: West 134E WSU [email protected]
Class web page can be found athttp://users.tricity.wsu.edu/~jhmiller
Required Text: Learning from Data by Abu-Mostafa, Magdom-Ismail and LinSuggested texts:
Building Neural Networks by David M. Skapura, Introduction to Machine Learning, 2nd ed by Ethem AlpaydinNeural Networks and Machine Leaning, 3rd ed by Simon Haykin
Grades:Tests and Assignments have equal weight
Tests: quizzes and final exam given in class with open books, lecture notes, and computers
Assignments:Prior approval is required for late submission. Full credit on resubmissions until tested on subject matter. 50% credit thereafter.
Graduate Project Reports:Topic approved by instructor3 – 5 pages double spacedDue last class period before dead week
Nuts and Bolts
IMPORTANT: Per new WSU policy effective August 24, I will ONLY be able to respond to emails sent from your WSU email address. I will NOT be able to respond to emails sent from your personal email address as of the first day of fall semester. Effective the 24th, the IT Department will switch the “preferred” email address in your myWSU to your WSU email address.
More nuts and bolts
Accommodations for Disabled Students: Reasonable accommodations are available for students who have a documented disability. If you have a documented disability, even temporary, make an appointment as soon as possible with the Disability Services Coordinator, Cherish Tijerina, 372-7352, [email protected] You will need to provide your instructor with the appropriate classroom accommodation form. The forms should be completed and submitted during the first week of class. Late notification may delay your accommodations. All accommodations for disabilities must be approved through Disability Services. Classroom accommodation forms are available through the Disability Services Office.
More nuts and bolts
Academic Integrity: As stated in the WSU Tri-Cities Student Handbook," any member of the University community who witnesses an apparent act of academic dishonesty shall report the act either to the instructor responsible for the course or activity or to the Office of Student Affairs."
The Handbook defines academic dishonesty to include "cheating, falsification, fabrication, multiple submission [e.g., submitting the same or slightly revised paper or oral report to different courses as a new piece of work], plagiarism, abuse of academic material, complicity, or misconduct in research."
Infractions will be addressed according to procedures specified in the Handbook.
More nuts and bolts
Safety: Should there be a need to evacuate the building (e.g., fire alarm or some other critical event), students should meet the instructor at the Cougar statue directly outside of the West building. A more comprehensive explanation of the campus safety plan is available athttp://www.tricity.wsu.edu/safetyplan/
The university emergency management plan is available athttp://oem.wsu.edu/emergencies/
Further, an alert system is available. You can sign up for emergency alerts (see http://alert.wsu.edu) through the zzusis site (http://portal.wsu.edu/).
More nuts and bolts
Student Concerns. If you have any student concerns, you can contact Carol Wilkerson the Director of Student Affairs in West 269F, (509) 372-7139, or [email protected]. If you have any concerns about this class, you should contact your instructor first, if possible. Attendance Policy. Absences should be avoided. Students should contact an instructor if an absence from class is unavoidable.
Students are encouraged to read Section 73 (Absences) of theWashington State University Academic Regulations, which is found in the WSU Tri-Cities Student Handbook.
More nuts and bolts
Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011
Predominance of ANN has diminished
Can be used as a black box
Makes nonlinear modeling easy
Magic due to biological basis
Why was ANN so popular?
Applications of ANN by subjectFrom “Neural Network Design” by Hagan, Demuth and Beale
Applications of ANN by subject
Applications of ANN by subject
Objectives of the class:
1. To learn the general principals of data mining
2. Lean to apply artificial neural networks to classification and regression problems
3. To compare artificial neural networks to other supervisedmachine-learning techniques
Topics:Basic data mining
fundamentals of machine learninglinear modelshigh-order polynomial modelsoverfitting and regularizationdimensionality reductionclustering
ANNperceptronmulti-layer perceptronfeed-forward ANNradial basis function ANN
Other techniquesself organizing mapssupport vector machines
Example of a Report-Type homework assignment
Dataset: Golub et al, Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring, Science, 286 (1999) 531-537
Download and become familiar with Weka software. Open the leukemia gene expression data in Weka. KNN technique is under the “lazy” menu of classifiers. Weka refers to KNN as “IBk” for “Instance-Based k”. After opening IBk, click on the text next to IBk to get a parameter menu. Set “KNN” to 5 and keep the default value of other parameters. Under “Test options” choose “Cross-validation” with “Folds” equal to 5.
Include the following in your report: •Objective and conclusions of the paper•Nature and Structure of the input data•Results (include the performance metrics)•Do your calculations support the authors’ conclusions
Example of a Programming-Type homework assignment
Generate 100 in silico data sets of 2sin(1.5x)+N(0,1) each with 50 random x-values between 0 and 5
Use 50 data sets for training and 50 data sets for validation
Use the training data sets for polynomial regression of orders 1 – 5
For each order calculate the following:RMS error for training data setsRMS error for validation data setsBias squaredVariance
Plot your result as error vs order
Interpret your findings in terms of the “bias – variance dilemma”
Derive the result
for Bayesian discriminant points in the 2-class problem with Gaussian class likelihoods. Assume the mean and variance of C1 are 3 and 1, respectively.Assume the mean and variance of C2 are 2 and 0.3, respectively.
For a sample size of 100, compare Bayesian discriminant points calculated from maximum likelihood estimators with those derived from the true means and variances.
Example of a Math-Type homework assignment
Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011
Availability of sophisticated machine-learning software packages, like WEKA, facilitates the application of multiple methods to the same problem
Tentative ScheduleTu Aug 21 Discussion of class syllabusTh Aug 23 Introduction to supervised machine learningTu Aug 28 Introduction to supervised machine learningTh Aug 30 Introduction to Bayesian statisticsTu Sep 4 Introduction to Bayesian statisticsTh Sep 6 Parametric methodsTu Sep 11 Parametric methodsTh Sep 13 Multivariate DataTu Sep 18 Multivariate DataTh Sep 20 Test #1Tu Sep 25 Artificial Neural NetworksTh Sep 27 Artificial Neural NetworksTu Oct 2 Artificial Neural NetworksTh Oct 4 Artificial Neural NetworksTu Oct 9 Artificial Neural NetworksTh Oct 11 Genetic AlgorithmTu Oct 16 Genetic AlgorithmTh Oct 18 Radial Basis FunctionsTu Oct 23 Radial Basis FunctionsTh Oct 25 Self-Organizing Maps Tu Oct 30 Self-Organizing MapsTh Nov 1 Test #2. Tu Nov 6 Advanced network designs Th Nov 8 Advanced network designsTu Nov 13 Advanced network designsTh Nov 15 Thanksgiving breakTu Nov 20 Thanksgiving breakTh Nov 22 Support Vector Machines Tu Nov 27 Support Vector MachinesTh Nov 29 Support Vector Machines Dec 3-7 ReviewDec 10-14 Finals week Test #3
