Upload
easter-mosley
View
225
Download
0
Embed Size (px)
DESCRIPTION
Goal of ABET To promote Continuous Quality Improvement in Applied Sciences, Computing, Engineering, and Technology education through faculty guidance and initiative 11/10/2015Criterion 4: Improvement Process
Citation preview
ABET AccreditationCriterion 4: Continuous Improvement
Direct Assessment of Learning Outcomes
Dr. Abdel-Rahman Al-QawasmiAssociate Professor
EE Department
11/10/2015 Criterion 4: Improvement Process
Quality Vice-Deanship in the college of EngineeringMajmaah University
ABET?• ABET, incorporated as the Accreditation Board for
Engineering and Technology.• ABET is the recognized U.S. accreditor of college and
university programs in applied science, computing, engineering, and technology
11/10/2015 Criterion 4: Improvement Process
Goal of ABET
• To promote Continuous Quality Improvement in Applied Sciences,
Computing, Engineering, and Technology education through faculty guidance and
initiative
11/10/2015 Criterion 4: Improvement Process
ABET Criteria
11/10/2015 Criterion 4: Improvement Process
Students(1)
Facilities )7( Institutional Support )8(Curriculum )5( Faculty )6(
Criteria for Accrediting Engineering Programs
• Program Educational Objectives • Student Outcomes
• Assessment • Evaluation
11/10/2015 Criterion 4: Improvement Process
Continuous Improvement )4(
11/10/2015 Criterion 4: Improvement Process
Important :Important :Should document your processes for regularly assessing and evaluating the extent to which the student outcomes are being attained
[Exams, student portfolios, senior project presentations]
Assessment: is defined as one or more processes that identify, collect, and prepare the data necessary for evaluation
Evaluation:is defined as one or more processes for interpreting the data acquired though the assessment processes in order to determine how well the student outcomes are being attained
11/10/2015 Criterion 4: Improvement ProcessAssessment for Quality AssuranceAssessment for Quality Assurance©©
LearningLearning OutcomesOutcomes
ConstituentsConstituents
Assessment: Assessment: Collection, Analysis Collection, Analysis
of Evidenceof Evidence
Evaluation:Evaluation:Interpretation Interpretation
of Evidenceof Evidence
Feedback Feedback for for
Continuous Continuous ImprovemeImproveme
ntnt
MeasurablMeasurable e
PerformanPerformance Criteriace CriteriaEducational Educational
Practices/StrategPractices/Strategiesies
MissionMissionEducationaEducational l
ObjectivesObjectives
Assess/Evaluate
11/10/2015 Criterion 4: Improvement Process
ABET Terms Definition
Objectives
Broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve.
Outcomes Statements that describe what students are expected to know and able to do by the time of graduation.
Performance Criteria Specific, measurable statements identifying the performance(s) required to meet the outcome; confirmable through evidence.
AssessmentProcesses that identify, collect, use and prepare data that can be used to evaluate achievement.
EvaluationProcess of reviewing the results of data collection and analysis and making a determination of the value of findings and action to be taken.
11/10/2015 Criterion 4: Improvement Process
Step-by-Step Assessment
11/10/2015 Criterion 4: Improvement Process
Choose your one of the Assigned to Choose your one of the Assigned to your course ABET student your course ABET student
OutcomesOutcomes
Students outcomes are assigned to your course and
Approved
Choose the assessment method Exams, Quiz, Project
Analyze the results of assessment KPIs and Rubrics
Final Result is below the Target for
Performance
Yes
recommendation and action plan
The internal Target for Performance is 75%
Your comments and observations
ABET student Outcomesa-k
• (a) an ability to apply knowledge of mathematics, science, and engineering • (b) an ability to design and conduct experiments, as well as to analyze and interpret
data • (c) an ability to design a system, component, or process to meet desired needs within
realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability
• (d) an ability to function on multidisciplinary teams • (e) an ability to identify, formulate, and solve engineering problems• (f) an understanding of professional and ethical responsibility• (g) an ability to communicate effectively • (h) the broad education necessary to understand the impact of engineering solutions in
a global, economic, environmental, and societal context • (i) a recognition of the need for, and an ability to engage in life-long learning • (j) a knowledge of contemporary issues • (k) an ability to use the techniques, skills, and modern engineering tools necessary for
engineering practice.
11/10/2015 Criterion 4: Improvement Process
11/10/2015 Criterion 4: Improvement Process
Data do not assist in driving Data do not assist in driving improvement. To drive any improvement. To drive any
improvement, one needs information - improvement, one needs information - a detailed presentation of the results. a detailed presentation of the results.
SLO, KPIs and Rubrics• ABET learning outcomes, Key performance
Indicators (KPIs) and Rubrics are ready and approved for every program in the college of Engineering.
• Refer to the program coordinator to get all information about your course
• All instructors should be involved in the evaluation process in order to evaluate the program
11/10/2015 Criterion 4: Improvement Process
Information for Driving Improvement
11/10/2015 Criterion 4: Improvement Process
Learning Outcome Assessment
Area1>B Area 2<B Area 3>B Area 5>BArea 4>B
Sub area 1>B Sub area 2<B Sub area 3>B Sub area 4<B
The recommended actions for improvement for the student outcome are related to Sub area 2 and Sub area 4
11/10/2015 Criterion 4: Improvement Process
What is a rubric, anyway?????
• A rubric is a set of categories which define and describe the important components of the work being completed, critiqued, or assessed.
• Each category contains a gradation of levels of completion or competence with a score assigned to each level and a clear description of what performance need to be met to attain the score at each level.
The recommended actions for improvement
• Actions Related to the Course Objectives• Actions Related to Student Outcomes• Actions Related to teaching Strategies• Actions Related to Assessment Strategies• Actions Related to Other aspects
11/10/2015 Criterion 4: Improvement Process
The recommended actions should be written by the end of semester in the course Report
11/10/2015 Criterion 4: Improvement Process
Assessment Methods
• Written surveys and questionnaires
• Exit and other interviews
• Standardized exams• Locally developed
exams• Archival records• Focus groups
• Portfolios• Simulations• Performance
Appraisal• External
examiner• Oral exams• Behavioral
observations
11/10/2015 Criterion 4: Improvement Process
Direct Measures
Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning outcomes
11/10/2015 Criterion 4: Improvement Process
Indirect Measures
Indirect measures of student learning that ascertain the opinion or self-report of the extent or value of learning experiences
11/10/2015 Criterion 4: Improvement Process
Direct Indirect• Exit and other interviews• Standardized exams• Locally developed exams• Portfolios• Simulations• Performance Appraisal• External examiner• Oral exams• Behavioral observations
• Written surveys and questionnaires
• Exit and other interviews
• Archival records• Focus groups
Student Learning Outcomes (code) Assessment Methods
a1 Reports, discussions and presentationsa2 Exams and presentationsb1 Standardized exams, Oral exams, Micro projectsb2 Reports and presentationsb3 Standardized exams, Oral exams, Micro projectsb4 Standardized exams, Oral exams, Micro projectsb5 Behavior observation and reportsc1 Behavior observation and presentationsc2 Discussionsc3 Reports, discussions and presentationsd1 Standardized exams, Oral exams, Micro projectsd2 Reports, discussions and presentationsd3 Exams, quizzes and reportsd4 Standardized exams, Oral exams, Micro projects
11/10/2015 Criterion 4: Improvement Process
LO:An ability to design and conduct experiments, as well as to analyze and interpret data
11/10/2015 Criterion 4: Improvement Process
Example:
11/10/2015 Criterion 4: Improvement Process
Question1: Experiment Lab )measuring current(SLO: An ability to design and conduct experiments, as well as to analyze and interpret dataMaximum grade :15Number of students: 14No Grades No Grades
1 7 8 5
2 8 9 6
3 10 10 7
4 3 11 1
5 11 12 14
6 12 13 12
7 8 14 15
Rubrics
RecommendationsWrite your recommendations based on your observations and the result of your used assessment method
KPI )2(:Experimental plan of data gathering
Unsatisfactory Range)0-9(
)1(No systematic plan of data gathering;
experimental data collection is
disorganized, even random,
and incomplete
DevelopingRange)9-12(
)2(Develops a simplistic experimental plan of data gathering, does not recognize entire scope of study (e.g. not all parameters
affecting the results are investigated)
SatisfactoryRange )12-15(
)3(Formulates an
experimental plan of data gathering to attain
a stated objective (develop correlation,
test a model, ascertain performance of equipment, etc.)
8 4 2
Final Result)1X8 + 4X2 + 2X3(/)3X14( =57.1% )weight(
Or )4+2(/14= 42% )Student percentage(
Results AnalysisStrength points: Strength points: 1- The students average performance is good.2- The plan of the experimental is well defined. Weaknesses points: 1- There is an average problem in applying the procedure.2- There is a problem in Identifying and collecting data in organizing wayRecommended actions: 1- Data collection methods and Statistical analysis should be give more attention2- Including a statistical analysis and data collection method in the course description or in other low level course.
11/10/2015 Criterion 4: Improvement Process
Make it Simple!
11/10/2015 Criterion 4: Improvement Process
Learning Outcome
Assessment Method Question
(a) First Exam Question 1
(e) Second Exam Question 3
(h) Mini-Project Presentation
(h) and (k) Final Exam Question 3 and Question 5
Your Plan for outcomes assessment:
Course Report• The required course Report by ABET at the
end of the semester is simple.• Collect all results from all assessment
methods you used during the semester to fill your report for every learning outcome assigned and approved for your course
11/10/2015 Criterion 4: Improvement Process
ABET LO a b c d e f j h i j k
EE XXX √ √ √ √
(%) 77 67 80 76
EE program SOs
11/10/2015 Criterion 4: Improvement Process
CourseSLO
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (EE1) (EE2) (EE3) (EE4)EE 101 X X XEE 111 X X X XEE 202 X X X X XEE 205 X X X EE 206 X X X X X EE 207 X X X X EE 208 X X X X XEE 212 X X X X EE 221 X X X X XEE 234 X X X XEE 270 X X X XEE 271 X X X X EE 288 X X X XEE 307 X X X XEE 308 X X X X EE 322 X X X XEE 323 X X X X EE 341 X X X X XEE 360 X X X X XEE 361 X X X
Table 4-1-a: Student outcomes mapping for general courses
11/10/2015 Criterion 4: Improvement Process
Automation of Assessment Process
11/10/2015 Criterion 4: Improvement Process
Thank you for Listening
11/10/2015 Criterion 4: Improvement Process