Embed Size (px)
Citation preview
4/1/2016
1
2016
Program Evaluator Candidate
Face-to-Face Training
Spring, 2016
2
We cannot solve our problemswith the same thinking we used
when we created them.~ Albert Einstein
• Facilitator Introductions• List of Participants – on each table• Introduce Yourself – first time you speak
Welcome!
4/1/2016
2
3
ABET HeadquartersRepresentatives
ABET Welcome
4
• Provide you with knowledge, skills, and abilities to conduct ABET accreditation evaluation.
• Learn how to provide guidance to programs for improvement.
• Our goal: To improve consistency of program evaluation.
Why Are We Here?
PEVC Course Purpose
4/1/2016
3
5
• PEVC Face-to-Face (F-2-F) Training is experiential workshop that simulates real visit as closely as possible.
• Works in conjunction with online Pre-Work modules, designed for your independent and convenient learning.
Course Purpose (cont’d)
6
At completion of PEVC F-2-F Training, you will be able to: • Plan effective, cordial, efficient campus visit.
• Interact with your team during visit.
• Given specific situation or evidence (e.g., transcripts, student work): Determine which criteria are relevant to determining
accreditation action.
Determine if program is in compliance with criterion. If not, determine level of compliance and accreditation action.
Why Are We Here? Learning Outcomes
4/1/2016
4
7
At completion of training, you will be able to: • Conduct interviews of faculty, staff, and students.
• Write exit statement to institution.
• Conduct informal debrief with program head and exit meeting with leaders of institution.
• Complete Program Evaluator forms.
• Demonstrate “in practice” effective use of PEV competencies in PEV Competency Model.
Learning Outcomes (cont’d)
8
Agenda – Day 1, Morning
Module Name Timing
1) Welcome and Introduction 8:00‐8:15 a.m.
2) Connection to Pre‐Work andAssessment Fundamentals
8:15‐9:30 a.m.
Break 9:30‐9:45 a.m.
3) Display Materials and Facilities 9:45‐11:00 a.m.
4) Sunday Team Meeting 11:00‐12:30 p.m.
Lunch 12:30‐1:15 p.m.
4/1/2016
5
9
Agenda – Day 1, Afternoon
Module Name Timing
5) Interviews 1:15‐2:05 p.m.
6) Monday Night Meeting 2:05‐3:35 p.m.
Break 3:35‐3:50 p.m.
7) Writing Exit Statements 3:50‐4:55 p.m.
8) Homework 4:55‐5:15 p.m.
Wrap‐Up Day 1 5:15‐5:25 p.m.
10
Agenda – Day 2, Morning
Module Name Timing
Day 2 Overview 8:00‐8:10 a.m.
9) Context for Judgment 8:10‐9:00 a.m.
10) Informal Program Debrief and Exit Meeting 9:00‐10:05 a.m.
Break 10:05‐10:20 a.m.
11) Exit Statements 10:20‐11:00 a.m.
12) Connection to PEV Competencies 11:00‐11:20 a.m.
13) Post‐Visit Work 11:20‐11:29 a.m.
14) Post‐Training 11:29‐11:38 a.m.
15) Closing Exercise 11:38‐11:45 a.m.
4/1/2016
6
11
• Speak with intention.
• Listen with attention.
• Know that wisdom is in room.
• Ensure representation from all pertinent voices.
• Model PEV competencies in all facets of training.
• Turn off/silence electronic devices (cell phones, laptops, etc.).
• Start/stop on time.
Agreements
12
• NotebookSlide Presentation
• Reference NotebookGeneral Accreditation Materials
Commission-Specific Materials
• Exercise Handouts• Table Supplies• Issue Bins
Training Materials
4/1/2016
7
13
Module 2: Connection to Pre-Work
14
1. Overview2. The Accreditation Process3. The Role of the Program Evaluator4. Continuous Improvement of Student
Learning5. Applying the Criteria6. The PEV Appraisal Process
Pre-Work Modules
4/1/2016
8
15
Pre-Requisite Work Modules:
• Overview
• The Accreditation Process
• The Role of the Program Evaluator
• CI of Student Learning
• Applying the Criteria
• The PEV Appraisal Process
Completed:
• Proficiency Assessments (3)
• Upper State University Self-Study and Transcript Analysis (PEV Report and PEV Worksheet as appropriate for Commission)
Completed Pre-Work
16
In your table groups:• Use refresher cards to test your knowledge of Pre-Work topics:
• Each take turn asking, answering, and recording the result.
• If you get incorrect response, another group member may “help” out.
• Your Support Facilitator has answers and is available for assistance.
• Record answers your team gets correctly first time, second time, and third time AND number of cards you have gone through.
• YOU HAVE 50 MINUTES.
Handout: Refresher Cards
Answer Key : Handout 2-2
Pre-Work Refresher Cards: Exercise
4/1/2016
9
17
• Pre-Work analysis provides starting point for evaluation of Upper State University.
• F-2-F training simulates campus visit to Upper State University. “On-site” activities reveal additional insight and
information about program.
• Therefore: Keep open mind.
Stay in information-gathering mode.
Do not look through lens of “reinforcing pre-drawn conclusions.”
Upper State University Pre-Work Analysis
18
At end of F-2-F training you’ll leave with:
• Table team decision whether program meets criteria:Team consensus.
My interpretation and your interpretation of like circumstances must be same.
Upper State University Site Visit
4/1/2016
10
19
• Accept responsibility for decisions you make...
• Serve only in areas of competence.
• Act as agent of ABET and avoid conflicts of interest.
• Keep decisions confidential.
• Be objective and truthful.
• Behave honorably, ethically and lawfully.
• Report concerns with respect to internal financial matters.
• Treat all persons fairly.
• Assist colleagues… in following the Code of Conduct.
• Support prompt and fair adjudication of alleged violations...
ABET Code of Conduct(Rules of Procedure Section EIGHT A & B)
20
• Current or past employment as faculty, staff, or consultant by institution or program
• Current or past discussion or negotiation of employment with institution or program
• Attendance as student at institution
• Receipt of honorary degree from institution
• Institution or program where close family relative is, or was, student or employee within past 10 years
• Unpaid official relationship within past 10 years with institution
Conflict of Interest(Rules of Procedure Section SEVEN A & B and APPM II.C)
4/1/2016
11
21
• Day 0: Business CasualUsually a Sunday afternoon campus visit. Time used to review course materials, assessment information, and tour laboratories.
Dress Code for Visits
22
• Day 1: Business Professional Meetings with Faculty, Students, Support Areas
• Day 2: Business ProfessionalExit Meeting with CEO of Institution
Dress Code for Visits
4/1/2016
12
23
24
Module 3Display Materials and Facilities
4/1/2016
13
25
Interview withLaboratory Manager
26
Labs/Design Studios
• Sufficient number and size of labs.
• Appropriate coverage across breadth of specializations within program.
• Appropriate equipment, in good repair.
• Appropriate student access (including evening and weekend access).
• Appropriate technician support and instructional support in lab.
What to Look For: Facilities
4/1/2016
14
27
Labs/Design Studios (cont’d)
• Safe physical arrangement and appropriate safety practices followed.
• Comments from students about lab experience.
• Is there some type of “Laboratory Plan” for maintaining and upgrading instructional laboratories? This is not required by ABET, but, in general, formal or informal plan of some sort should exist.
What to Look For: Facilities (cont’d)
28
Classrooms• Appropriate physical arrangement, support for computer projection,
etc.
• Not overcrowded.
Support facilities• Sufficient computer access, with appropriate off-hours access.
• Appropriate spaces for students to gather (not explicit criterion but relates to several criteria).
• Appropriate shop with parts, repair facilities, etc.
Faculty offices• Sufficient size, privacy.
What to Look For: Facilities (cont’d)
4/1/2016
15
29
Panel Discussion
30
Display Materials
4/1/2016
16
31
Display Materials
32
• Course materials including course syllabi, textbooks, example assignments and exams, and examples of student work, typically ranging from excellent through poor.
• Evidence program educational objectives are based on needs of stated program constituencies.
• Evidence of assessment, evaluation, and attainment of student outcomes.
• Evidence of actions taken to improve program.
Display Materials (APPM II.G.6.b)
4/1/2016
17
33
Descriptions of materials for:• Curriculum/Course Work
• Senior Design Projects, Capstone or Integrating Experience (if required)
• Demonstration of Student Outcomes
• Assessment Materials
Handouts 3‐1 & 3‐2
Small Group Sessions with USU Display Materials
34
• Courses appear appropriate to accomplish Program Educational Objectives and Student Outcomes.
• Student work indicates active engagement, demonstration of learning, and reasonable grading standards.
• Evidence with respect to specific Student Outcomes (including those in criteria) as appropriate to assessment plan.
• Few missing items, or even entire missing courses, are not necessarily systemic problems. Pursue any major gap to see if represents serious problem.
What to Look For: Course Materials
4/1/2016
18
35
• Evidence (lab reports) of appropriate student learning (not just cookbook).
• Evidence to support Student Outcomes.
What to Look For: Laboratory Reports
36
• Evidence all students complete major design, capstone or integrating experience (if criteria require) that draws on previous courses and incorporates standards and realistic constraints.
• Student work should demonstrate via a complete project report or some other mechanism.
What to Look For: Major Design or Integrating Experience
4/1/2016
19
37
• Examples of instruments used and actual data collected.
• Summaries of the data with results reported in usable form.
• Recommendations for program improvement based on data.
• Implementation and results.
What to Look For: Assessment Materials
38
• Institutional catalogs and similar publications must clearly indicate the programs accredited by the commissions of ABET as separate and distinct from any other programs or kinds of accreditation.
• Each accredited program must be specifically identified as “accredited by the _________ Accreditation Commission of ABET, http://www.abet.org.”
• Each ABET-accredited program must publicly state the program’s educational objectives and student outcomes.
• Each ABET-accredited program must publicly post annual student enrollment and graduation data per program.
Handout: 3-4
Documentation (APPM II.A.6)
4/1/2016
20
39
40
Module 4:Sunday Team Meeting
4/1/2016
21
41
• Agenda Review• Visit Schedule• Program Evaluator Pre-Visit Briefings
Program Strengths Potential Program Shortcomings Recommended Action
• Display Material and Facility Findings• Agree on Common Findings and Plans to
Investigate• Debrief Handout 4-1
Sunday Team Meeting Agenda
42
• Usually involves review of multiple programs with one PEV per program.
• Discussions will focus on common shortcomings across programs and what team needs to do to ensure consistent evaluation and recommended action.
• Usually two meetings: Before review of display materials and facility tours.
After review of display materials and facility tours.
Actual Visit Sunday Meeting
4/1/2016
22
43
44
Module 5: Interviews
4/1/2016
23
45
• Direct interface with members of program: Facilitates understanding of program.
Provides an opportunity to clarify items in Self-Study and to probe for further details.
Effective way to identify problems not addressed in Self-Study.
Why Interview?
46
Panel Discussion
4/1/2016
24
47
Advisor Interview
48
In your table groups:• Draw card from scenario card deck.
• Share ideas for dealing with situation: What would you say?
• Facilitator: Capture ideas on sticky-notes.
• Post ideas/responses for each scenario on appropriate flipchart in room.
Handout: Interview Cards
Managing Problem Interviews: Exercise
4/1/2016
25
49
• All interview questions should be DESIGNED to elicit detailed response:Determine and clarify objectives BEFORE
interview.
Prepare questions.
The Art of Asking Questions
50
• Begin with:Why? How? What? When? Where?
• Avoid leading questions that assume an answer.
• Prompt for specific information:“Show me”
The Art of Asking Questions (cont’d)
4/1/2016
26
51
• Turn interviewee into teacher:“Can you explain that to me?”
• Ask an obvious question to establish interviewee as expert.
• Periodically verify what you heard:“Do I understand you to say…?”
• Remember, silence is golden.
The Art of Asking Questions (cont’d)
52
• Suggest interview of junior/senior level class (~20 minutes without instructor present)
• Briefly describe ABET and Comprehensive Review
• Possible questions: How well are laboratory facilities and equipment working?
How satisfactory and available are computer laboratories?
How helpful are enrollment and career advising?
How do courses from Chem, Phys, and Math support
engineering courses?
Handout 5-1
Student Interviews
4/1/2016
27
53
54
Module 6: Monday Night Meeting
4/1/2016
28
55
• Each PEV Candidate in turn brief team on potential finding including: Criterion to be cited
Evidence gathered
Compliance level
• Discuss evidence, criteria, and compliance as team and develop consensus for finding.
• You have maximum of 60 minutes.
Monday Night Meeting
56
• Similarities with simulation
• Differences with simulation
Actual Monday Team Meeting
4/1/2016
29
57
58
Module 7 Writing Exit Statements
4/1/2016
30
59
Handout 7-1
Types of Statements
60
• Statement will generally include one or more types of findings: Strength
Deficiency
Weakness
Concern
Observation
• Format will vary depending on type of visit:General Review
Interim Visit
Types of Findings
4/1/2016
31
61
General Review
62
• Introduction
• Program Strengths
• Program Shortcomings
• Deficiencies (in order of Criteria)
• Weaknesses (in order of Criteria)
• Concerns (in order of Criteria)
• ObservationsNOTE: ETAC does not require an Exit Statement.
The PAF serves as the Exit Statement.
Handout 7-2
Outline of Exit Statement
4/1/2016
32
63
• Include information such as:Type or special characteristics of program.
Emphasis area(s).
Number of enrolled students.
Size of most recent graduating class.
Number of faculty members.
Other information that could be helpful to next team.
Introduction Statement
64
“The industrial engineering BS program prepares students for careers in system design, optimization, and ergonomics. The program has 150 students, and 14 full-time faculty members support the program. The program had 39 graduates in the 2014-2015 academic year. All courses in the major are available by distance education, but students must come to campus or have facilities available through their employers for laboratory experiences in two courses.”
Introduction: Example
4/1/2016
33
65
• Contents:Criterion/policy citationWhat was observedEffect on program
Program Shortcoming Statements
66
• Criteria/Policy Citation State which criterion or policy applies Quote excerpts as needed
Criteria/Policy Citation
4/1/2016
34
67
• What was observed:Describe what was observed (evidence).
Describe how your finding points to lack of compliance with criterion or policy or points to lack of strength of compliance.
• Effect: Describe negative effect it has on the program (D or
W) with respect to criterion.
Describe potential future effect on program (C) with respect to criterion.
What Was Observed & Effect
68
Students Criterion
Criterion 1 requires that the program must have and enforce procedures to ensure and document that students who graduate meet all graduation requirements.
Deficiency: Example (Part 1)
4/1/2016
35
69
While most students who graduated successfully completed all requirements, there were a few students who did not pass all of the required courses designated by the program.
Students can graduate from program without passing all the required courses. Thus, program is not in compliance with Student criterion.
Deficiency: Example (Part 2)
70
Program Educational Objectives Criterion
Criterion 2 requires that there must be a documented, systematically utilized, and effective process, involving program constituencies, for the periodic review of these PEOs that ensures they remain consistent with the institutional mission, the program’s constituents’ needs, and these criteria.
Weakness: Example (Part 1)
4/1/2016
36
71
The program has relied on a review of the PEOs every three years by the faculty only. The other constituencies (alumni, employers, students) were not involved in this review.
Without involvement of all constituencies, program can not ensure that PEOs meet needs of all constituencies. Thus strength of compliance with criterion is lacking.
Weakness: Example (Part 2)
72
Institutional Support Criterion
Criterion 8 requires that the resources available to the program must be sufficient to attract, retain, and provide for the continued professional development of a qualified faculty. The resources available to the program must be sufficient to acquire, maintain, and operate infrastructures, facilities, and equipment appropriate for the program, and to provide an environment in which student outcomes can be attained.
Concern: Example (Part 1)
4/1/2016
37
73
At present, it appears that resources are adequate to support the program. However, there have recently been large reductions in the operating budget of the program.
If these budgetary reductions continue, future compliance with criterion may be jeopardized.
Concern: Example (Part 2)
74
Handouts 7-3 & 7-4
Statement Critiquing: Exercise
4/1/2016
38
75
• Must be based on conclusions from evidence found by team to be credible and significant.
• Names of individuals or titles that identify individuals must not be used.
• Avoid attributing opinions, conclusions or recommendations to other individuals or groups of individuals besides Commission.Poor: The team discovered that…
Better: Evidence indicated that…
Additional Guidelines for Writing Statements
76
• Provide enough detail that program and institution will know precisely what is inadequate.
• Provide enough detail to Commission to justify type of finding.
• Provide enough detail that team making the next general or interim review will be able to determine amount of progress that has been made since original finding.
Additional Guidelines for Writing Statements (cont’d)
4/1/2016
39
77
• If wording does not add to or clarify finding, do not include it.
• Avoid use of acronyms or other abbreviations.
• Avoid combining findings on two criteria into one statement.
Additional Guidelines for Writing Statements (cont’d)
78
• ONLY document left with institution at end of the visit.
• MUST be consistent with Exit Statement.• Level of compliance.• Wording to describe what was observed
and effect on program.Handout 7-5
Program Audit Form [PAF]
4/1/2016
40
79
Program Audit Form [PAF] (cont’d)
80
Program Audit Form [PAF] (cont’d)
4/1/2016
41
81
• Introduction• Program Strengths• Program ShortcomingsDeficiencies (in order of Criteria)
Weaknesses (in order of Criteria)
Concerns (in order of Criteria)
• Observations
Format of Exit Statement
82
• Use statement rubric to carefully review your exit statement before turning in to Team Chair.
• Ask another PEV to review statement for clarity and consistency with PAF.
• Read statement out loud to your fellow team members.
Reviewing Your Statement
4/1/2016
42
83
Table Discussion
Statement Writing Summary
Handout 7‐6
84
4/1/2016
43
85
Module 8Homework
86
• Read two Judgment Scenarios.
• Update your PEV Worksheet to reflect new information from today’s activities: Interviews, display materials, information from Dean,
and team discussions.
• Draft your Exit Statement following statement format and writing guidelines. Introduction, and shortcomings based on your team
consensus for Upper State University.
• Complete the Program Audit Form.
Handouts 8-1 & 9-1
Homework Assignment
4/1/2016
44
87
• You will email completed Exit Statement and Program Audit Form to ABET Staff person whose email address will be provided.
• PAF for your commission is available online in Module 7.
• Homework will be evaluated by your table Support Facilitator using statement rubric.
• We will discuss Judgment Scenarios Sunday morning.
Handouts 8-1 & 9-1
Homework Assignment (cont’d)
88
• Day One once-around the roomWhat do you leave with today?
What’s “top of mind” about the day?
Thank you for your time, energy,and attention today!
Day 1, Closing
4/1/2016
45
89
Welcome Back!
Day 2
PEV F-2-F Training
Do the right thing. It will gratify some
people and astonish the rest.
~ Mark Twain
90
Agenda – Day 2, Morning
Module Name Timing
Day 2 Overview 8:00‐8:10 a.m.
9) Context for Judgment 8:10‐9:00 a.m.
10) Informal Program Debrief and Exit Meeting
9:00‐10:05 a.m.
Break 10:05‐10:20 a.m.
11) Writing Exit Statements 10:20‐11:00 a.m.
12) Connection to PEV Competencies 11:00‐11:20 a.m.
13) Post‐Visit Work 11:20‐11:29 a.m.
14) Post‐Training 11:29‐11:38 a.m.
15) Closing Exercise 11:38‐11:45 a.m.
4/1/2016
46
91
Module 9The Context for Judgment
92
• Evaluation of program for accreditation requires PEV judgment on extent to which each criterion is satisfied.
• PEV judgment is based on evidence obtained during evaluation process.
• Judgment was demonstrated in exit statements written last night for Upper State University as discussed yesterday.
Context for Judgment
4/1/2016
47
93
Throughout case study:• Where did you use evidence as basis for
judgment?• How and when did you see evidence-based
judgment applied by others? • What did you struggle with? • Why is evidence-based judgment
important?
Context for Judgment (cont’d)
94
• Accreditation has moved away from rigid “bean counting” process to one that is based on continuous quality improvement processes.
• ABET Criteria are based on principles of Continuous Quality Improvement (CQI).
Context for Judgment (cont’d)
4/1/2016
48
95
An educational program CQI process should reflect clear understanding of:• Mission
• Constituents
• Objectives
• Outcomes
• Processes (internal practice to achieve outcomes)
• Facts (data collection)
• Evaluation (interpretation of facts)
• Action (feedback to support decision making and improve processes)
Context for Judgment (cont’d)
96
Handout 9-2
Judgment Scenario: Exercise
4/1/2016
49
97
• Many questions of judgment center around assessment and continuous improvement.
• When such issues arise, ask:Are assessment processes adequate to determine
achievement of each of outcomes?
Are assessment processes robust enough to identify shortcomings and achievements?
Are assessment processes sustainable?
Will assessment processes lead to program improvements if appropriate?
Context for Judgment (cont’d)
98
• Definition:Program educational objectives are broad statements
that describe what graduates are expected to attain within a few years of graduation. Program educational objectives are based on the needs of the program’s constituencies.
• Key Words:Broad Statements
Expected to Attain
Needs of Constituencies
Program Educational Objectives
4/1/2016
50
99
• Should be Forward Looking• Should use Expected to Attain language
and not Preparing Graduates to Achieve• Must periodically review PEOs• Must ensure PEOs remain consistent with
institutional mission, program’s constituents’ needs, and these criteria
• Can include language from Student Outcomes
PEOs: Important Concepts
100
• Judgment centers on observed evidence that supports achievement of each criterion.
• When issues arise about specific components of criteria, ask: Does evidence support my decision?
What additional evidence could I seek?
Summary
4/1/2016
51
101
102
Module 10Informal Program Debrief & Exit Meeting
4/1/2016
52
103
• Must have complete Exit Statement and Program Audit Form at this stage.
• Must be able to elaborate on statement and explain conclusions without falling into debate with program head.
• Must be able to relate every shortcoming to Criteria.
Informal Debrief Preparation
104
• Must understand what not to say, particularly recommended accreditation action.
• May make comments intended to be helpful while making it clear that these comments (which may appear in the report as Observations) have no relation to findings with respect to Criteria.
• Must remain completely consistent in all statements to every person.
Informal Debrief Preparation (cont’d)
4/1/2016
53
105
Note:This is NOT Upper State University
Handout 10-1
Informal Debrief Demonstration
106
Simulating Informal Debrief: Exercise
4/1/2016
54
107
• Must have complete Exit Statement and Program Audit Form that follow all ABET guidelines.
• Statement is read verbatim.• Must be no deviation from written Exit
Statement.• If questions are asked, answers must
essentially re-state Exit Statement without introducing new material.
Exit Meeting
108
• Always consult with Team Chair if doubt about appropriate response to question.
• Always be courteous and respectful.• Thank institution for campus hospitality.
Exit Meeting (cont’d)
4/1/2016
55
109
• Do not deviate from Exit Statement.
• Do not make comments regarding aspect of program or institution other than those in Exit Statement.
• Do not compare visited institution to your own institution.
• Do not make disparaging comments about students or anyone else.
• Do not disagree with other team members in any way.
• Do not make any social conversation that could possibly be misinterpreted.
• Do not arrive late or leave early.
What NOT TO DO at Exit Meeting
110
4/1/2016
56
111
Module 11Writing Exit Statements & Other Items
112
• Observations and Statement Format• Interim Actions• Evaluation of Online/Hybrid and Multi-
Site Programs
Writing Exit Statements & Other Items
(cont’d)
4/1/2016
57
113
Handout 11-1
Types of Statements
114
Strengths & Observations
4/1/2016
58
115
• Strength is exceptionally strong, effective practice or condition that stands above norm, and has positive effect on program.
• For each Strength, state:What was observed
What makes it stand above norm
Positive effect it has on program
ETAC Note:
Strengths are considered observations.
Strength Statements
116
“The program judiciously integrates laboratory experience with classroom work in the courses.
Offering hardware and software based laboratories and project experiences help students better appreciate the course material, promotes excellence in student education, provides hands-on experience, enhances teamwork, and better prepare students for the real world.
The positive feedback from students attests to utility of their laboratory and project experiences.”
Strength: Example
4/1/2016
59
117
Definition:“Comment or suggestion which does not relate directly to the accreditation action but is offered to assist the institution in its continuing efforts to improve its programs”
- ABET Accreditation Policy & Procedures Manual Section II.G.9.a.(3)
Statement of Observation
118
“Use of CSE 2221, Computing in Engineering & Science, as a prerequisite to senior-level courses could be considered as a way to enhance the computational experience of the students.”
Statement of Observation: Example
4/1/2016
60
119
• Take 5 minutes and read Strength statements on page 1 of handout.
• Determine whether each statement is appropriate Strength statement or not.
• Discuss as table.• Repeat for Observation statements on
page 2 of handout.• You have 20 Minutes. Handout 11-2
Strength and Observation: Exercise
120
• If program undergoing General Review Visit and receives one or more Weaknesses or Deficiencies, then subject to Interim Action:
Weakness Interim Report orInterim Visit
Deficiency Show Cause Report orShow Cause Visit
Interim Actions
4/1/2016
61
121
• Interim Report or Show Cause ReportTeam Chair writes Draft Statement
• Interim Visit or Show Cause VisitPEVs and Team Chair writes Exit Statement,
PAF, and Draft Statement For Interim Statement, experienced PEVs
involved – some differences in format from General Review
Interim Actions (cont’d)
122
• More common that programs are offered online/hybrid/multi-site.
• Criteria is same regardless of delivery method.
• For program offered using multiple modalities, program must be able to demonstrate program is equivalent in all modalities. If offered at multiple sites, program must be able to demonstrate program is equivalent at all sites.
Online / Hybrid / Multi-Site Programs
4/1/2016
62
123
• Online/hybrid/multi-site program may require greater time commitment in preparation and evaluation than normal for single-site program delivered face-to-face.
• “Weakest link” concept applies to program evaluation. If issue is found within one delivery modality or at specific site, finding and resulting accreditation action (if impacted by finding) will apply to program in its entirety, regardless of delivery method.
Online / Hybrid / Multi-Site Programs (cont’d)
124
Online / Hybrid / Multi-Site Programs(cont’d)
• If program or one or more required courses supporting program are delivered by more than one method or at different physical sites, evaluators expect to see separate course/assessment materials for each delivery method/location.
4/1/2016
63
125
• Most online programs are delivered through a learning management system (LMS). If program is dependent on LMS, PEV will be provided with necessary access to course materials, discussion boards, chat rooms, and graded student work, preferably well before physical visit. Video conferencing or other electronic means may be used to facilitate student, faculty and constituents interviews.
Online / Hybrid / Multi-Site Programs (cont’d)
126
4/1/2016
64
127
Module 12 Connection to Competencies
128
• Technically Current• Effective Communicator• Interpersonally Skilled• Team-Oriented• Professional• Organized
Handout 12-1
ABET PEV Competency Model
4/1/2016
65
129
At your table, discuss:
• Which competencies were effectively demonstrated within group? How?
• Which competencies can we individually and collectively improve?
PEV Competency Model: Exercise
130
• Competencies form basis of performance appraisal:Evaluation of PEV Candidates: Evaluated by your table Support Facilitator and
PEVC peers at end of F-2-F training.
Evaluation of PEVs: Each PEV will be evaluated by institution, Team
Chair, and PEV peers after each visit.
Competency-Based Performance Appraisal Process
4/1/2016
66
131
• Serves as a vehicle for professional development.
• Tool for continuous improvement.• Clearly communicates what is expected.• Everyone gets evaluated every time. • Evaluate against Competency Model.
Competency-Based Performance Appraisal Process (cont’d)
132
• 360-Degree Process: Team Chair
Team Members
Institution
• “Met Expectations” (rating of 3) describes how competency demonstrated pre-visit and onsite. This is benchmark.
• Actionable comments for items < 3 rating.
Competency-Based Performance Appraisal Process (cont’d)
4/1/2016
67
133
Performance Appraisal Process
134
4/1/2016
68
135
Module 13Post-Visit Work
136
The Accreditation Timeline
JanuaryInstitution requests
accreditation for programs
February - MayInstitution prepares
self-evaluation (Program Self-Study
Report)
March - JuneTeam chairs
assigned, dates set, team members
chosen
September - DecemberVisits take place, draft statements written and
finalized following7-day response period
December - FebruaryDraft statements editedand sent to institutions
February - AprilInstitutions respondto draft statement
and return to ABET
May - JuneNecessary changes
to statement,if any, are made
JulyCommission meets to take final action
AugustInstitutions notified
of this action
Year 1 Year 2
4/1/2016
69
137
• 7-Day Response from institution.
• Draft Statement prepared, edited, and sent to institution.
• 30-Day Due Process Response from institution.
• Final Statement prepared and edited.
• Commission takes final accreditation action.
• ABET sends Final Statement and accreditation letter to institution.
Post-Visit Process
138
• Within one (1) week after visit is completed, the PEV must:Submit visit expenses using online expense
system; include original itemized receipts.
• And, PEV should:Complete Team Chair and Peer PEV
Performance Appraisal Forms online.
PEV Responsibilities – Post-Visit
4/1/2016
70
139
• PEV should not communicate with program concerning ABET matters after visit has ended.
• Team Chair may ask PEV to help evaluate Due Process Response from program.
PEV Responsibilities – Post-Visit (cont’d)
140
• Always use latest ABET Travel Policy and Procedures Manual.
• Make reservations through ABET’s travel agent.
• Submit expenses online using ABET online expense Web tool.
• Scan original receipts for ALL lodging, meals, and transportation (tolls, parking, taxi, bus, rail) or submit original receipts by mail.
ABET Travel Policy
4/1/2016
71
141
Module 14Post-Training
142
• Leave behind completed Exit Statement and Program Audit Form (PAF).
• Evaluation of Exit Statement and PAF by Facilitator.
• Evaluation by Facilitator and peers using modified PEV performance appraisal.
• Member society notified of completion of training and availability of evaluation results.
• Update your bio on ABET secure website with dates available for 2016 Fall visit.
What Happens Next?
4/1/2016
72
143
• Training Mentor there to support you!• Debrief with Training Mentor soon.Focus on key learnings from F-2-F
training.Ask any follow up questions.Share any other feedback on training
(Pre-Work and F-2-F).
Contact Your Training Mentor
144
• Member society makes assignments based on institutions’ program needs and PEV pool.
• Some societies will require Observer visit first.
• Institutions check team list for any conflict of interest.
• Team Chair approves your participation and obtains approval from institution then contacts you.
Getting Assigned
4/1/2016
73
145
Questions on what happens after the training?
Handout 14-1
Questions and Answers
146
Module 15Closing
4/1/2016
74
147
Compared to yesterday morning,how prepared do you now believe you areto go out on an ABET accreditation visit?
What Have We Accomplished?
A Lot!
148
• Complete evaluations and leave in evaluation envelope at your table.
Support FacilitatorPEV Candidate PeersJust-in-Time EvaluationsOverall Training Evaluation
Evaluations
4/1/2016
75
149
GOODBYE
150
Thank you for your time, energy, and
attention.
Safe travels!
