Download ppt - Using WIDS to Document How Colleges Assess Learning Outcomes

Using WIDS Using WIDS to to Document Document How How Colleges Colleges Assess Assess Learning Learning OutcomesOutcomes

Using WIDS Using WIDS to to Document Document How How Colleges Colleges Assess Assess Learning Learning OutcomesOutcomes

Overview

What is assessment? What are the drivers? How does curriculum fit in? How does WIDS help?

“Assessment of student learning is a participatory, iterative process that:

Provides data/information you need on your students’ learning Engages you and others in analyzing and using this

data/information to confirm and improve teaching and learning Produces evidence that students are learning the outcomes

you intended Guides you in making educational and institutional

improvements Evaluates whether changes made improve/impact student

learning, and documents the learning and your efforts.”

NCA Higher Learning Commission

From “Student Learning, Assessment and Accreditation: Criteria and Contexts”, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006.

Assessment of Learning

Informed judgments about achievement of intended learning outcomes

Based on evidence of what students can do and their capacity to apply what they know (data) at the completion of the learning experience (course, certificate, program, or other credential)

Valid, reliable, and fair

Built into the plan for teaching

Data-driven evaluation that is actively used for continual improvement of teaching and learning

Assessment of Learning

Sound Assessment

ValidValidOutcomes based on standards (industry)Measures intended outcomesMeasures application and critical thinking

ReliableReliablePerformance assessment based on consistent rubrics, scoring guides, and rating scalesConsistent process - each learner is assessed in same way as other learners

FairFairValidReliableLearners informed of expectations up front!Feedback to learners

Summative/Formative Assessment

What are the drivers?

1. Learning & continuous improvement of teaching and learning

2. Accreditation (NCA, AQIP, Industry)

3. Carl Perkins IV

HLC/AQIP asks 5 fundamental questions: How are your stated learning outcomes appropriate to your

mission, programs, students and degrees?

How do you ensure shared responsibility for student learning & assessment of student learning?

What evidence do you have that students achieve your stated learning outcomes?

In what ways do you analyze and use evidence of student learning?

How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?

NCA Higher Learning Commission

From “Student Learning, Assessment and Accreditation: Criteria and Contexts”, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006.

The What: Both secondary and postsecondary must develop a process for measuring technical skills that lead to an industry-recognized credential, or state-recognized credential, or locally-developed credential

The Why: Both secondary and postsecondary education systems must provide, as a separate measure of accountability, the extent of skill development in Career and Technical Education (CTE) programs

The How: Both secondary and postsecondary education must develop valid and reliable assessments

Carl Perkins IV – The New Law

Postsecondary Indicators Technical Skill Attainment Credential, Certificate, or Diploma Student Retention or Transfer Student Placement Nontraditional Participation Nontraditional Completion



WTCS will move toward a balance of silver and gold assessment of learning outcomes.

Gold Standard (ex. NCLEX, ASE, Barber/Cosmetology)

External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation.

Silver StandardWTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation.

May be developed and/or implemented by the state system. May be developed and/or implemented locally according to system guidelines

and with industry (advisory committee) approval. Must be valid and reliable.

Bronze Non-assessment related indicators such as indirect measures.

GPA and/or Course Completion Program Completion – Graduation Teacher-developed exams that do not meet silver standard (not externally

approved).


Summary of Standards for Assessment of Technical Skill Attainment

Plan• Establish outcomes• Build assessments• Design learning

DO• Deliver• Assess• Document

CHECK• Analyze assessment

data• Propose

improvements

ACT•Adjust design

•Adjust delivery

•Adjust assessment

Assessm

en

t Pro

cess a

nd

C

urric

ulu

m

The Learning Assessment Design Technology Tool Kit

Tra

cDat

B

lack

boar

d

iWeb

folio

eF

olio

Que

stio

nmar

k

Peo

ple

Sof

t B

anne

r D

atat

el

Assessment Mgt SoftwareAssessment Mgt Software(ex. TracDat)(ex. TracDat)

• Describe Outcomes (Learning/Org)• Build Assessment Plans• Track Continual Improvement

Activities• Document/Report Results

WIDSWIDS•Establish Learning Outcomes•Build Assessments•Design and Plan Learning

Enterprise SystemEnterprise System(ex. PeopleSoft DataTel or (ex. PeopleSoft DataTel or

Banner)Banner)•Manage Student Records•Manage Enrollment/Scheduling•Manage Budget•Etc.

Assessment SoftwareAssessment Software(ex. Questionmark)(ex. Questionmark)

• Author Q & A Assessments and Surveys

• Deliver Assessments/Surveys• Record Learners Performance

(In terms of outcomes)

Electronic PortfolioElectronic Portfolio(ex. iWebfolio/MSNCU eFolio)(ex. iWebfolio/MSNCU eFolio)

• Present evidence of individual learning/professional experience

• Document connection between artifacts and outcomes

• Owned by the student

WIDS Learning Design System populates assessment, electronic portfolio, enterprise, and assessment systems with learning outcomes and performance standards.

Assessment Management Software provides feedback loop for continual improvement of learning design using WIDS.

Web-based Electronic Portfolio System presents evidence of individual’s learning and professional experience.

Learning Outcom

es

Learning Outcomes

Lea

rnin

g O

utco

mes

Cur

ricul

um D

ocum

ents

Learning Outcomes

Rubrics

Student Learning

Docum

entationC

ontin

uous

Im

pro

vem

ent

Fee

dbac

k

Student Data

What do they need to be able to achieve?

Who are the learners?

How will we know and show when they’ve achieved it?

How will they get there?

3 of 4 components answer assessment questions

WHO

WHAT

WHEN

HOW

WIDS Model

Linked Competencies

(performance assessments)

Exit Learning Outcomes

Program Outcomes(summative assessment)

Core Abilities

(summative assessment)

Gen Ed Outcomes(summative assessment)

Job Task AnalysisJob Task Analysis(DACUM)(DACUM)

Job Task AnalysisJob Task Analysis(DACUM)(DACUM)

External StandardsExternal Standards(Research-based)(Research-based)

External StandardsExternal Standards(Research-based)(Research-based)

Drives Learning Design

Assesses Learning ResultsIntegrating the OutcomesIntegrating the Outcomes

Linked Competencies


Linked Competencies


Formative assessment of program outcome

Summative assessment of learning outcomes

External

Program-Level

Course-Level

Drives Learning Design

Assesses Learning Results

External

Program-Level

Course-Level

Dental Hygiene ExampleDental Hygiene ExampleDental Hygiene ExampleDental Hygiene Example

Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services.

Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies

Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist.

ADA Dental Hygiene Education Standard

Dental Hygiene Program Outcome

Dental Hygiene Competency Formative assessment of program outcome

Summative assessment of Learning outcomes

Challenges

How do we communicate WTCS long-standing commitment to industry input and standards?

How do we continue WTCS commitment to performance assessment whenever possible (versus testing?)

How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?)

How do we ensure that we are doing what our model and assessment plans imply?

1. Create Program Design files for all programs

2. Develop Exit Learning Outcomes Core Abilities (mission level) Program Outcomes (discipline level) General Education Outcomes (Optional)

3. Determine summative assessments

4. Define criteria for summative assessments

5. Link courses to exit learning outcomes

Using WIDS Program Design

Design. Document. Align

6. Create Performance Assessment Tasks (with scoring guides and rubrics) to support summative assessment Embed in program courses, using the

program file as the master file Create a separate Summative

Assessment course file for program

Using WIDS Learning Design


7. Generate Analyzer reports Learning Outcomes Matrix Program Course List Matrix Assessment Task Outcomes Matrix

Using WIDS Analyzer


Learning Outcomes Matrix

Outcomes Courses

Documents where outcomes are addressed

Summative Assessments

This report requires the following: WIDS Program File - program outcomes; direct measures and WIDS Course Files - course title; program outcomes (linked to competencies-optional); and Performance Assessment Tasks that target or link to program outcomes.

Cla

ssifi

catio

n C

hart

Res

each

Stu

dy

Com

mun

ity D

enta

l Hea

lth P

ropo

sal

Com

mun

ity H

ealth

Den

tal P

rogr

am S

ervi

ce P

roje

ct

Ref

lect

ion

Inf

ectio

n C

lass

ifica

tion

Ana

lysi

s

HA

ZC

OM

Wor

kshe

et a

nd L

abel

s

Per

sona

l and

Pat

ient

Pro

tect

ion

Ski

ll D

emo

Tre

atm

ent

Roo

m R

epro

cess

ing

Sim

ulat

ion

Ins

trum

ent

Re

proc

essi

ng D

emo

Was

te M

anag

emen

t E

xam

Pat

ient

His

tory

Rol

e P

lay

Vita

l Sig

ns S

kill

Dem

onst

ratio

n

Em

erge

ncy

Eq

uipm

ent

Mai

nten

ance

Per

form

ance

Tes

t

Che

mic

al E

me

rgen

cy S

imul

atio

n an

d In

cid

ent

Rep

ort

Med

ical

Em

erge

ncy

Sim

ulat

ion

and

Inci

dent

Rep

ort

Inf

ectio

us E

xpos

ure

Sim

ulat

ion

and

Inci

dent

Rep

ort

Law

s an

d E

thic

s C

ase

Stu

dy A

ssig

nmen

t

Pro

fess

iona

l Dev

elop

men

t P

lan

Den

tal T

eam

Cha

rt

Con

fiden

tialit

y C

ase

Stu

dy

Inf

orm

ed C

onse

nt R

ole

Pla

y

Pro

fess

iona

l App

eara

nce

Che

cklis

t

Clin

ical

Eva

lua

tion

Pat

ient

Den

tal H

ygie

ne C

are

Pla

ns

Cas

e M

anag

em

ent

Eva

luat

ion

Off

ice

Sim

ulat

ion

Ethics and Professionalism: Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies

L L L L L L L L T L L L

Ethics and Professionalism: Model dental hygiene professional code of ethics in a rapidly changing environment

L L L L L T L

Ethics and Professionalism: Pursue lifelong professional growth and development through self-directed learning, participation in professional organizations, and continuing education

Promoting Oral Health: Counsel clients/patients to reduce health risks L T L

Assessment Task Outcomes Matrix - Dental Hygiene Sample

Program Outcomes

Community Dental Health

Dental Health Safety Dental Hygiene Ethics and Professionalism

Dental Hygiene Process IV

Assessment Task Outcomes Matrix

Performance Assessment Tasks By Course

Shows the assessment and the link/target

Maximize the potential of WIDS to support assessment planning:

Create/maintain Program Outcome Summaries for all new and existing programs.

Develop/maintain Course Outcome Summaries for all courses.

Design Performance Assessment Tasks for course level (formative) and beyond-the-course (summative) assessments.

Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses.

Present WIDS documentation as a central feature in accreditation.