View
215
Download
0
Tags:
Embed Size (px)
Citation preview
What’s in the works for General Education Assessment?
Dan McCollum, Ph.D.
Associate Director of Academic Assessment
Objectives Define Assessment Explain the Assessment Cycle of Improvement Explore Purposes of Assessment Review Six Assessment Terms Identify Present Elements Describe Present into Future Elements Describe Desired Future Elements Address Ambiguities & Challenges of the Future Ask, what’s with these new G.E. outcomes? Leave time for Q & A
Define Assessment Trudy Banta – “the systematic collection, review
and use of information about educational programs for the purpose of improving student learning and development”
By definition, all assessment is formative assessment
Logically, if it’s not formative, it’s not assessment
Expanded from: Commentary by Trudy Banta, 2008 IUPUI Assessment Institute
Explain the Assessment Cycle of Improvement
PLANNING(Re) Invent Outcomes
(Re) Design Curriculum(Re) Develop Measures
LEARNINGOpportunity to Learn
ASSESSINGUse MeasuresAnalyze Data
Discuss Results
Adapted from: Suskie, L. (2004). Assessing student learning: A common sense guide.
Explore Purposes of Assessment1. Committing to student success
What are students learning? How well? (baseline) Comparison for improvement (cyclical/longitudinal) Comparison to local standards (local criteria for success) Planning curriculum improvements & innovations
2. Meeting accreditation standards of NWCCU How do we know what and how well students are learning? Accreditation – a catalyst for student success
3. Informing outside constituents of program/university performance Accountability – a catalyst for student success Comparison for external standards (e.g., national criteria/ranks)
Adapted in part from: Linda Suskie, Middle States Commission on Higher Education (MSCHE)
Review Six Assessment Terms
1. Direct Measure
2. Indirect Measure
3. (Course) Embedded Measure
4. Common Measure
5. Local Measure
6. External Measure
Adapted in part from Andrea Leskes of AAC&U: http://www.aacu.org/peerreview/pr-sp02/pr-sp02reality.cfm
Identify Present Elements A set of desired Learning Outcomes Course-embedded, local and external,
direct measures A common, local, indirect measure
Identify Present Elements Learning Outcomes
English Composition World Literature Constitutions Mathematics Multicultural International Distribution: (1) Humanities & Fine Arts, (2) Life and
Physical Sciences & Analytic Thinking, (3) Social Sciences
Source: UNLV Undergraduate Catalog Fall 2008 – Spring 2010
Identify Present Elements Course-embedded, local, direct measures
Aligned to student learning objectives within courses, but sometimes aligned to learning objectives within sections of a course (a faux pas)
Aligned with learning outcomes?
Currently being expanded Studying online tracking, analysis, and reporting
systems
Identify Present Elements Course-embedded, external, direct
measure: A Standardized Test Collegiate Assessment of Academic Proficiency
(CAAP) – Writing Essay and Writing Skills Used to assess English Composition course-
sequence objectives aligned to the English Composition General Education Outcome
For more on CAAP: http://www.act.org/caap/
Identify Present ElementsCommon, Local, Indirect measure: Graduating Senior Exit Survey (GSES) includes items to address how well students believe they achieved in:
1. Improving writing skills (English Composition)2. Comparing literature of different cultures and times (World Literature)3. Able to interpret the U.S. and Nevada constitutions in broad contexts
(Constitutions)4. Learn about mathematical/statistical reasoning (Math)5. Analyze multiple, contemporary U.S. Cultures (Multicultural)6. Learn about sociopolitical complexities of interconnected world cultures
(International)7. Appreciate fine arts (Distribution – Fine Arts)8. Understand individual & group behaviors and the impacts of policies on
them (Distribution – Social Sciences)9. Learn about the role of science & engineering in society (Distribution –
Life and Physical Sciences & Analytic Thinking)
Describe Present into Future Elements Sampling plans being developed for senior pilot of
CAAP’s Writing Essay, Critical Thinking, & Math A Baseline Measurement
Sampling plans being developed for first full, freshman implementation of same CAAP tests for VSA
Learning Outcomes being re-invented Will be the focus of future General Education Assessment See: http://generaled.unlv.edu/geac.html
For more on Voluntary System of Accountability (VSA): http://www.voluntarysystem.org/index.cfm?page=homePage
Describe Desired Future Elements Common, Local, Indirect Measure of self-
perceptions of outcomes (entering, graduating, alumni)
Common, External, Indirect Measure of engagement (entering & graduating)
Common, Local, Direct Measure of Outcomes (entering, mid-level, graduating)
Common, External, Direct Measure of Writing, Critical Thinking & Math via CAAP for VSA (entering & graduating)
For more on Voluntary System of Accountability (VSA): http://www.voluntarysystem.org/index.cfm?page=homePage
Describe Desired Future Elements: Full Model with Timing
Entry(1) External Direct (2) Local Direct (3) Local Indirect (4) External Indirect
Mid-Level (Entry for Transfers)
(2) Local Direct
Graduating(1) External Direct (2) Local Direct (3) Local Indirect (4) External Indirect
Alumni(3) Local Indirect
For more on External Direct and Indirect Measures: http://www.voluntarysystem.org/docs/cp/CollegePortraitDetails.pdf
Address Ambiguities and Challenges - CAAP
Pilot Senior Sampling (Spring 2009) ~600 students will be sampled Proportional Quota Sampling
Sample students from each College (college is the Condition) proportionate to the College’s overall part of the University’s graduating population
Motivators for Students Required by course policy Explain use of data for public “portrait” of UNLV Rewards: (1) gift cards (2) graduation fee rebates (3) cash
(4) extra credit (5) public recognition of performance (6) raffle
Address Ambiguities and Challenges - CAAP Full Entry Sample (Fall 2009?)
~ 750 students needed to represent the population Simple Random Sample Proportional Quota Sample Stratified Random Sample No sample Census
Motivators for Students Required part of orientation/registration by university policy Scores on transcripts Rewards
Address Ambiguities and Challenges - CAAP Full Senior Sample (Spring 2010?)
~ 750 students needed to represent population Simple Random Sample Proportional Quota Sample Stratified Random Sample No sample Census
Motivators for Students Required for graduation Scores on transcripts Rewards
Address Ambiguities and Challenges – Desired Full Model
Development of Local Measures by Assessment Measures Development Committees
One committee per outcome To include developing blueprints and/or rubrics To include item writing and/or assignment development To include standard setting for each level
Assessment Days to administer measures to entering, mid-level, and graduating students
Scoring of Local, Direct Measures by Assessment Measures Scoring Committees
One committee per outcome
Address Ambiguities and Challenges – Committees
Gaining faculty participation in measure development and scoring
Having expert measurement people to train and guide the developers and scorers
Address Ambiguities and Challenges: Costs
Faculty stipends and/or releases Proctors Rewards for students CAAP: ~ $39/student for test materials Indirect, external engagement measure
Ask, what’s with these new G. E. outcomes?
What will they be? See: http://generaled.unlv.edu/geac.html
Possibly implemented alongside the curriculum that follows the current General Education outcomes
Quasi-experimentation can be used to compare students’ performance in new and current curriculums