Communication

Problem Solving

Civic Engagement

Cultural & Global

Engagement

Information LiteracyIntegrative Learning

Critical Thinking

Ethical ReasoningTeamwork

Reading

Taking Institutional Level Assessment to the Course Level

MEET YOUR PRESENTERS

Diana Bullen Faculty – Director Business

Management ProgramsFormer Chair, Student Outcomes Committee

Marjorie LetaFaculty Librarian

Assessment CoordinatorChair, Student Outcomes

Committee

ABOUT MCC

22,700+ Students Multiple Sites Online Classes International Students

TODAY’SLEARNING OUTCOMES Create broad Institutional Student Learning

Outcomes (ISLOs) that connect to institutional values, college curriculum and your community

Map curriculum to ISLOs Connect assessment practices to what is

actually happening in the classroom Define and identify student artifacts for

assessment purposes

ASSESSMENT HISTORY AT MCCCIRCA 1996 Faculty driven via

Student Outcomes Committee (SOC)

Supported by VPAA & Institutional Effectiveness

Faculty clusters create/select assessment tools

OUTCOMES ASSESSED ANNUALLY ON A ROTATING BASIS

ASSESSMENT WEEK

February/March Faculty

Volunteers Compare pre &

post groups

Winner of 2007 CHEA Award for Institutional Progress in Student

Learning Outcomes

HOWEVER, THE PROCESS WAS DISCONNECTED Assessment not

relevant to individual courses

Students never saw scores

Faculty participation was limited

‘Closing the Loop’ was difficult

WE KNEW IT WAS TIME FOR A CHANGE…

New Assessment Criteria from HLC:

Clearly stated goals for student learning and effective processes

Uses the information gained from assessment to improve student learning

Substantial participation of faculty and other instructional staff members

INFORMED IMPROVEMENT (ii)SOC began the ii process in fall 2012 to research

assessment options, gather evidence and implement an action plan for change.

Ask a Question:How can the SOCii Team facilitate the effective integration of general education outcomes assessment throughout all MCC programs and departments?

STEP ONE: CREATE BROAD ISLOsOBJECTIVE: Develop ISLOs that are relevant to all subjects

that can be assessed at the course level allowing for substantial participation and application of data collected.

MCC’S PREVIOUS ISLOsCommunication 

1. Write a clear, well‐organized paper using documentation and quantitative tools when appropriate.

2. Construct and deliver a clear, well‐organized, verbal presentation.

3. Interact in a collaborative, synergistic manner within a small group problem solving meeting.

4. Maintain an interpersonally effective climate within a one to one dyadic interchange.

CREATING BROAD ISLOs: THE PROCESS

Consider… College Vision, Mission

& Values Current/Previous ISLOs Curriculum CommunityConsult… Bloom’s Taxonomy AAC&U VALUE Rubrics

Information Literacy Numeracy

Problem Solving

Critical Thinking

Critical Thinking

CREATING BROAD ISLOs In fall 2013, SOC held a series of open, drop-in workshops inviting all faculty to participate in the revision and creation of new ISLOs that are:

relevant to all subjects

assessable at the course level

meaningful to all stakeholders

supported by all faculty

BLOOM’S TAXONOMY

Remember

Understand

ApplyAnalyze

EvaluateCreate

AAC&U VALUE RUBRICS

Valid Assessment of Learning in Undergraduate Education

The VALUE rubrics were developed by teams of faculty experts representing colleges and universities across the United States through a process that examined many existing campus rubrics and related documents for each learningoutcome and incorporated additional feedback from faculty.

-AAC&U

MCC’S 4CS ARE BORN…

CommunicationCivic EngagementCritical ThinkingCultural & Global Engagement

The Keys to Deeper Learning

TAKE 3 ACTIVITY: CREATING BROAD ISLOsDiscuss the ISLO given to your teamConsider the following:1. Is it applicable across curriculum?2. Where does it fall within Bloom’s

Taxonomy – below, at, or above application level?

CREATING BROAD ISLOsSPECIFIC

COURSE/PROGRAM

Define culture and subculture.

BROADACROSS CURRICULUM

Demonstrate an ability to engage respectfully with others in a diverse society.

Construct and deliver a clear, well-organized verbal presentation.

Apply language of a discipline in an appropriate and accurate manner to demonstrate comprehension.

ASSESSING ISLOs

Source: Knowing What Students Know and Can Do NILOA, January 2014

National Student Surveys Rubrics Classroom-Based Assessment**

Alumni Surveys

85%

69%66% 64%

Most Commonly Used Assessment Tools

**course or program embedded assessment that capture student performance where teaching and learning occur

STEP TWO: CREATE SCORING GUIDELINES FOR ASSESSING ISLOs

Expert

Proficient

Developing

Needs Improvement

Bloom’s

Mastery

CRITICAL THINKING VALUE RUBRICfor more information, please contact value@aacu.org

DefinitionCritical thinking is a habit of mind characterized by the comprehensive exploration of issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion.

Evaluators are encouraged to assign a zero to any work sample or collection of work that does not meet benchmark (cell one) level performance.

Capstone

4

Milestones

3 2

Benchmark

1

Explanation of issues Issue/ problem to be considered critically is stated clearly and described comprehensively, delivering all relevant information necessary for full understanding.

Issue/ problem to be considered critically is stated, described, and clarified so that understanding is not seriously impeded by omissions.

Issue/ problem to be considered critically is stated but description leaves some terms undefined, ambiguities unexplored, boundaries undetermined, and/ or backgrounds unknown.

Issue/ problem to be considered critically is stated without clarification or description.

EvidenceSelecting and using information to investigate a point of view or conclusion

Information is taken from source(s) with enough interpretation/ evaluation to develop a comprehensive analysis or synthesis. Viewpoints of experts are questioned thoroughly.

Information is taken from source(s) with enough interpretation/ evaluation to develop a coherent analysis or synthesis.Viewpoints of experts are subject to questioning.

Information is taken from source(s) with some interpretation/ evaluation, but not enough to develop a coherent analysis or synthesis.Viewpoints of experts are taken as mostly fact, with little questioning.

Information is taken from source(s) without any interpretation/ evaluation.Viewpoints of experts are taken as fact, without question.

Influence of context and assumptions Thoroughly (systematically and methodically) analyzes own and others' assumptions and carefully evaluates the relevance of contexts when presenting a position.

Identifies own and others' assumptions and several relevant contexts when presenting a position.

Questions some assumptions. Identifies several relevant contexts when presenting a position. May be more aware of others' assumptions than one's own (or vice versa).

Shows an emerging awareness of present assumptions (sometimes labels assertions as assumptions).Begins to identify some contexts when presenting a position.

Student's position (perspective, thesis/hypothesis)

Specific position (perspective,thesis/ hypothesis) is imaginative, taking into account the complexities of an issue.L imits of position (perspective, thesis/ hypothesis) are acknowledged.Others' points of view are synthesized within position (perspective,thesis/ hypothesis).

Specific position (perspective,thesis/ hypothesis) takes into account the complexities of an issue.Others' points of view are acknowledged within position (perspective,thesis/ hypothesis).

Specific position (perspective,thesis/ hypothesis) acknowledges different sides of an issue.

Specific position (perspective,thesis/ hypothesis) is stated, but is simplistic and obvious.

Conclusions and related outcomes (implications and consequences)

Conclusions and related outcomes (consequences and implications) are logical and reflect student’s informed evaluation and ability to place evidence and perspectives discussed in priority order.

Conclusion is logically tied to a range of information, including opposing viewpoints; related outcomes (consequences and implications) are identified clearly.

Conclusion is logically tied to information (because information is chosen to fit the desired conclusion); some related outcomes (consequences and implications) are identified clearly.

Conclusion is inconsistently tied to some of the information discussed; related outcomes (consequences and implications) are oversimplified.

COMMUNICATION – SCORING GUIDELINESDEFINITION: Communication is the purposeful development, expression and reception of a message through oral, written or nonverbal means.   

Descriptors: Expert Proficient Developing Needs Improvement

1. PURPOSE:Establish a clear central focus for a message which demonstrates an understanding of context, audience and task

Purpose is PRECISELY STATED AND MEMORABLE for audience, context, and task

Purpose is PRECISELY STATED for audience, context, and task

Purpose is STATED for audience, context, and task

Purpose is NOT CLEARLY STATED for audience, context, and task

2. CONTENT:Develop appropriate, relevant content logically sequencing ideas and/or information

Relevant content is developed EXPERTLY, LOGICALLY sequencing IDEAS AND/OR INFORMATION

Relevant content  is developed APPROPRIATELY, LOGICALLY sequencing IDEAS AND/OR INFORMATION

Relevant content is developed SPORADICALLY, sequencing ONLY POINTS

Message  developed  is  SIMPLE and LACKS relevant content and sequencing

3. LANGUAGE:Apply language of a discipline in an appropriate and accurate manner to demonstrate comprehension

Language choices and/or  modes of expression are THOUGHTFUL, MEMORABLE, COMPELLING AND APPROPRIATE to audience AND discipline

Language choices and/or  modes of expression are THOUGHTFUL AND APPROPRIATE  to audience AND discipline

Language choices and/or  modes of expression  are APPROPRIATE to audience OR discipline

Language choices and/or  modes of expression  DETRACT from message  

4. EXECUTION:Convey a message effectively

CONSISTENTLY conveys intended message

MOSTLY conveys intended message

SELDOM conveys intended message

FAILS TO convey intended message

5. RECEPTION:Utilize appropriate oral, written or nonverbal means to receive and/or interpret a message effectively

CONSISTENTLY utilizes appropriate oral, written or nonverbal means to receive and/or interpret intended message

MOSTLY utilizes appropriate oral, written or nonverbal means to receive and/or interpret intended message

SELDOM utilizes appropriate oral, written or nonverbal means to receive and/or interpret intended message

FAILS TO utilize appropriate oral, written or nonverbal means to receive and/or interpret intended message

MCC’s 4Cs: COMMUNICATION

STEP THREE:MAP CURRICULUM TO ISLOs

Course Mappingverb The process of aligning (mapping) course competencies with any program competencies and Institutional Student Learning Outcomes (ISLOs) which ensures the integration of learning.

Your Course

ISLO(s)

DISCIPLINE MAPPING: COMMUNICATION

CTE PROGRAM MAPPING – VET TECH

LESSONS LEARNED Competencies do not match what is happening in the classroom! Many competencies need updating to reflect

current trends Many competencies are written below Bloom’s

application level Some faculty resistance on process

STEP FOUR: IDENTIFY STUDENT ARTIFACTS FOR ASSESSMENT

Artifactnoun Examples of student work used to assess if student(s) meet course, program or institutional outcomes.

ALIGNING ARTIFACTS TO ISLOS AND SCORING GUIDELINES

Civic Engagement1. Inclusiveness2. Application of Knowledge3. Demonstration of Civic Identity & Commitment4. Civic Communication5. Engagement in Civic Action & Reflection

Scoring Guidelines:• Critical Thinking (CT)• Communication (CO)• Civic Engagement

(CE)• Cultural & Global

Engagement (CG)Data

Collection via

Canvas

Course or Prefix Outcome

Faculty/Departments/Programs Office of Institutional Effectiveness

1. Identify one ‘C’ that best aligns with your course

2. Identify a cumulative

assignment that assesses ALL criteria

of the aligned ‘C’

3. Attach ‘C’ Scoring Guidelines to the

cumulative assignment rubric in Canvas & score mastery upon completion

Course Level Assessment Pilot :

Spring 2015

EXS101=CT

SPA115=CG

Art116=CO

EDU282=CE

CONCLUSIONS – THE ROAD AHEAD It’s a slow and steady process A dedicated assessment position is necessary for

continuity of institutional learning assessment Assessment Coordinator/Director

College-wide collaboration and support is essential Administration, Center for Teaching & Learning, Institutional

Effectiveness, Student Services, Institutional Advancement We are breaking down silos

Greater faculty sharing within departments More creative and innovative assignments Growing excitement across the college

Learning Assessment should not be a secret…

QUESTIONS?

Diana.Bullen@mesacc.eduMarjorie.Leta@mesacc.edu

For further information visit:mesacc.libguides.com/SOC

SOURCESNational Institute for Learning Outcomes Assessment (NILOA) http://www.learningoutcomesassessment.orgAmerican Association of Colleges & Universities• http://www.aacu.orgThe Primer Series by Ruth Stiehl and Les Lewchuk http://www.outcomesnet.comBloom’s Taxonomy Bloom, et al. (1956). Taxonomy of educational objectives: The classification

of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.