Assessing Student Learning in Public Higher Education

Working Group Membership • Dr. Lois Alves - Vice President for Enrollment Services, Research, and Planning, Middlesex

Community College • Dr. Mario Borunda - Dean of Education, Lesley University, Member, Board of Higher Education • Dr. Patricia Crosson, Chair - Professor Emeritus of Higher Education, University of Massachusetts

Amherst; Senior Advisor for Academic Policy, Department of Higher Education • Dr. Michael Fiorentino - Executive Vice President / Academic Affairs, Fitchburg State University • Dr. Lane Glenn - Vice President of Academic Affairs, Northern Essex Community College • Dr. Kate Harrington - Associate Vice President, Academic Affairs, Student Affairs, and

International Relations, University of Massachusetts President‘s Office • Liam Hogan - Parliamentarian of the Student Senate, Holyoke Community College • Dr. Jonathan Keller - Associate Commissioner for Research, Planning, and Information Systems,

Massachusetts Department of Higher Education • Dr. Nicholas Massa - Professor of Engineering Technology, Springfield Technical Community

College • Dr. Patricia Maguire Meservey - President, Salem State University • Dr. Christopher J. O’Donnell - Professor of Mathematics, Massachusetts Maritime Academy • Dr. Patricia Plummer - Senior Advisor to the Senior Vice President for Academic Affairs, Student

Affairs, and International Relations, University of Massachusetts President‘s Office • Dr. Robert Pura - President, Greenfield Community College • Dr. Michael Young - Associate Provost for Academic Planning and Administration, Bridgewater

State University • Former members: Both served until June 2010

• Lois Martin - Professor of Mathematics, Retired, Massasoit Community College • Julianne Cormio - Student Board Representative, Framingham State University

Respond visibly to domains of legitimate external concern.

Show action on the results of assessment.

Emphasize assessment at the major transition points in a college career.

Embed assessment in the regular curriculum

Source:  Ewell, P. T. (2009).  Assessment, accountability, and improvement: Revisitng the Tension.  (NILOA Occasional Pater No. 1) Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.

Four Principles to Guide Policy and Practice

5

LEAP Essential Outcomes

Knowledge of Human Cultures and the Physical and Natural World Through study in the sciences and mathematics, social sciences,

humanities, histories, languages, and the arts

Focused by engagement with big questions, both contemporary and enduring

Intellectual and Practical Skills, including Inquiry and analysis Critical and creative thinking Written and oral communication Quantitative literacy Information literacy Teamwork and problem solving

Practiced extensively, across the curriculum, in the context of progressively more challenging problems, projects, and standards for performance

Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty-first-century challenges by gaining:

Personal and Social Responsibility, including Civic knowledge and engagement—local and global Intercultural knowledge and competence Ethical reasoning and action Foundations and skills for lifelong learning

Anchored through active involvement with diverse communities and real-world challenges

Integrative Learning, including

Synthesis and advanced accomplishment across general and specialized studies

Demonstrated through the application of knowledge, skills, and responsibilities to new settings and complex problems

Taken from Association of American Colleges and Universities, “College Learning for the New Global Century,” p. 3.

LEAP Essential Outcomes (cont.)

Inquiry and Analysis

The ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand.

Critical Thinking

Critical thinking is a habit of mind characterized by the comprehensive exploration of issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion.

Creative Thinking

Creative thinking is both the capacity to combine or synthesize existing ideas, images, or expertise in original ways and the experience of thinking, reacting, and working in an imaginative way characterized by a high degree of innovation, divergent thinking, and risk taking.

Written Communication

Written communication is the development and expression of ideas in writing. Written communication involves learning to work in many genres and styles. In can involve working with many different writing technologies, and mixing texts, data, and images. Written communication abilities develop through iterative experiences across the curriculum.

LEAP Value Rubrics

Oral Communication

Oral communication is a prepared, purposeful presentation designed to increase knowledge, to foster understanding, or to promote change in the listeners‘ attitudes, values, beliefs, or behaviors.

Reading

Reading is ―the process of simultaneously extracting and constructing meaning through interaction and involvement with written language‖ (Snow et al., 2002).

(From www.rand.org/pubsresearch_briefs/RB8024/index1.html) Phase Two Report: Working Group on Student Learning Outcomes and Assessment Page C-2

Quantitative Literacy

Quantitative Literacy (QL)-also known as Numeracy or Quantitative Reasoning (QR)-is a ―habit of mind,‖ competency, and comfort in working with numerical data. Individuals with strong QL skills possess the ability to reason and solve quantitative problems from a wide array of authentic contexts and everyday life situations. They understand and can create sophisticated arguments supported by quantitative evidence and they can clearly communicate those arguments in a variety of formats (using words, tables, graphs, mathematical equations, etc., as appropriate).

Information Literacy

The ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand.-The National Forum on Information Literacy

Leap Value Rubrics (cont.)

Teamwork

Teamwork is behaviors under the control of individual team members (effort they put into team tasks, their manner of interacting with others on team, and the quantity and quality of contributions they make to team discussions.)

Problem Solving

Problem solving is the process of designing, evaluating, and implementing a strategy to answer an open-ended question or achieve a desired goal.

Civic Engagement

Civic engagement is ―working to make a difference in the civic life of our communities and developing the combination of knowledge, skills, values, and motivation to make that difference. It means promoting the quality of life in a community, through both political and non-political processes.‖ (Excerpted from Civic Responsibility and Higher Education, edited by Thomas Ehrlich, published by Oryx Press, 2000, Preface, page vi). In addition, civic engagement encompasses actions wherein individuals participate in activities of personal and public concern that are both individually life enriching and socially beneficial to the community.

Intercultural Knowledge and Competence

Intercultural Knowledge and Competence is ―a set of cognitive, affective, and behavioral skills and characteristics that support effective and appropriate interaction in a variety of cultural contexts.‖ (Bennett, J.M. 2008. Transformative training: Designing programs for culture learning. In Contemporary leadership and intercultural competence: Understanding and utilizing cultural diversity to build successful organizations, ed. M.A. Moodian, 95-110. Thousand Oaks, CA: Sage.)

Leap Value Rubrics (cont.)

Ethical Reasoning

Ethical Reasoning is reasoning about right and wrong human conduct. It requires students to be able to assess their own ethical values and the social context of problems, recognize ethical issues in a variety of settings, think about how different ethical perspectives might be applied to ethical dilemmas, and consider the ramifications of alternative actions. Students‘ ethical self-identity evolves as they practice ethical decision-making skills and learn how to describe and analyze positions on ethical issues.

Foundations and Skills for Lifelong Learning

Lifelong learning is ―all purposeful learning activity, undertaken on an ongoing basis with the aim of improving knowledge, skills and competence‖. An endeavor of higher education is to prepare students to be this type of learner by developing specific dispositions and skills (described in this rubric) while in school. (From The European Commission. 2000. Commission staff working paper: A memorandum on lifelong learning. Retrieved September 3, 2003, from www.see-educoop.net/education_in/pdf/lifelong-oth-enl-t02.pdf.)

Integrative Learning

Integrative learning is an understanding and a disposition that a student builds across the curriculum and cocurriculum, from making simple connections among ideas and experiences to synthesizing and transferring learning to new, complex situations within and beyond the campus.

Source: Association of American Colleges and Universities. (2010). Assessing outcomes and improving achievement: Tips and tools for using rubrics. ed., T. L. Rhodes, Washington DC: Association of American Colleges and Universities.

Leap Value Rubrics (cont.)

Definition

Inquiry is the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand. – The National Forum on Information Literacy

Framing Language

This rubric is designed for use in a wide variety of disciplines. Since the terminology and process of inquiry are discipline-specific, an effort has been made to use broad language which reflects multiple approaches and assignments while addressing the fundamental elements of sound inquiry and analysis (including topic selection, existing, knowledge, design, analysis, etc.) The rubric language assumes that the inquiry and analysis process carried out by the student is appropriate for the discipline required. For example, if analysis using statistical methods is appropriate for the discipline then a student would be expected to use an appropriate statistical methodology for that analysis. If a student does not use a discipline-appropriate process for any criterion, that work should receive a performance rating of "1" or "0" for that criterion.

In addition, this rubric addresses the products of analysis and inquiry, not the processes themselves. The complexity of inquiry and analysis tasks is determined in part by how much information or guidance is provided to a student and how much the student constructs. The more the student constructs, the more complex the inquiry process. For this reason, while the rubric can be used if the assignments or purposes for work are unknown, it will work most effectively when those are known. Finally, faculty are encouraged to adapt the essence and language of each rubric criterion to the disciplinary or interdisciplinary context to which it is applied.

Inquiry and Analysis VALUE Rubric

GlossaryThe definitions that follow were developed to clarify terms and concepts used in this rubric only.

•Conclusions: A synthesis of key findings drawn from research/evidence.

•Limitations: Critique of the process or evidence.

•Implications: How inquiry results apply to a larger context or the real world.

Inquiry and Analysis VALUE Rubric (cont.)

Capstone Milestones Benchmark

Topic selection Identifies a creative, focused, and manageable topic that addresses potentially significant yet previously less-explored aspects of the topic.

Identifies a focused and manageable/doable topic that appropriately addresses relevant aspects of the topic.

Identifies a topic that while manageable/doable, is too narrowly focused and leaves out relevant aspects of the topic.

Identifies a topic that is far too general and wide-ranging as to be manageable and doable.

Existing Knowledge, Research, and/or Views

Synthesizes in-depth information from relevant sources representing various points of view/approaches.

Presents in-depth information from relevant sources representing various points of view/approaches.

Presents information from relevant sources representing limited points of view/approaches.

Presents information from irrelevant sources representing limited points of view/approaches.

Design Process All elements of the methodology or theoretical framework are skillfully developed. Appropriate methodology or theoretical frameworks may be synthesized from across disciplines or from relevant subdisciplines.

Critical elements of the methodology or theoretical framework are appropriately developed, however, more subtle elements are ignored or unaccounted for.

Critical elements of the methodology or theoretical framework are missing, incorrectly developed, or unfocused.

Inquiry design demonstrates a misunderstanding of the methodology or theoretical framework.

Analysis Organizes and synthesizes evidence to reveal insightful patterns, differences, or similarities related to focus.

Organizes evidence to reveal important patterns, differences, or similarities related to focus.

Organizes evidence, but the organization is not effective in revealing important patterns, differences, or similarities.

Lists evidence, but it is not organized and/or is unrelated to focus.

Conclusions States a conclusion that is a logical extrapolation from the inquiry findings.

States a conclusion focused solely on the inquiry findings. The conclusion arises specifically from and responds specifically to the inquiry findings.

States a general conclusion that, because it is so general, also applies beyond the scope of the inquiry findings.

States an ambiguous, illogical, or unsupportable conclusion from inquiry findings.

Limitations and Implications

Insightfully discusses in detail relevant and supported limitations and implications.

Discusses relevant and supported limitations and implications.

Presents relevant and supported limitations and implications.

Presents limitations and implications, but they are possibly irrelevant and unsupported.