Minutes of the Second AMCOA Meeting, June 27, 2011

Prepared by Kerry McNally

Host Campus: Northern Essex Community College

I. Attendance

The second AMCOA meeting was hosted by Northern Essex Community College from 10:00 a.m.-12:00 Noon on June 27, 2011. Representatives from 26 institutions attended the meeting (See list in Appendix A), as well as , Peggy Maki, Consultant under the Davis Educational Foundation Grant awarded to the Department of Higher Education, who also chaired the meeting, and Kerry McNally, Administrative Assistant for the AMCOA project.

II. NECC President-to-be Lane Glenn Opening Remarks

Dr. Glenn welcomed the AMCOA Project Team and stressed the importance of the work that the Project encompasses. He is knowledgeable about learning outcomes and acknowledged that there are challenges of looking at them as a university system versus locally. Assessment is important work, and critical thinking an important goal of an education. Although the goals and changes may at times seem daunting, Dr. Glenn stated that the University system should “continually be moving in the direction of the ideal,” quoting Richard Paul.

III. Peggy introduced Kerry McNally to the AMCOA Team, as the AMCOA Project’s new Administrative Assistant.

1

IV. Update and feedback on September 30th Conference planning – Kris Bendikas, Chair

Peggy asked Team Members to invite colleagues, especially faculty, to attend the September 30th Conference. Peggy said that she will notify the Presidents and Vice Presidents.

Kris Bendikas gave her presentation of the Conference’s Planning Recommendations, a copy of which is attached to these minutes as Appendix B, along with Ellen Wentland’s suggested outcomes for the September 30th Conference, Appendix C. Kris emphasized that partnerships should be overarching for all conferences and recommended concurrent partnership-type sessions.

Peggy agreed to be the keynote speaker for the Conference and any concurrent sessions.

Kris said that it is critical that the LEAP program be explained to participants at the Conference. Rumors are circulating that Massachusetts is becoming a LEAP state, which may not be true, and if it is, what does it mean?

Kris further recommended collaborative models for faculty and IR assessment staff.

Based on the recommendation of the conference planning group, Peggy asked if the AMCOA Team felt she should invite Jonathan Keller from the Department of Higher Education to speak about his work at an AMCOA Team meeting or at the Conference. The group agreed that she should.

2

Kris brought up the fact that NSSE and CSSE data with other data are being used on campuses. Could something like the Keene State Model be done in Massachusetts?

Some additional topics included:1. Models that link curricular and co-curricular learning2. Articulation of Achievement Levels3. Gainful Employment Reporting

In conclusion, we need to get baseline information on what people are doing now, focus on collaborations between faculty and assessment people, discuss what LEAP means, and define how to use learning outcomes to make changes.

Peggy polled Team Members to see which aspects of the Planning recommendations proposal should be adopted. Additional Team comments are listed below:

1. The Team agreed that partnership and community are essential in the process of improving assessments.

2. The Team agreed to the idea of concurrent sessions.3. Team members said that people are not clear on what LEAP

means and that education about what it means at the IR and faculty levels is critical. Presidents, Chief Academic Officers, Faculty, and IR people should all be made aware of it. There should be discussions about it with faculty, academic affairs officers and the co-curricular side. Peggy emphasized that LEAP does not prescribe curriculum; rather it identifies the 15 most prevalent learning outcomes for General Education that arose from a survey of institutions across the US. An institution determines how it addresses some of these outcomes in its core or General Education based on its mission and purposes.

4. Members feel that campuses need an introduction to the process of addressing LEAP outcomes and using VALUE rubrics

3

and will need time to adjust to the LEAP concept to see who will participate. In fact, several AMCOA members stated that it is important to explain how LEAP emerged since many people do not understand its origin as the work of AAC&U. Some faculty members equate LEAP with NCLB (No Child Left Behind) and dismiss the idea, so more work needs to be done with faculty to help them understand that LEAP provides a national framework for identifying and assessing the most common learning outcomes that US higher education identifies it prepares students to achieve.

5. Peggy asked if any of the Team Members were using LEAP on their campuses. Of the institutions represented at the meeting, Holyoke Community College, Greenfield Community College, Salem State, and Worcester State are using or beginning to use LEAP on their campuses. The AMCOA Team stated it would be useful to have Team Members from these campuses describe their experience with LEAP.

6. One Team Member commented that IR/Assessment people are interested in the assessment process, how it is reported and done, but faculty are not. The IR/Assessment perspective is more aggregate, while the faculty perspective in evaluating students is more individual.

7. Faculty should have a presence in the planning process because of what they can learn.

8. One Team Member said that there is a separation between faculty who are using the LEAP process and IR assessors who are doing it. This does not work if they are not doing it together. What works on some campuses that could grow on others? IR offices are often viewed as data receptacles.

9. One Team Member feels that we should be focusing on where we are and not the “Beyond.” Beyond signifies change, learning a new process, more work, etc. Defining where we are feels more comfortable for now.

4

10. It is important to inform faculty about changing processes early on. Faculty express anxiety about programs imposed from the top down; thus they need to be included in the process.

11. “What the faculty wants for assessment” would be more inviting for a faculty meeting than organizing a LEAP meeting for them.

12. In the Working Group Report, there is trend data.13. Peggy stressed that LEAP does not impose a curriculum on a

campus or dictate pedagogy; rather it identifies the 15 most prevalent outcomes that campuses across the US address in their GE or core curriculum. Peggy is willing to lead a panel to explain LEAP.

14. “Liberal Education and America’s Promise (LEAP)” would be a good info session for faculty.

15. Make LEAP part of a collaborative model, rather than separate.16. It is important to distinguish among the kinds of reports that IR

prepares—from retention data to student performance data.17. It is important to present assessment changes to faculty. How

do we present developments to them?18. The issue of using commercially developed software for

assessment was also raised. Peggy stated that there are at least 15 commercial designers of assessment management systems several of which incorporate eportfolio capacity. A session on this would be useful for faculty and IR professionals to learn about as a way to centrally store assessment data and derive evidence of student learning.

V. Dates

a. AMCOA meetings. Below are the dates for future AMCOA meetings for fall, 2011. Sites have not been determined yet. Peggy requested members to consider hosting future meetings.

i. No July meetingii. Thursday, August 18th

5

iii. Wednesday, September 14th (perhaps at a central location)iv. Tuesday, October 18th

v. Wednesday, November 9th

vi. No December meeting for now. Peggy will revisit the need for this month’s meeting in the fall.

b. Three Statewide or regional conferences. Sites have not been determined.

i. Thursday, November 17th. It would be good if this were in either the Eastern or Western Region.

ii. Thursday, February 9th. This should be in the opposite region from the November meeting.

iii. Monday, April 23rd.

VI. Chairs and Planning Committees for three statewide or regional conferences. Stipend for Chairs.

Peggy will e-mail members to see which campuses can host the future meetings and conferences and who might be interested in chairing each of the three other conferences planned for 2011-2012.

VII. A National Development in Assessing Student Learning

Peggy distributed a chart and written description of the proposed Undergraduate Degree Profile developed by for work students should produce at the end of their associate’s, bachelors’ and master’s levels. The UDP is currently being piloted in two accrediting associations and the Council of Independent Colleges; it is informed by LEAP.I specifically identifies 5 areas of student performance: applied knowledge, intellectual achievement, specialized knowledge, broad integrative knowledge, and civic learning. Peggy issued a caveat that the pilot program is still being tested, so it is unknown how it will pan out. In

6

particular, the focus on students’ knowing a second language may be an issue since that is not a requirement at many institutions, Others have interpreted “a second language” as referring to non-native learners who need to demonstrate knowledge of English.

VIII. Foci of First Statewide Report on Outcomes

Peggy reported that Commissioner Freeland has requested that institutions initially report on students’ achievement under general education in three areas: critical thinking, writing, and quantitative reasoning because these are the most common outcomes that institutions prepare students to demonstrate.

IX. Foci of Future AMCOA Meetings

Peggy stated that we may use future meetings to focus on specific activities or topics such as discussing the kinds of assignments and student responses to those assignments that demonstrate students’ achievement of critical thinking, writing, and quantitative reasoning at the end of their studies. Based on our further discussions we may decide to offer sessions at our conferences that bring people together to discuss assignments and perhaps score them. One of the Team reminded participants to obtain students’ permission for using their work at AMCOA meetings/conferences to respect their privacy rights. There may be other foci that AMCOA members would like to address at our monthly meetings. Given that time ran out, we will return to this topic at our next meeting.

The next AMCOA meeting will take place on August 18th from 10:00 a.m.-12:00 Noon. A campus has not yet been selected for this meeting. Peggy Maki will send an agenda and driving directions to the site one

7

week before the meeting. Please let Kerry know as soon as possible if you plan to attend the meeting or cannot.

Mark Your Calendars: September 30, 2011: First Statewide Assessment Conference at Worcester State University

8

Appendix A

Institutions Represented at the AMCOA June 27th Meeting:

Berkshire Community CollegeBridgewater State UniversityBunker Hill Community CollegeCape Cod Community CollegeFitchburg State UniversityFramingham State UniversityGreenfield Community CollegeMassachusetts College of Art and DesignMassachusetts College of Liberal ArtsMassachusetts Maritime AcademyMassasoit Community CollegeMassBay Community CollegeMiddlesex Community CollegeMount Wachusett Community CollegeNorth Shore Community CollegeNorthern Essex Community CollegeQuinsigamond Community CollegeRoxbury Community CollegeSalem State UniversitySpringfield Technical Community CollegeUniversity of Massachusetts Amherst University of Massachusetts Boston University of Massachusetts LowellUniversity of Massachusetts President's OfficeWestfield State UniversityWorcester State University

9

Appendix B

AMCOA Planning Group recommendations

“Partnerships” should be an overarching theme for all four conferences. o There are a number of partnerships that need to occur as part of assessment:

academic -co-curricular, faculty-IR and Assessment, campus-community., etc.o This theme is recommended as being of importance to achieving our individual

and collective assessment and improvement goals.

If possible, the conference should be organized with concurrent sessions so that everyone can attend all the sessions.

The conference should begin with a Keynote address (Peggy?) to frame the theme of “partnership.”

Discussion session: “LEAP and Beyond” o Include an initial discussion about LEAP and the Lumina Degree Profile at this

conference, with follow up in subsequent conferenceso The BHE has recommended that Massachusetts become a LEAP state, but there is

still uncertainty about what that means for campuses and the system

Presentation Session: Collaborative models for faculty-IR-assessmento These roles require very distinctive skills sets. How can they work together most

productively? What collaborative models are working for campuses? o What are the particular difficulties on smaller campuses when two or more roles

are filled by the same person?

Discussion session with Jonathon Keller, DHE o An invitation for him to speak about the challenges of mining and reporting data,

particularly in the context of the expectations of the Vision Project

Presentation session: Identifying models for using NSSE/CSSE data o How can this data be triangulated with other data to support change? What

models have campuses used to mine, review and report on this data?

Presentation session: Reviewing the selection of commercially developed assessment instruments as well as data gathering, analyzing and reporting instruments available

10

o Although the Working Group has not endorsed standardized testing as part of the Vision Project, campuses that want to use commercially developed tools have a multitude of choices.

o Which ones have been used? Which ones have been supported by faculty? How well have they worked?

Additional topics were generated, but were thought more suited to the conferences that will involve faculty more directly. They are:

Developing models that would link curricular and co-curricular learningo Describing Best Practice models of collaborationo Identifying ways to increase collaboration and learning about the benefits of

doing so.

Articulating Achievement Levelso Determining what a shared outcome would look likeo Exploring the pros and cons of moving towards shared criteria/benchmarkso Developing a process for moving that forward that with another institution

Gainful Employment Reportingo The requirements for reporting are not yet clear, but sharing information about

what and how information could be collected to serve this and other assessment purposes at the same time would be helpful.

11

Appendix C

AMCOA Conference, September 30, 2011

EW Suggested Outcomes – 6/21/11Session ideas Details from Yammer feed Possible outcomes - Participants will be able to:Collaborative models in MA In what ways can these groups work together? What models or

examples of collaboration exist in Massachusetts? I have a sense that the assessment component, on many campuses, is the newer piece. How can that work be informed by IR data, and possibly vice versa? What might be an example, given the very specific tasks that must be accomplished by IR, of a "best practices" IR/ assessment partnership? For us to be able to report results of assessment we will need to agree on shared criteria and standards of judgment.

Identify and describe models of IR/ Assessment collaboration that support and enhance the effectiveness of campus outcomes assessment efforts.

Apply some of the tools and processes identified in the successful collaborative models to strengthen IR/ Assessment connections towards the goal of expanding and coordinating information sources relevant to outcomes assessment and evaluation.

Articulating achievement levels

Very related to this scoring issue is the issue of criteria or articulation of achievement levels. Although I think a first step for each institution, for the purpose of creating ownership, is the identification of their own outcomes and of description what those outcomes should "look like", it is also the case that most of us will define similar outcomes and similar levels of desired achievement. Do we want to move towards shared criteria/benchmarks for student expand   »

Develop a plan for identifying and/or creating institutional-level clear, specific and measurable learning outcomes.

Identify for each developed institutional-level outcome the specific assessment methods to be used and the achievement criteria that will be applied.

Prepare a summary description of the institutional level work for the purposes of cross-institution sharing and collaboration.

Discuss the pros and cons of developing shared criteria and benchmarks.

Gainful employment reporting

One issue facing all of us is the issue of "gainful employment," and the requirement that for financial aid we will need to somehow document the employment of our students. This may fall to IR offices, or assessment offices, or other entities.

Discuss the issues and the specifics involved in the financial aid requirement

Determine how to best institutionally address this requirement in a way that serves multiple purposes and contributes to outcomes assessment work

Triangulating NSSE/CSSE data with other indicators

We could look at NSSE/CCSSE data and discuss ways to triangulate this data with other indicators of student achievement on our campuses.

Discuss some of the specific items included in the NSSE/CCSSE instrument with respect to their relationship with other campus outcomes measures or achievement indicators.

Identify possible models that would provide a way

12

to include and consider the information gathered from multiple indicators/ methods.

Linking curricular and co-curricular learning

Another topic for a conference for Assessment Directors, IR folks, and faculty might focus on linking of curricular and co-curricular learning towards more comprehensive student achievement of an institution's Gen Ed SLOs.

Describe models of collaboration between the curricular and co- curricular areas that have demonstrated effectiveness in terms of campus-wide learning outcomes assessment.

Identify ways to increase curricular and co- curricular collaborations, thereby enhancing the institutional – level assessment efforts.

Describe the benefits of these types of institutional collaborations/partnerships.

Roundtable with a speaker Possibly inviting someone (Jonathan Keller?) to update group and answer questions re data collection/comparison for VISION project

Describe various types of data collection and comparisons relevant to the VISION project.

Open forum Also would it be useful for this first conference to have a problem-solving session--one during which people identify an issue or problem they may be facing, such as how to represent assessment findings that trigger discussions about pedagogy, and those of you who are on the planning committee propose strategies to address those problems? I see these conferences as opportunities to connect people across campuses and identify the human resources and expertise that exist in the state.

Describe the issues or problems faced across the different campuses with respect to outcomes assessment.

Identify some of the common issues or problems being faced, for use in future conference planning related to possible strategies.

Identify strategies used by campuses to address some of the articulated issues/ problems.

“Effective Practices in..." 

13