Minutes of the Eighth AMCOA Meeting, February 29, 2012

Prepared by Kerry McNally

Host Campus: Northern Essex Community College, Haverhill Campus

I. Attendance

The eighth AMCOA meeting was hosted by Northern Essex Community College (NECC), Haverhill from 10:00 a.m.-1:00 p.m. on February 29, 2012. Representatives from 22 institutions attended the meeting (See list in Appendix A), and Peggy Maki, Consultant under the Davis Educational Foundation Grant awarded to the Department of Higher Education, opened and chaired the meeting.

Peggy thanked NECC for hosting the meeting.

II. Welcome: President Lane Glenn, Northern Essex Community College

President Glenn welcomed AMCOA Team members to this month’s working session focused on scoring student work that demonstrates written communication. He congratulated members on their progress and emphasized his commitment to this project.

III. Updates and Foci of Next Two Working Sessions: Peggy Maki

Peggy asked the group to please sign up for leading a group, contributing an institutionally developed scoring rubric that demonstrates critical thinking or quantitative reasoning, or submitting an exit-level student sample.

IV. Update on the Fourth Statewide Assessment Conference: Jim Gubbins

Jim reported that he has received three or four new proposals in addition to the planned descriptions of the assessment experiments. He also stated that he would like campuses that haven’t been represented before at conferences or meetings to step up and present. His planning group, consisting of Chris

Cratsley, Neal Bruss, Donna Kuizenga and Dawne Spangler, will have its first Skype meeting next week. (Appendix B is the Call for Proposals for the April 23rd AMCOA conference.)

Requests for more Assessment 101-type sessions were spread across the Comments in the February 9th Evaluations. Peggy Maki volunteered to give sessions on beginning assessments. The question was raised as to what Assessment 101 means. Peggy said that it would entail very basic terminology and overarching assessment principles and practices. She suggested that she would invite institutional reps to share their approaches in this basic session.

V. Summaries of Two Assessment Experiments: Susan Chang, Director of Assessment, Framingham State University, and Joanne M. Preston, Dean of Humanities, Bristol Community College

Framingham State University (FSU) and MassBay Community College are collaborating on an assessment experiment that aims to identify ways to improve the academic experience and success of students who transfer from a community college to a 4-year institution by using assessment tools. While there is an emphasis on a transfer student’s academic success, there is a recognized need to collect qualitative data to understand how the overall college experience of a transfer student can be improved. Some of the transfer students reported feeling not connected to the campus, citing lack of knowledge about resources. There are six primary objectives of the experiment: (1) continue to build upon assessment processes already in development at MassBay and FSU; (2) identify strengths and weaknesses in the General Education curriculum at MassBay and FSU; (3) utilize demographics for a seamless transition and identify indicators of student success; (4) explore technology as a cross-campus assessment platform; (5) influence and inspire changes to the curriculum; and (6) create and continue a partnership between a two- and four-year institution.

Some of the progress they are making and some of the issues that are emerging include:

Conversations at the administrative level that don’t trickle down to the faculty. That needs to be improved.

Exploration of students’ strengths and weaknesses in written communication, quantitative reasoning and critical thinking are.

Identification of indicators of student success. Review of different kinds of assessment software applicable to

documenting 2-year and 4-year students’ achievements

The project is starting with the AAC&U VALUE rubrics, asking if they will work. FSU has an assessment advisory group and they know where they are going. Are these rubrics appropriate for both 2-year and 4-year schools? MassBay is reviewing these rubrics to adopt for their first assessment cycle.

The first assessment cycle will provide data to identify strengths and weaknesses in the General Education curriculum.

FSU is collecting qualitative data to complement the quantitative information they normally gather. They are establishing benchmarks and numbers. For example, in Fall 2008 16% of FSU transfer students came from MassBay Community College. Of those who transferred from MassBay, 50% of them graduated from FSU as of Fall 2011, compared to 39% of the total transfer cohort.

Yesterday, FSU looked at software with people from MBCC. They need information for both schools to make informed decisions by the end of this calendar year. They want to inspire improvement, but they need the data.

The discussion of STEM rolled from there. They are working with PARCC and collaborating on improving developmental courses in math. They are also looking at a potential bridge program. And, they are working on improving the academic experience for transfer students, for example, an orientation program that is designed for them.

After reviewing the first timeline, they developed a second one to give faculty more time to think about the process. By May 2013 they will look at the overall findings and start the next assessment cycle.

For a full description of the FSU and MassBay experiment progress and future timeline, please see a copy of the PowerPoint presentation in Appendix C.

The CONNECT AMCOA Assessment Experiment will engage faculty from the CONNECT Consortium to learn about and apply the Liberal Education and America’s Promise (LEAP) VALUE rubric for written communication and support their exploration of the LEAP Essential Learning Outcomes framework.

The Assessment Experiment will evaluate common assignments using the Writing Rubric used at Massasoit Community College, the Writing Rubric used at Bristol Community College and the LEAP VALUE rubric to compare and contrast the three rubrics. There are three assessment teams which are each made up of two instructors who will assess writing using the rubrics. Two of the teams are Institutional Scoring Teams (two faculty from BCC) and (two faculty from MCC). These team members are instructors who teach a comparable first general education course that emphasizes composition (English 101). The third team, the Central team, consists of one BCC faculty member and one MCC faculty member.

The common assignment was administered to 4 writing classes at MCC and 4 writing classes at BCC. The faculty who volunteered to use the common assignment in their course, received a small stipend after turning in the writing. JP Nadeau, Professor at BCC, created a random sample of 25 assignments.

The two Institutional Scoring teams will use their college’s rubric: the MCC team will use the CONNECT Rubric used at MCC and the BCC team will use the rubric used at BCC to assess the stratified random sample of 25 papers. After scoring with their college rubric, the two teams will use the LEAP VALUE Rubric to score several papers not from the original sample of 25 to gain some experience with the VALUE rubric. The two teams will then use a structured guideline to document their experiences using the LEAP Rubric noting the similarities and differences in the rubric when compared with their local

rubric and their perceptions of barriers and facilitators to using the LEAP VALUE Rubric.

Afterwards, the Central Team, comprised of two composition instructors selected, will assess the same 25 papers using the LEAP VALUE Rubric. Each member of the Central Team will then score several additional papers using their own institution’s rubric and will follow the same structured guideline to document differences and similarities across rubric sets and identify any challenges and facilitators to using the LEAP Rubric in comparison with their own rubric.

The project will be completed by the end of May and the write-up of the Experiment will be completed by the end of June. Individuals involved in this project include:

JP Nadeau is the Project Coordinator for the Experiment The BCC Team is Farah Habib and Michael Geary The MCC Team is Melissa Winchell and Rita Jones-Hyde The Central Team is Deb Anderson (BCC) and Susan Keith (MCC)

VI. Summary of New Davis Proposal: Pat Crosson, Senior Advisor for Academic Policy

Pat described the current Draft AMCOA II Proposal (Feb. 27, 2012) as addressing three components:

Component A, system-wide learning outcomes assessment focused on developing a system-wide plan, including the development of models, metrics, dashboards, and implementation plans. She noted that the concept of a composite indicator has been taken off the table. Component A work, therefore, will need to identify other kinds of evidence we would include in a system-wide assessment reporting approach that uses multiple sources of evidence about student learning. Within this component there will be two new task forces and those task forces will need to connect with the AMCOA team as well.

Component B, support for campus assessment based on initiatives from Phase 1 and the need to expand the good work of AMCOA across our campuses through such activities recommended by co-chairs and AMCOA team members as conferences, experiments, on-call help teams, “thorny Issues” workshops, and continued use of the project consultant on a more limited basis next year.

Component C, expanded involvement that seeks to reach a larger faculty and staff audience to promote engagement with learning outcomes and assessment and encourage commitment to using results for program improvement. The proposal seeks funds for a web-based assessment repository and tool kit, use of AMCOA team members to present at campuses’ assessment days, and creation of opportunities for faculty and staff to look together at the ways that a total campus environment and student engagements with the curriculum and co-curriculum affect learning for all students.

Pat explained that the current draft requests support for travel and support for some release time for roles, such as for the co-chairs, since individuals will be taking on additional responsibilities this next year.

Pat asked for comments from the AMCOA team about the proposal and hoped that they saw their ideas integrated into the draft. She also stated she could receive comments from team members or institutions until this coming Monday. Shortly thereafter she needs to send out the proposal. She asked representatives to let their campus leaders know about their support for this proposal as well. She stated that unless there is support from campuses, Davis is unlikely to fund the proposal. The current copy is marked confidential because not all presidents have received it or had a chance to respond. It can be discussed on campus, but there is such a short time between when it is due and when comments can be incorporated into the current draft.

Pat’s Draft AMCOA II Proposal-2/27/2012 is attached as Appendix D.

VII. Introduction of Faculty Participating in Today’s Working Session and Group Leaders; Orientation to Today’s Working Session Focused on Assessing Exit-Level Student Writing: Peggy Maki

Peggy said that developing the VALUE Rubrics has taken hundreds of experts and years of work. They were created as a way to develop and use a common language across the states. The VALUE rubrics are more general than many institutionally developed rubrics. Conversely, many institutionally developed rubrics are more holistic than the AAC&U rubrics. What we are trying to do, she stated, is see how well institutionally developed rubrics align with these nationally developed rubrics—even though they may be more detailed or more holistic. Peggy also stated that creating these rubrics has become an alternative way to assess student work – as opposed to using standardized tests as the sole means of reporting student achievement.

Peggy introduced the group leaders and presenters:Group 1: Chuck Prescott, presented Berkshire Community College’s rubric

Elise Martin, Middlesex Community College, served as Group LeaderGroup 2: Ellen Wentland presented Northern Essex Community College’s

rubric and served as Group LeaderGroup 3: Suzanne Van Wert presented Northern Essex Community College’s

rubricMark Patrick, Mass Maritime Academy, served as Group Leader

Group 4: Jennifer Arner Welsh presented Quinsigamond Community College’srubricBonnie Orcutt, Worcester State University, served as Group Leader

Group 5: Neal Bruss presented UMass Boston’s rubricMartha Stassen, UMass Amherst, served as Group Leader

Group 6: Chris Cratsley presented Fitchburg State University’srubricPaula Haines, UMass Lowell, served as Group Leader

Group 7: Tim McLaughlin presented Bunker Hill Community College’srubricJames Gubbins, Salem State University, served as Group Leader

Appendix A: Institutions Represented at the AMCOA February 29th Meeting:

Berkshire Community CollegeBristol Community CollegeBunker Hill Community CollegeCape Cod Community CollegeFitchburg State UniversityFramingham State UniversityGreenfield Community CollegeMassachusetts Maritime AcademyMassasoit Community CollegeMassBay Community CollegeMiddlesex Community CollegeMount Wachusett Community CollegeNorthern Essex Community CollegeQuinsigamond Community CollegeRoxbury Community CollegeSalem State UniversityUniversity of Massachusetts AmherstUniversity of Massachusetts DartmouthUniversity of Massachusetts LowellUniversity of Massachusetts President’s OfficeWestfield State UniversityWorcester State University

Appendix B: Call for Proposals for the April 23rd AMCOA Conference

Appendix C: “Two- or four-year institution … doesn’t matter: Student success is student success,” a collaborative PowerPoint presentation by Susan Chang, Ellen Zimmerman and Yves Salomon-Fernandez outlining the progress of the Framingham State University and MassBay Community College assessment experiment.

(Please double-click the image below to open the presentation. Then, click once to move from one page to the next.)

Appendix D: DRAFT AMCOA II Proposal-2/27/2012 for AMCOA Team (Double-click image below, then, use down-arrows to move through pages.)