Minutes of the Ninth AMCOA Meeting, March 26, 2012

Prepared by Kerry McNally

Host Campus: Bunker Hill Community College

I. Attendance

The ninth AMCOA meeting was hosted by Bunker Hill Community College

(BHCC) from 10:00 a.m.-1:00 p.m. on March 26, 2012. Representatives from

22 institutions attended the meeting (See list in Appendix A), and Peggy Maki,

Consultant under the Davis Educational Foundation Grant awarded to the

Department of Higher Education, opened and chaired the meeting.

Peggy thanked BHCC for hosting the meeting.

II. Welcome: Vice President of Academic Affairs & Student Services James F.

Canniff, Bunker Hill Community College (BHCC)

Vice President James F. Canniff welcomed AMCOA Team members to BHCC

and this month’s working session focused on scoring student work that

demonstrates critical thinking. He scanned the list of participants and

commented on the broad representation of faculty coming together over the

most critical of issues – assessment. He was happy that BHCC could host this

session and praised the AMCOA members and faculty for taking the time to

attend this meeting. Faculty and assessment personnel will help student

learning. This is not a fad; it is not going away. AMCOA and SLOAP (Student

Learning Outcomes Assessment Project) at BHCC are programs grounded on

- 1 -

faculty leadership. We need to think about rubrics and methodology. We

need more work in the disciplines, and we are progressing. We have

wonderful leadership from the DHE. Peggy Maki visited all the schools in the

Massachusetts Public Higher Education system, and it is great to have her

leadership and expertise.

Dr. Canniff said that he came to BHCC from New York, where faculty feared

“the test.” He said: “We cannot let that happen here. The faculty cannot let

that happen. The faculty have to be leading this process of developing

rational rubrics for student assessment and improved student learning.

Welcome and thank you.”

III. Initial Discussion about Dates and Times and Foci for Four AMCOA Meetings in

2012-2013, Three Conferences, and Identification of Areas of Assessment You Are

Willing to Address for Campus Visits or Assessment Days: Bonnie Orcutt and

Peggy Maki

A. Survey Monkey

Peggy Maki said that Mo Melvin Sowa created an electronic survey in Survey

Monkey to find out which dates would be best for meetings and conferences

next year. Response was very light; there were only 14 replies, which is too

small to make a consensus. The survey will be kept open. Peggy asked the

Team to please reply to it by April 20th, and then she will report the results to

the Team at the May meeting.

B. Form to Identify Areas of Expertise for Campuses

Peggy distributed a rough draft of recommendations, resources, and thorny

issues that Bonnie Orcutt assembled based on AMCOA team members listing

- 2 -

potential ideas. Bonnie asked that people think of teams that might visit

campuses; alternatively, she can look at responses and suggest teams. Since

there were no suggestions for changes from the group at the meeting, Bonnie

will prepare a form to send out to AMCOA team members asking them to

identify their areas of expertise. This form should be returned to her by April

20th.

C. Initial Plans for Next Academic Year’s Meetings

Peggy is working with Worcester State University and Framingham State

University to have them serve as centralized locations for next year’s AMCOA

meetings. In the Survey Monkey responses she has seen, Mondays and

Fridays are the most open days to hold meetings and conferences among the

people who responded. She again emphasized that team members should

respond to the survey by April 20th.

IV. Update on the Fourth Statewide Assessment Conference: Jim Gubbins

Jim Gubbins announced that the electronic registration site for the April 23rd

AMCOA Conference at UMass Boston is open and requested that people

register. He said that there would be two sets of five presentations in the

morning and that the campuses that received funding for assessment

experiments would report on a panel to the full conference in the afternoon.

His understanding is that all or a large number of the community college

presidents plan to attend. The Boston Globe may also be there to report on

the event. Pat Crosson will close the conference with a talk on “The year that

Lies Ahead.” There are 200 seats available in the Ballroom at UMass Boston’s

Campus Center for conference attendees and free parking for the first 100

registrants. He again reminded presenters, as well as attendees, to register.

- 3 -

V. Summaries of Two Assessment Experiments: Ellen Wentland, Associate Dean of

Academic & Institutional Effectiveness, Northern Essex Community College, and

Kristina Bendikas, Associate Dean for Assessment and Planning, Massachusetts

College of Liberal Arts

Elise Martin from Middlesex Community College was on the Agenda to present

this report, but was not able to attend. Ellen Wentland from Northern Essex

Community College, one of the other campuses participating in the

experiment, agreed to speak about their progress.

“Using Assessment to Develop Interdisciplinary Writing Standards Across

College Levels: A Collaborative Model for Two- and Four- Year Institutions”

The University of Massachusetts Lowell (UML along with two feeder

community colleges, Middlesex Community College (MBCC), and Northern

Essex Community College (NECC), are collaborating on an assessment

experiment that focuses on transfers from 2-year schools to UML, particularly

in the three majors of Criminal Justice, Ecology, and Business based on scoring

of students’ written work. There might be some variation in outcomes

depending on the program, but they are comparing students from the same

majors. NECC and MBCC are looking at students with 15-29 credits at the

end of the first year, and 45-59 credits at the end of the second year. UML

will look at students with the same credit-hour levels and majors, but also the

junior year students.

- 4 -

Using the VALUE rubrics on an existing assignment, institutions are reviewing

students with the same credit hours. When the student work samples are

used, the raters will not know which school or major is involved, although

they may be able to guess the major by the subject matter of the paper.

The institutions are now having a norming session this week to look at the

rubric and adjust it. Once there is some consensus on the rubric, they can

proceed to rate the work product. The goal is that no matter the major,

faculty and administrators can establish norms for particular credit hours and

set standards for end of first year and second year writing performance with

respect to skills achievement of University students.

“Writing Assessment Experiment: Massachusetts College of Liberal Arts,

Berkshire Community College”

Faculty from Massachusetts College of Liberal Arts (MCLA) and Berkshire

Community College (BCC) have been working with Dr. Kathleen Yancey, a

nationally known expert in college writing, to develop scoring guides for

written communication at the sophomore/AA level and for the

senior/bachelor’s level. Both groups will review student artifacts, as well as

NSSE/CCSSE data, and determine action steps for coordinating writing

expectations within their own institutions, and increasing student success in

writing for those that transfer. The experiment also includes the creation of

an intranet or internet sites at MCLA and BCC where students can go to access

and view the writing rubrics and models of student work.

- 5 -

Kristina Bendikas was on the Agenda to speak about this experiment, but did

not arrive from Western Massachusetts until later in the meeting, so she was

not able to present her report. However, she has given us a written update

below:

On February 28 each of the taskforces (MCLA-BCC and MCLA) worked for a

half-day with Dr. Kathleen Yancey on determining values that would be

incorporated into a rubric for the assessment. Members reviewed samples of

student work and discussed the merits of criteria and formats for their scoring

guides. There was more agreement in the MCLA-BCC group than in the MCLA

group where disciplinary perspectives came into play. Each of the taskforces

have meetings scheduled to further develop their rubrics (MCLA on March 27,

MCLA-BCC on April 2) while the samples are being collected.

VI. Summary of February Worksheets (posted on Yammer as of March 12, 2012)

Peggy Maki summarized the worksheets from the February 29th workshop on

written communications. These summaries were posted on Yammer on

March 12, 2012.

Some of her findings are:

Many people said that there should be a “0” column, which Peggy said

was a good idea and could easily be incorporated.

The institutional rubrics are much more specific whereas the VALUE

rubric is a meta-rubric. Perhaps as we continue to use the VALUE

- 6 -

rubric, we may need to train raters with the meta-rubric supported by

more detail about the criteria encompassed within each of the VALUE

components.

Having consulted with AAC&U about the typical audience for student

work that we score, Peggy reported that AAC&U said that audience

should be “an educated person walking down the street” because we

are assessing at the level of general education. Further, AAC&U stated

that disciplines are refining or adapting the writing VALUE rubric based

on a more in-depth disciplinary assessment of student work.

The results of the writing workshop indicate that people scored papers

similarly—at the lower end of performance based on the VALUE rubric

and even on the institutional rubrics used.

Tom Curley, Berkshire Community College, said that his school is interested in

the 2nd level, but they are also looking at the 4th level. At the 4th level they

look more rigorously at ability to write, etc.

VII. May 1 AMCOA Meeting at Framingham State University

Regarding the May 1st Meeting at Framingham State University, Peggy Maki

announced that she is inviting six to eight teams with representatives from

different campuses to speak on how they are assessing Quantitative

Reasoning. This will create a good grounding on how Quantitative Reasoning

is being evaluated on the campuses. She asked Team members to volunteer

themselves for this panel discussion. She said that the Team can further

discuss this topic in the fall. At issue is reaching agreement about what we

are seeking as evidence of QR in exit-level work. Further, she stated that she

- 7 -

intends to invite a representative from AAC&U to meet with the AMCOA team

in Fall, 2012.

At the May 1st Meeting Pat Crosson will discuss with the Team the year ahead

regarding the Davis Grant.

Finally, there will be some formal planning for next year at the May 1st

meeting.

VIII. Introduction of Faculty Participating in Today’s Working Session and Group

Leaders; Orientation to Today’s Working Session Focused on Assessing Exit-Level

Critical Thinking: Peggy Maki

Peggy reviewed AAC&U’s Critical Thinking (CT) VALUE rubric in preparation for

scoring student work at the meeting. She noted that numerous teams

composed of both four-year and two-year faculty and other educators were

involved over two years in reaching consensus about the criteria included in

the CT rubric. The rubric addresses the range of thinking that is involved in

CT, including analysis, synthesis, and evaluation.

Once AMCOA team members and invited faculty scored a student example,

Natalie Oliveri, BHCC, distributed and described the scoring rubric developed

at BHCC; then, team members applied that to the same sample they scored

using the CT VALUE rubric.

Peggy said that she will summarize results of scoring student work using both

the VALUE and the BHCC CT rubric and post the results on Yammer for the

- 8 -

Team. Going forward Peggy will be seeking input from faculty and

assessment professionals on the campuses.

Peggy introduced the group leaders and assigned participants to groups:

Group 1: Lisa Plantefaber, Westfield State University, served as Group LeaderBill Berry, Cape Cod Community CollegeMay Callahan, Northern Essex Community CollegeJohn Cunningham, University of Massachusetts Presidents’ OfficeKate McLaren, Massachusetts Maritime AcademyTom Curley, Berkshire Community College

Group 2: Ellen Zimmerman, Framingham State University, served as Group LeaderKate Finnegan, Greenfield Community CollegePeter Johnston, Massasoit Community CollegeHolly Noun, Westfield State UniversityTom Curley, Berkshire Community CollegeMartha Stassen, University of Massachusetts Amherst

Group 3: Susan Chang, Framingham State University, served as Group LeaderMark Patrick, Massachusetts Maritime AcademyCharles Prescott, Berkshire Community CollegeDawne Spangler, North Shore Community CollegeNatalie Oliveri, Bunker Hill Community CollegeDonna Kuizenga, University of Massachusetts Boston

Group 4: Bonnie Orcutt, Worcester State University, served as Group LeaderSeverin Kitanov, Salem State UniversityElizabeth Johnston O’Connor, Cape Cod Community CollegeJohn McColgan, Roxbury Community CollegeRuth Slotnick, Mount Wachusett Community CollegeJudy Raper, Greenfield Community CollegePaula Haines, University of Massachusetts Lowell

Group 5: Yves Salomon-Fernandez, MassBay Community College, served as Group LeaderKristina Bendikas, Massachusetts College of Liberal ArtsJavad Moulai, Roxbury Community CollegeSusan Taylor, Mount Wachusett Community CollegeEllen Wentland, Northern Essex Community College

- 9 -

David Leavitt, Bunker Hill Community CollegeMary Fowler, Worcester State University

Group 6: Carol Lerch, Worcester State University, served as Group LeaderStephen Sutherland, Presenter of UMass Boston RubricTim McLaughlin, Bunker Hill Community CollegeJudy Turcotte, Holyoke Community CollegeMargaret Stephenson, Berkshire Community CollegeSaradha Ramesh , North Shore Community CollegeJames Gubbins, Salem State University

- 10 -

Appendix A: Institutions Represented at the AMCOA March 26th Meeting:

Berkshire Community College

Bunker Hill Community College

Cape Cod Community College

Fitchburg State University

Framingham State University

Greenfield Community College

Holyoke Community College

Massachusetts College of Liberal Arts

Massachusetts Maritime Academy

Massasoit Community College

MassBay Community College

Mount Wachusett Community College

North Shore Community College

Northern Essex Community College

Roxbury Community College

Salem State University

University of Massachusetts Amherst

University of Massachusetts Boston

University of Massachusetts Lowell

University of Massachusetts President’s Office

Westfield State University

Worcester State University

- 11 -