68
Introduction The purpose of this document is to compile various assessments in the required areas of general education at Buffalo State so that instructors, staff and administrators can review strengths and weaknesses in student performance and discuss ways to promote student learning. General Education is called “Intellectual Foundations” at Buffalo State and will be so called in this document. Multiple measures are used to assess the various areas of Intellectual Foundations which include analyses of student written work using rubrics, objective tests, standardized and benchmarked national assessments, surveys, etc. For lack of a better term, these will be called “dashboards” but that is not to imply that student learning can be captured solely by quantitative measure. Rather this is an attempt to present facts gleaned from several assessments and to encourage evidence-based conversations that lead to recommendations, contribute to Intellectual Foundations program review and ultimately contribute to enhancing student learning. Everyone is encouraged to review these dashboards and use the information contained within in whatever ways seem most appropriate. Suggestions include: Refer to the general education area in which you teach a course (or courses) and review the assessment information. Afterwards, think of ways you might enhance student learning. Volunteer to join an Intellectual Foundations Outcomes and Assessment Committee (OAC) in an area where you teach. Contact the Assistant Dean of Intellectual Foundations Office with suggestions for improving assessment in general education. Join a workshop or discussion group offered by Professional and Faculty Development. Get in touch with your Associate Dean for discussion groups in areas taught by your school This information was used in the general education program review process that resulted in a revised general education program in 2006 (Intellectual Foundations) and again in the review of IF in 2010-2012. Similarly, many changes were made in individual content areas over this period of time—ranging from hiring 4 full time faculty in the writing program to course revisions/additions/deletions, etc. as a result of the assessment reported here. Also, the Faculty and Professional Development Center has offered many workshops and focus groups for faculty that address not only teaching and learning but assessment as well. Lastly, CASTL has been engaged for several years in a study on academic rigor. This stemmed directly from the results of several administrations of NSSE. I hope you find the information in each dashboard meaningful and useful. Please do not hesitate to contact the Curriculum and Assessment office should you have questions or comments about this document or about student learning outcomes assessment in general. Rosalyn Lindner Associate Vice President Office of Curriculum and Assessment SUNY - Buffalo State [email protected] 716-878-5915 1

Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Introduction The purpose of this document is to compile various assessments in the required areas of general education at Buffalo State so that instructors, staff and administrators can review strengths and weaknesses in student performance and discuss ways to promote student learning. General Education is called “Intellectual Foundations” at Buffalo State and will be so called in this document. Multiple measures are used to assess the various areas of Intellectual Foundations which include analyses of student written work using rubrics, objective tests, standardized and benchmarked national assessments, surveys, etc. For lack of a better term, these will be called “dashboards” but that is not to imply that student learning can be captured solely by quantitative measure. Rather this is an attempt to present facts gleaned from several assessments and to encourage evidence-based conversations that lead to recommendations, contribute to Intellectual Foundations program review and ultimately contribute to enhancing student learning. Everyone is encouraged to review these dashboards and use the information contained within in whatever ways seem most appropriate. Suggestions include:

• Refer to the general education area in which you teach a course (or courses) and review the assessment information. Afterwards, think of ways you might enhance student learning.

• Volunteer to join an Intellectual Foundations Outcomes and Assessment Committee (OAC) in an area where you teach.

• Contact the Assistant Dean of Intellectual Foundations Office with suggestions for improving assessment in general education.

• Join a workshop or discussion group offered by Professional and Faculty Development. • Get in touch with your Associate Dean for discussion groups in areas taught by your school

This information was used in the general education program review process that resulted in a revised general education program in 2006 (Intellectual Foundations) and again in the review of IF in 2010-2012. Similarly, many changes were made in individual content areas over this period of time—ranging from hiring 4 full time faculty in the writing program to course revisions/additions/deletions, etc. as a result of the assessment reported here. Also, the Faculty and Professional Development Center has offered many workshops and focus groups for faculty that address not only teaching and learning but assessment as well. Lastly, CASTL has been engaged for several years in a study on academic rigor. This stemmed directly from the results of several administrations of NSSE. I hope you find the information in each dashboard meaningful and useful. Please do not hesitate to contact the Curriculum and Assessment office should you have questions or comments about this document or about student learning outcomes assessment in general. Rosalyn Lindner Associate Vice President Office of Curriculum and Assessment SUNY - Buffalo State [email protected] 716-878-5915

1

Page 2: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Dashboards Contained in this Document* Arts Basic Communication: Writing and Oral Communication/Writing Across the Curriculum Diversity Foreign Language Foundations of Civilizations: American History, Western Civilization and Non-western Civilization Foundations of Inquiry (BSC 101): Critical Thinking and Information Management Humanities Mathematics/Quantitative Reasoning Natural Sciences Social Sciences It is important to keep in mind that learning outcomes were refined/changed with the implementation of the new Intellectual Foundations (general education) program effective fall, 2006 and again, some were changed following the 2010-2012 program review.

Dashboards These dashboards contain assessment information for use by faculty, assessment committees, staff and administrators who wish to obtain information about the status of student learning in each area. The purpose of the dashboard is to offer a snapshot of various assessments. Those who view the dashboards are encouraged to review all of the information, ask questions and make conclusions with regard to how to enhance student learning at Buffalo State.

Outcomes and Information Portfolios Every three-five years, faculty committees (OAC) conduct assessments in each content area of Intellectual Foundations and submit a report. Faculty are chosen for these assessment committees based upon one criteria: they must teach at least one course in the area to be assessed. Multiple measures have been used including: rubrics for student work product, standardized tests, locally developed tests and surveys/focus groups/etc. depending on the outcomes to be assessed. Classes and students are selected at random for inclusion in any particular cycle of assessment. NSSE (National Survey of Student Engagement) The NSSE has been administered to freshmen and seniors at Buffalo State in 2006/07, 2008/09, 2011/12 and 2014-15. Targeted NSSE results are coupled with other assessment methods and included in the dashboards. SUNY SOS (Survey of Student Satisfaction) The SOS was administered to a random sample of all undergraduate students in 2003/04, 2006/07. 2008/09, 2011/12 and 2014/15. Targets results are coupled with other assessment methods and included in the dashboards.

2

Page 3: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

CLA (Collegiate Learning Assessment) The CLA has been administered to freshmen and seniors in 2007/08, 2009/10 and 2011/12. (The 2009/10 administration did not yield enough seniors with Entering Academic Ability scores to provide a valid comparison). The CLA+ was administered to freshmen in 2014. The CLA is used for assessment in writing, critical thinking and quantitative reasoning. Alumni Survey Results Alumni who graduated one to one and a half years ago complete this survey. The alumni are asked how well they were prepared in various learning outcomes while at Buffalo State. (The survey in 2010/2011 was greatly shortened and did not include learning outcomes in an attempt to see if the response rate would increase by using a very short survey. It did not and outcomes will again be included going forward.) Other In addition to these regularly scheduled surveys, the FSSE (Faculty Survey of Student Engagement) was administered to all regular faculty members in 2006/07 and 2014/15. This survey is not administered as often as the NSSE because faculty turnover is slower. FSSE numbers are not included here but available on the assessment website. CLASSE (Classroom Survey of Student Engagement) was administered in 2006 to instructors in the Writing Program as a pilot. The CLASSE was administered and returned to participating faculty through the CASTL (Carnegie Academy for the Scholarship of Teaching and Learning) program. A focus group was conducted with the participating faculty by CASTL leadership.

3

Page 4: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Outcomes and Information: Writing Student portfolios from BSC 102 include the following:

• Letter of reflection: personal writing in which the student introduces the portfolio and reflects on the content of the portfolio, development as a writer during the class and discusses future goals in writing

• An in-class essay: An essay based on thematic readings and written during one class period; • A Process Set: A formal paper which demonstrates the essay’s progression from early prewriting to final draft • A formal research paper with process set demonstrating the writer’s use of resource materials and proper

citation format. Process materials will demonstrate student competence in the writing process: prewriting, drafting, revision and final editing.

The student portfolios are collected by instructors in the writing program and include two copies of the research/process sets. One copy of the above research/process sets is collected by the Writing Program Director. Twenty percent of these are selected randomly and delivered to faculty readers for assessment according to a rubric designed by faculty across SUNY. Institutional Outcomes Institutional Student Learning outcomes at Buffalo State include the following:

• Basic Communication: Writing • Critical Thinking • Information Management • Diversity

Assessment for these outcome areas are reported here. They are not only assessed at the introductory level in Intellectual Foundations but also across the curriculum in all major programs.

4

Page 5: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Basic Communication Writing and Oral Communication

Portfolio Assessment CLA [Collegiate Learning Assessment} NSSE (National Survey Student Engagement) Institutional Assessment

5

Page 6: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Produces a Coherent Written Text

Portfolio: Ability to Revise and Improve Text

3.022.76

3.06

2.69

0

1

2

3

4

2003 2006 2009 2011

3.03

2.61 2.64

3.34

0

1

2

3

4

2003 2006 2009 2013

High Proficiency

Low Proficiency

High Proficiency

Low Proficiency

6

Page 7: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Research Topic, Develops Argument and Organize Supporting Details

2.03

2.713.01

3.44

0

1

2

3

4

2003 2006 2009 2013

High Proficiency

Low Proficiency

7

Page 8: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

CLA Writing Mechanics

CLA Writing Effectiveness

3.003.27

3.60

0

1

2

3

4

2010 2015

Freshmen Seniors

2.60

3.80

3.60

0

1

2

3

4

2010 2015

Freshmen Seniors

High Proficiency

Low Proficiency

High Proficiency

Low Proficiency

8

Page 9: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Prepared 2 or More Drafts

NSSE: Number Written Pages >5

2.58 2.49 2.42 2.40

2.972.69 2.60 2.70

0

1

2

3

4

2007 2009 2012 2015

NSSE: Prepared 2 or More Drafts

Seniors Freshmen

2.94

3.00

3.04

2.87

3.05

2.84

0

1

2

3

4

2007 2009 2012

NSSE: Number Written Pages > 5 Pages

Seniors Freshmen

Very Often

Never

Very Often

Never

9

Page 10: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: 5 – 19 Pages

NSSE: Between 6 and 10 pages (adjusted number of pages – 2015)

2.53 2.55 2.45

2.21 2.28 2.23

0

1

2

3

4

2007 2009 2012

NSSE: 5 - 19 Pages

Seniors Freshmen

2.32.9

0

1

2

3

4

2015

Freshmen Seniors

Very Often

Never

Very Often

Never

10

Page 11: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: 11 pages or more (adjusted number of pages – 2015)

NSSE 2015: How many papers, reports, or other writing tasks during current school year:

1.41.4

0

1

2

3

4

2015

Freshmen Seniors

6.1

7.3

1

2

3

4

5

6

7

8

9

10

Freshmen Seniors

Very Often

Never

11

Page 12: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE 2015: Estimated number of assigned pages of student writing

NSSE: Writing Clearly and Effectively

53.4

63.4

0

10

20

30

40

50

60

70

80

Freshmen Seniors

3.05 3.11 3.04

2.82.94 3.02 3.01

2.8

0

1

2

3

4

2007 2009 2012 2015

Seniors Freshmen

Very Large Contribution

No Contribution

12

Page 13: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

SUNY SOS: College Contribution to Writing Clearly and Effectively

Comparison of Native and Transfer Writing Skill – 2008

3.213.36 3.44

3.39

3.72

3.093.34 3.37

3.423.69

0

1

2

3

4

2003 2007 2009 2012 2015

SUNY Comprehensive Buffalo State

3.49 3.48

0

1

2

3

4

1 2

Very Large Contribution

No Contribution

Took 101/102 at BSC

Did Not Take 101/102 at BSC

13

Page 14: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Syllabi for Writing Across the Curriculum Courses

Alumni Survey: Writing Clearly and Effectively

32%30%

38%

0%

10%

20%

30%

40%

50%

1 2 3

2.62 2.63 2.64

0

1

2

3

4

2006 2007 2008

Very Large Contribution

No Contribution

Meets/ Exceeds

Standards

Approaches Standards

Does Not Meet

Standards

14

Page 15: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Oral Communication

Portfolio: Develop Proficiency in Oral Discourse (Rubric from National Speech Association Criteria)

Portfolio: Evaluate Oral Presentation According to Established Criteria

3.26

0

1

2

3

4

2004 2007

3.01

0

1

2

3

4

2004 2007

15

Page 16: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Speaking Clearly and Effectively

SOS: Speaking Clearly and Effectively

2.98 2.99 2.992.80

2.71 2.84 2.882.60

0

1

2

3

4

2006 2008 2012 2015

Seniors Freshmen

3.18 3.20 3.32 3.343.64

3.00 3.15 3.26 3.303.58

0

1

2

3

4

2003 2006 2009 2012 2015

SUNY Comparables Buffalo State

Very Large Contribution

No Contribution

Very Large Contribution

No Contribution

16

Page 17: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Alumni Survey: Speaking Clearly and Effectively

2.61 2.48

3.11

0

1

2

3

4

1 2 32006 2007 2008

17

Page 18: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Foreign Language

Portfolio Assessment NSSE (National Survey Student Engagement)

18

Page 19: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Basic Proficiency in Language

Portfolio: Cultural Knowledge

2.602.89 2.94

0

1

2

3

4

1 2 3

3.11

2.41

1.92

0

1

2

3

4

1 2 32003 2006 2009

2003 2006 2009

19

Page 20: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Percentage of Student Taking Foreign Language Coursework

NSSE: Percentage of Students Who Study Abroad

45%42% 41%

19% 18%

23%

0%

10%

20%

30%

40%

50%

2006 2008 2012

Seniors Freshmen

9%

6%8%

9%

1%2%

3%5%

0%

5%

10%

15%

20%

2006 2008 2012 2015

Seniors Freshmen

20

Page 21: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Critical Thinking

Portfolio Assessment NSSE (National Survey Student Engagement) SUNY SOS (Student Opinion Survey)

21

Page 22: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: I.D., Analyze and Evaluate Arguments

California Critical Thinking Skills Test: Develop Well Reasoned Arguments

2.36 2.35

0

1

2

3

4

1 2

2.36

3.03

0

1

2

3

4

2005 2010

2005 2008

22

Page 23: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Analyze basic elements of an idea, experience or theory

NSSE: Thinking Critically and Analytically

3.04

2.882.88

3.05

0

1

2

3

4

2006 2008

Seniors Freshmen

3.18

3.03

3.24 3.10

3.03

3.18

3.17 3.00

0

1

2

3

4

2006 2008 2012 2015

Seniors Freshmen

23

Page 24: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE 2015: Analyzing an idea, experience or line of reasoning in depth by examining its parts

NSSE 2015: Evaluating a point of view, decision, or information source

2.903.10

0

1

2

3

4

Freshmen Seniors

2.90 3.00

0

1

2

3

4

Freshmen Seniors

24

Page 25: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE 2015: Forming a new idea or understanding from various pieces of information

SOS: Acquiring Analytical Thinking Skills

2.90 3.00

0

1

2

3

4

Freshmen Seniors

3.52 3.62 3.593.82

3.38 3.43 3.45

3.83

0

1

2

3

4

2007 2009 2012 2015

SUNY Comparables Buffalo State

25

Page 26: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Social Sciences

Portfolio Assessment

26

Page 27: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Basic Knowledge of Methods of Gathering Evidence and Understanding What Constitutes Acceptable and Appropriate Evidence

Portfolio: Knowledge of concepts, models and issues and Ability to Apply Them

2.61

3.00

2.41

0

1

2

3

4

1 2 3

2.59

2.98

2.26

0

1

2

3

4

1 2 3

2003 2007 2010

2004 2007 2010

27

Page 28: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Social Science 2015

Describe social environments, behaviors and social issues in the context of course subject matter.

Apply basic concepts and terminology of social science.

17%

60%

23%

0%

10%

20%

30%

40%

50%

60%

70%

Exceeds Meets Does Not Meet

23%

56%

21%

0%

10%

20%

30%

40%

50%

60%

Exceeds Meets Does Not Meet

28

Page 29: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Evaluate the basic knowledge of methods of gathering evidence in social science.

30%

45%

25%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Exceeds Meets Does Not Meet

29

Page 30: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Mathematics/Quantitative Reasoning

Portfolio Assessment NSSE (National Survey Student Engagement)

30

Page 31: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Algebra, Arithmetic, Geometry*

*Learning Outcome as Defined by SUNY (Changed after 2003)

Portfolio: Data Analysis, Quantitative Reasoning*

*Learning Outcome as Defined by SUNY (Changed after 2003)

1.90

0

1

2

3

4

1

1.87

0

1

2

3

4

1

2003

2003

31

Page 32: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Interpret and draw inferences from mathematical models

Portfolio: Represent mathematical information symbolically, visually, numerically and verbally

2.943.12

0

1

2

3

4

1 2

2.90

2.57

0

1

2

3

4

1 2

2009 2006

2009 2006

32

Page 33: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Employ quantitative methods such as arithmetic, algebra, geometry and statistics to solve problems

Portfolio: Estimate and check mathematical results for reasonableness

2.282.64

0

1

2

3

4

1 2

2.46

3.44

0

1

2

3

4

1 2

2009 2006

2009 2006

33

Page 34: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Recognize the limits of mathematical and statistical methods

NSSE: Analyzing quantitative problems

2.08

3.07

0

1

2

3

4

1 2

2.853.08 2.99

2.722.96 2.93

0

1

2

3

4

2006 2009 2012

Seniors Freshmen

2009 2006

34

Page 35: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Reach Conclusions based on your own analysis of numerical information

Evaluated what others have concluded from numerical information.

2.5 2.5

0

1

2

3

4

Freshmen Seniors

2.3 2.2

0

1

2

3

4

Freshmen Seniors

35

Page 36: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Used numerical information to examine a real-world problem or issue.

2.3 2.2

0

1

2

3

4

Freshmen Seniors

36

Page 37: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Arts

Portfolio Assessment NSSE (National Survey Student Engagement)

37

Page 38: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Understanding one principle form of artistic expression and the creative processes inherent therein*

*Outcome as Defined by SUNY (Changed in 2006)

Portfolio: Ability to read and understand visual and/or performed language.

2.48

0

1

2

3

4

1

2.67 2.66 2.53

0

1

2

3

4

2007 2010 2014

2003

38

Page 39: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Understanding of the meaning and use of artistic symbols in social context

Portfolio: Ability to interpret visual and/or performed work, including an understanding of purposes and processes of creative endeavors

2.39 2.342.56

0

1

2

3

4

2007 2010 2014

2.41 2.362.52

0

1

2

3

4

2007 2010 2014

39

Page 40: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Ability to identify the persuasive and/or emotive aspects of visual and/or performed work

Portfolio: Understanding of artistic criteria for evaluating visual and/or performed work

2.36 2.352.60

0

1

2

3

4

2007 2010 2014

2.18 2.15

2.48

0

1

2

3

4

2007 2010 2014

40

Page 41: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Attended art exhibit, gallery, play, dance or theater performance

ARTS Assessment 2014- Suggestions going forward:

1. Schedule regular meetings of faculty teaching IF courses to regularly review and discuss the SLOs, assessment results, and pedagogy related to student achievement of the learning outcomes at the department, deans, and college levels.

2. Develop the next prompt and rubric for the following five year assessment cycle and encourage faculty teaching IF courses to use the rubric for various assignments and perhaps embed the prompt in the course each year so faculty can have regular assessment results. This process would also serve to increase consistent use, interpretation, and regular refinement of the assessment materials.

3. In future assessments of IF Arts, consider using multiple measures of assessment including review of syllabi, course assignments, and narrative/teaching portfolios of faculty teaching IF courses. It is suggested that assessment methods be authentically embedded within the course as appropriate.

4. As results of this assessment are disseminated and discussed, it is suggested that the improvement recommendations and actions to be taken be revised after all constituents have had the opportunity for input.

2.22

2.05 2.081.98

2.18 2.28

1

2

3

4

2006 2008 2012

Seniors Freshmen

41

Page 42: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Humanities

Portfolio Assessment

42

Page 43: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Conventions and Methods of at least one discipline in the Humanities* *Outcome as Defined by SUNY (Changed in 2006)

Portfolio: Reflect on basic questions of life with goal of understanding the world and one’s place in it

2.56

0

1

2

3

4

2003

2.933.27 3.25

0

1

2

3

4

2007 2010 2014

43

Page 44: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Articulate and defend critically informed values

Portfolio: Recognize and demonstrate creative thought in producing answers to individual and social questions

2.823.00 2.97

0

1

2

3

4

2007 2010 2014

2.59

2.932.61

0

1

2

3

4

2007 2010 2014

44

Page 45: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Engage in close reading of text (Employ close reading, analysis, and discussion (oral and written) of significant primary texts.

HUMANITIES 2014 Assessment - Suggestions going forward

1. Schedule regular meetings of faculty teaching IF courses to regularly review and discuss the SLO’s, assessment results, and pedagogy related to student achievement of the learning outcomes at the department, deans, and college levels.

2. Develop the next prompt and rubric for the following five year assessment cycle and encourage faculty teaching IF courses to use the rubric for various assignments and perhaps embed the prompt in the course each year so faculty can have regular assessment results. This process would also serve to increase consistent use, interpretation, and regular refinement of the assessment materials.

3. In future assessment of IF Arts, consider using multiple measures of assessment including review of syllabi, course assignments, and narrative/teaching portfolios of faculty teaching IF courses. It is suggested that assessment methods be authentically embedded within the course as appropriate.

4. As results of this assessment are disseminated and discussed, it is suggested that the improvement recommendations and actions to be taken be revised after all constituents have had the opportunity for input.

2.692.88

2.56

0

1

2

3

4

2007 2010 2014

45

Page 46: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Information Management

Portfolio Assessment SUNY SOS (Student Opinion Survey) NSSE (National Survey Student Engagement) Alumni Survey

46

Page 47: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Local Exam: Basic Computer Use

* Assessment in Senior Level Classes Across Curriculum ** Assessment in BSC 101 Freshmen Classes *** Assessment in CWP 102 Freshmen Classes

Local Exam: Basic Research Technology

* Assessment in Senior Level Classes Across Curriculum ** Assessment in BSC 101 Freshmen Classes *** Assessment in CWP 102 Freshmen Classes

2.79

2.29 2.21

0

1

2

3

4

2005* 2008** 2014***

2.71

1.71

2.21

0

1

2

3

4

2005* 2008** 2014***

47

Page 48: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Local Exam: Locate, Evaluate and Synthesize Information

* Assessment in Senior Level Classes Across Curriculum ** Assessment in BSC 101 Freshmen Classes *** Assessment in CWP 102 Freshmen Classes

SOS: Using computer information technology effectively

3.02

2.03 2.06

0

1

2

3

4

2005* 2008** 2014***

3.053.11

3.19

3.083.39

2.91

3.19

3.09

3.11

3.54

0

1

2

3

4

2003 2007 2009 2012 2015

SUNY Comparables Buffalo State

48

Page 49: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Used electronic medium to discuss or complete assignment

NSSE: Using computing and information technology

2.702.87

2.74

2.48 2.64 2.47

0.00

1.00

2.00

3.00

4.00

2006 2009 2012

Seniors Freshmen

3.16 3.21 3.14

2.85 3.05 2.98

0

1

2

3

4

2006 2009 2012

Seniors Freshmen

49

Page 50: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Using computers in academic work

Alumni Survey: Using computer technology

3.35 3.47 3.34

3.23 3.33 3.27

0

1

2

3

4

2006 2008 2012

Seniors Freshmen

2.382.51 2.44

0

1

2

3

4

1 2 32006 2008 2007

50

Page 51: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Improvements suggested as a result of this assessment: Regarding SLO #1, which had the greatest number below standard, the Information Management Oversight and Assessment Committee (IM OAC) would suggest improving the student’s ability to use citation management functionality from within the document processing application. This would improve citation formatting consistency, which was the main problem of students’ information usage, and alleviate many of the basic mistakes like indentation, use of et al, italics, and other APA/MLA rule automatically built into the logic of the citation management software. Use of citation management software at the time of information discovery would also alleviate students’ inability to choose/discern the proper citation document type, especially for those students who cited every source as a Web page simply because the information accessed was done so via the Web, regardless of the document type (online newspaper, journal article, e-book, etc.). Many students even cited the library’s online resources as web pages. Regarding SLOs #2 and #3, overall the students were successful in discovering appropriate information sources and integrating them into their papers. Continued success will be assured by the ongoing implementation of the new common writing handbook (Diana Hacker’s “A Writer’s Reference”) with emphasis on the sections for research and for MLA/APA information synthesis.

Action to be taken in addressing these assessment findings (Action Plan): The E.H. Butler Library, in conjunction with the College Writing Program, will develop instructional videos showing students how to take advantage of citation management software. These videos will be make available on the library website for inclusion in course learning management systems so that students may repeatedly access and benefit from the citation application instruction asynchronously at their convenience or at the appropriate time.

Suggestions going forward: Incorporate the new Framework for Information Literacy for Higher Education produced by the Association of College & Research Libraries (ACRL) when completed late 2014. The draft version of this framework (published June 2014) provides 6 frames for institutions to use in developing information management/information literacy programs. The six frames are “scholarship is a conversation,” “Research as Inquiry,” “Authority is Constructed and Contextual,” “Format as a Process,” and “”Searching as Exploration.” Earlier drafts of this framework, with the same or similar frames, influenced the IM OAC’s interpretation of the SUNY Information Management competency during IM OAC deliberations on the new Intellectual Foundation program. ACRL framework was instrumental in deciding to integrate the IM SLOs with the Writing SLOs when infusing these outcomes into the majors. After the IF2014 initiation this fall, the ACRL framework will help departments understand the goals and logic behind our interpretation of SUNY’s committee has for producing information literate graduates.

51

Page 52: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

SUNY IM SLO Exceeds Standard (3) Meets Standard (2) Below Standard (1) SLO 1: “Perform the basic operations of personal computer use” (as evinced by the electronic document’s formatting and submission).

Submitted an electronic copy of their document with proper style and citation formatting.

Submitted an electronic copy of their document with proper citation formatting.

The student was able to submit an electronic copy but the citation formatting was improper.

SLO 2: “Understand and use basic research techniques” (as evinced by the appropriateness of the resources the student was able to find and cite).

All references are from professional periodicals or other appropriate academic sources.

References are primarily from professional periodicals and appropriate sources or usage of other source types is contextualized and justified.

References are primarily from sources of questionable authority or legitimacy and their usage is not adequately justified.

SLO 3: Students will locate, evaluate, and synthesize information from a variety of sources (as evinced by source summaries, comparison, and arrangement).

The student provides exemplary information source summary (evaluate), establishes relevancy (locate), and integrates resources meaningfully (synthesize).

The student provides sufficient information source summary (evaluate), establishes some relevancy (locate), and integrates most resources meaningfully (synthesize).

The student provides minimal information source summary (evaluate) and does not establish relevancy (locate) or sufficiently integrate resources (synthesize).

52

Page 53: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Natural Sciences

Portfolio Assessment CAAP

53

Page 54: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Understanding of Methods Used to Explore Natural Phenomena

Portfolio: Application of data

3.14

0

1

2

3

4

1

3.10

0

1

2

3

4

1

Fall 2003

Fall 2003

54

Page 55: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

CAAP Exam

61.70

59.60 58.74

20.00

40.00

60.00

2007 2010

National Mean

Buffalo State

55

Page 56: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Western Civilization

Portfolio Assessment

56

Page 57: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Development of Distinctive Features of Western Civilization

Portfolio: Relationship to Other Regions of the World

2.30 2.33

0

1

2

3

4

2005 2008 2016

2.34 2.35

0

1

2

3

4

2005 2008 2016

57

Page 58: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Non Western Civilization

Portfolio Assessment

58

Page 59: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Knowledge of a Broad Outline of World History

2.622.73

0

1

2

3

4

2005 2008 2016

59

Page 60: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

American History

Portfolio Assessment

60

Page 61: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Knowledge of Basic Narrative of American History

Portfolio: Knowledge of Common Institutions in American Society

2.422.69

2.34

0

1

2

3

4

2003 2006 2012

2.412.66

2.15

0

1

2

3

4

2003 2006 2012

61

Page 62: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Understanding of American Relationships with the Rest of the World

2.852.65

2.02

0

1

2

3

4

2003 2006 2012

62

Page 63: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Diversity

Syllabi Analysis Portfolio Assessment SUNY SOS (Student Opinion Survey NSSE (National Survey Student Engagement) Alumni Survey

63

Page 64: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Diversity Syllabi 2008: Were outcomes included?

Diversity Syllabi 2015: Course activities were aligned with outcomes.

31%

15%

54%

0%

10%

20%

30%

40%

50%

60%

1 2 3

43%

37%

0%

10%

20%

30%

40%

50%

Adjuncts Tenure/Tenure Track

Yes Yes/No No

64

Page 65: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Diversity Syllabi 2008: Were activities aligned with outcomes?

Diversity Syllabi 2015: Outcomes were included in syllabi.

31%

23%

38%

8%

0%

10%

20%

30%

40%

50%

1 2 3 4

57%

47%

0%

10%

20%

30%

40%

50%

60%

70%

Adjuncts Tenure/Tenure Track

Yes Yes/No No N/A

65

Page 66: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Portfolio: Critically examine the past, current or prospective influences of diverse groups in American Society.

SOS: Understanding and appreciating ethnic/cultural diversity and individual differences

2.07

0

1

2

3

4

2010

3.07

3.18

3.32

3.36 3.59

2.97

3.26

3.24

3.373.63

0

1

2

3

4

2003 2007 2009 2012 2015

SUNY Comparables Buffalo State

66

Page 67: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

NSSE: Included diverse perspectives in class assignments or writing assignments

NSSE: Understanding people of other racial and ethnic backgrounds

2.87 2.83

2.852.70

2.80 2.80

2.88

2.60

0

1

2

3

4

2006 2008 2012 2015

Senior Freshmen

57%

47%

0%

10%

20%

30%

40%

50%

60%

70%

Adjuncts Tenure/Tenure Track

67

Page 68: Introduction - Buffalo State Collegebscintra.buffalostate.edu/assessment/GenEdDashboard.pdf · Student portfolios from BSC 102 include the following: • Letter of reflection: personal

Understand and Appreciate Ethnic/Cultural Diversity and Other Individual Differences. Alumni Survey (4 = Very Large Contribution, 0 = No Contribution)

2.53 2.47 2.59

0

1

2

3

4

2006 2007 2008

68