Assessment at California Baptist University

Assessment at CBU 1

Assessment at California Baptist University

The goal of assessment is the continual improvement of the learning environment.

Student Outcomes Assessment Cycle Components of the Cycle

Defining Collecting Analyzing Reporting Improving

Assessment at CBU 2

Schools and Departments develop PSOs for each major or program with “critical assignments” and evaluation rubrics

DEFINING

Students produce “critical assignments” in noted courses using pictured rubrics as guide as well as suggested formats, etc. Assignments are shared with professors for evaluation

COLLECTING

Course / Program & University Student Out Matrix

I = Introduced; P = Practiced; D = Demonstrated

Schools and Departments complete the CPUSO matrices to link USOs & PSOs with particular courses in which specific critical assignments will be evaluated

DEFINING

Professors provide guidance in syllabi/rubrics and direct critical assignments based on School/Dept planning

DEFININGProfs evaluate student’s critical assignments using rubrics, etc. In LiveText, the rubric results are gathered for ALL that are evaluated using that rubric

ANALYZING

Results are analyzed & will

help identify strengths and weaknesses in curriculum, pedagogy, methods, etc.

ANALYZING -REPORTING

Plans for improvement are implemented and assessment continues. External constituencies will be canvassed to validate outcomes & PSO listings.

IMPROVING

Schools and Departments develop plans for curricular improvement.

IMPROVING

Annual Student Outcomes Assessment Process, page 2

Goal: Continuous Improvement of the Learning Environment

asmtz/asoprocess.ppt

Initial Report ID’s “AYs PSO Focus”

LiveText Rubrics serve to provide for formative assessment for students and summative assessment for critical assignmnts/PSOs

Collected data is arranged in various charts for analysis

Final Reports explain analyses from the standpoint of intended improvements

Annual Recap Reports and FEEE display process progress and intended adjustments

Assessment at CBU 3

Assessment CycleAssessment CycleSchools and Departments develop PSOs for each major or program with “critical assignments” and evaluation rubrics DEFINING

Course / Program & University Student Out Matrix

I = Introduced; P = Practiced; D = Demonstrated

Schools and Departments complete the CPUSO matrices to link USOs & PSOs with particular courses in which specific critical assignments will be evaluated DEFINING

Professors provide guidance in syllabi/rubrics and direct critical assignments based on School/Dept planning DEFINING

LiveText Rubrics serve to provide for formative assessment for students and summative assessment for critical assignments/PSOs

Profs evaluate student’s critical assignments using rubrics, etc. In LiveText, the rubric results are gathered for ALL that are evaluated using that rubric

ANALYZING

Students produce “critical assignments” in noted courses using pictured rubrics as guide as well as suggested formats, etc. Assignments are shared with professors for evaluation

Data is analyzed and compiled in LiveText

Students submit assignments via LiveText

Schools and Departments analyze results, identify strengths and weaknesses in curriculum, pedagogy, methods, etc. for focus in

INITIAL ANNUAL REPORT

Assessment at CBU 4

44

ANALYZING -REPORTING

Plans for improvement are implemented and assessment continues. External constituencies will be canvassed to validate outcomes & PSO listings.

Schools and Departments develop plans for curricular improvement.

IMPROVING

Continuous Program Improvement

Final LiveText Reports explain analyses from the standpoint of intended improvements

Annual LiveText Recap Reports and FEEE display process progress and intended adjustments

Initial Report ID’s “AYs PSO Focus” Submitted in LiveText

Assessment at CBU 5

DEFINING

Fall 2008 Fall 2007

Student Outcomes Assessment Cycle

COLLECTING DATA

ANALYZING RESULTS

ACTION TAKEN

CONTINUED IMPROVEMENT Faculty Workshop Theme

Writing Across the Curriculum

USO 3 Critical Thinking and Literacy Skills

Faculty implement plans and report results and plans for follow-up at Fall Assessment Colloquium.

Academic units develop a plan for enhancing writing instruction and assignments requiring critical thinking.

Faculty across campus report decline in student written and oral skills when expressing ideas

Identify points at which writing and expressing ideas are introduced, practiced, evaluated

Assessment at CBU 6

USOs, PSOs and Discipline Rubric Relationships

University Student Outcomes (6)

Discipline X PSOs w/PSO Rubric

Course X1 w/ Disp Rubric incl PSO



Discipline Y PSOs w/PSO Rubric

Course Y1 w/ Disp Rubric incl PSO



Discipline Z PSOs w/PSO Rubric

Course Z1 w/ Disp Rubric incl PSO



1. “Disciplines” represent the Schools, Departments and their individual Majors or Progams. Individual Majors or Programs have distinct PSO sets that Identify a graduate in that discipline. The PSO Rubric is developed to evaluate the critical elements of any student’s “outcomes” with the ultimate goal of determining efficacy of the program.

2. The subordinate courses use the same rubric, but can have distinctive elements of the discipline evaluated as a subordinate component of the rubric by using “milestones.” Milestones can be time-related such as entry level, mid-range and capstone. Other milestone options could be differing components of a single discipline, etc.

3. Using these LiveText rubrics and the milestone characteristic allows the various disciplines to develop a variety of reporting levels that support the course, PSO, discipline and university evaluation of student outcomes.

Assessment at CBU 7

Step 1, Defining

In this stage the development of the Program Student Outcomes (PSO) occur.

In one major or program, the list of PSOs DEFINE what knowledge, skills, abilities and behaviors a graduate in that discipline should possess.

This stage also is where the Course and Program/University Student Outcomes matrix is developed (and reviewed and revised)

Assessment at CBU 8

Step 2, Collecting

Stage one merges into this Collecting stage as the courses and critical assignments are identified for data collection planning

In each annual cycle approximately one-fifth of the PSOs are identified for evaluation

The specific means of assessment are decided upon and plans are put in place for collecting the data

Assessment at CBU 9

Step 3, Analyzing

Stage two merges into this Analysis stage as the noted means of assessment occur and data is collected.

Faculty and assessment coordinators evaluate the collected data, develop means for analysis and conduct the analysis

Results of the analyses are prepared for reporting.

Assessment at CBU 10

Step 4, Reporting

Stage two merges into this Reporting stage as the data and analyses are placed in the Annual Final Report and Recap Report

A VERY key step in this stage is arriving at the “so what” of the analysis in terms of what changes will be made to improve

In addition to the two reports noted above, each unit will also complete a Self-Evaluation on their assessment Process (the FEEE)


Step 5, Improving

This final step is the culmination of all of the assessment effort Improving the learning environment as a result of acting on what is discovered in the annual cycle.

Units will take action such as modifying pedagogy or curriculum, etc.

This step merges back into the first step of Defining as the cycle repeats with a review of the PSOs, etc.


Examples of “Improving”, p.1

Reviewing the evaluation process for standardization Review the use and integration of writing instruction Enhance the development of the “voice” concept in

writing instruction Better incorporate the involvement of faculty in

assessment development Increase coverage of effective modes and media for

diverse population communication Develop a rubric for better evaluation of PSO Develop a new required course to provide for missing

skills and knowledge


Examples of “Improving”, p.2

Review standards Review exit review process Enhance curricular and assessment processes

Note: These examples are extracted from the CBU “Assessment Activities Review ‘Sampler’ for the 2006-2007 Academic Year.


DEFINING


COLLECTING DATA

ANALYZING DATA

ACTION TAKEN

CONTINUED IMPROVMENT

School of Behavioral Sciences Masters in Family Therapy

PSO 1

Dr. Collins and Dr. Stokes will continue to collect exam pass rate data for the next academic year

CA State Comp Exam writing pass rates = 14% at 80 – 90%; 82% at 90% or above; 4% incomplete.

Most instruction is adequate or better and not indicative of any significant shortcomings.

This early evaluation needs to be reviewed for standardization.


DEFINING


COLLECTING DATA

ANALYZING DATA

ACTION TAKEN

CONTINUED IMPROVEMENT

School of Behavioral Science BA in Sociology

PSO 3 = Written Communications

Dr. Minton and Professor Moore will continue the development of process and procedures.

Continue to use and review use of writing/integration instruction.

90% of students scored at 78% or better on writing rubric.

Standard of success was met (80% at 75th percentile).


DEFINING


COLLECTING DATA

ANALYZING DATAACTION TAKEN


School of Business BA/S in Business

PSO 5 = Integration of Faith

Dr. Herrity and Dr. Chapman will collect and analyze rubric data and peer evaluation of faculty using new methods.

Training of faculty on the use of Power Posters and other assessment methods. Rubrics will be developed.

30% exemplary, 20% proficient, 9% adequate, 9% inadequate, 32% not evaluated.

Generally acceptable instruction, but process of evaluation needs improvement.


DEFINING


COLLECTING DATA

ANALYZING DATA

ACTION TAKEN


School of Education Graduate Programs

PSO 4 = Research skills

Dr. Crist, Dr. Timmons, and all program coordinators will continue the development of process and procedures

MS/SS Rubric- 43% exemplary; 55% proficient. SOE Portfl. Rubric- 69% exemplary; 30% N/A

Curriculum is clear and students have good opportunities; some weakness remains in establishing their “voice”.

Continue/Improve focus on writing; Enhance developing “voice” concept; Continue review and development of PSO.


DEFINING


COLLECTING DATA

ANALYZING DATA

ACTION TAKEN


School of Education Graduate Programs

PSO 6 = Cross Cultural Communication

Professor Cabral will coordinate integration of new curricular elements in targeted classes in each grad program.

Increase coverage of effective modes and media for diverse population communication.

Class average 80.7% on critical assignments evaluated.

Instruction on working with diverse populations is adequate.


DEFINING

Faculty Self-Assessment Cycle

COLLECTING DATA



Personal Development Plan

Faculty

Increased Scholarly Activity

Faculty Self Studyreports results of efforts and personal level of satisfaction with growth.

Faculty Self Study identifies number of conferences, papers, and presentations for previous year

Faculty member implements improvement plan (answers additional calls for papers, collaborates with colleagues, takes advantage of internal opportunities.)

In conference with supervisor, faculty member formulates an improvement plan.


DEFINING


COLLECTING DATA



History, Political Science, Criminal Justice BAS

PSO 3 = History and Political Science

Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient.

Instruction, curriculum and/or evaluation and MOAs need to be improved/revised.

Dr. McHorney and Political Science and History faculty will continue the development of process and procedures.

Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well.


DEFINING


COLLECTING DATA



History, Political Science, Criminal Justice BAS

PSO 3 = History and Political Science

Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient.

Instruction, curriculum and/or evaluation and MOAs need to be improved/revised.

Dr. McHorney and Political Science and History faculty will continue the development of process and procedures.

Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well.


DEFINING


COLLECTING DATA



Modern Languages and Literature MA in English

PSO 1 Integration of FaithPSO 4 Original Scholarship

Dr. Lu and Graduate English faculty will pilot a new Thesis Defense rubric in 2008-09.

Success was achieved with all students meeting the standards at some level

Evaluation rubric for thesis proposals will be designed; further refinement will continue.

Thesis proposal process is developing properly to include these elements.


DEFINING


COLLECTING DATA

ANALYZING DATA

ACTION TAKEN


Modern Languages and Literature BA in Spanish

PSO 2 Respect DiversityPSO 6 Literature Knowledge

Professor Flores and Spanish faculty will monitor newly implemented curriculum in SPA 125.

Success was generally achieved in these evaluations with small exceptions in each PSO.

Literature will be introduced in SPA 125 allowing SPA 115 focus on vocabulary, language, and culture.

As a new program this evaluation is good news . The noted shortcomings are the lack of time in SPA 115 for literature.

Documents

Assessment at California Baptist University