Graduate Program Graduate Program AssessmentAssessment

Dianne Horgan Tom RhodesDianne Horgan Tom Rhodes Grad College Grad College

Keep it SIMPLEKeep it SIMPLESome ExamplesSome Examples

Outcomes are pretty easyOutcomes are pretty easy

(from Walvoord, 2004)(from Walvoord, 2004)

AME PhDAME PhD Engage in independent and critical thinking by Engage in independent and critical thinking by

obtaining significant understanding of the obtaining significant understanding of the primary and secondary areas of concentration, primary and secondary areas of concentration, including minors including minors

Develop publishable research of the highest Develop publishable research of the highest standards under supervision of a faculty mentor standards under supervision of a faculty mentor

Become an expert in an area of research, fully Become an expert in an area of research, fully prepared to independently continue in future prepared to independently continue in future positions in academia, industry or government positions in academia, industry or government laboratories laboratories

MFA in DanceMFA in DanceThe MFA in dance offers course work which enables The MFA in dance offers course work which enables students to gain skills and knowledge in these areas. At the students to gain skills and knowledge in these areas. At the end of the degree program, students will be able to:end of the degree program, students will be able to:

Perform at a professional level in diverse styles of dance including ballet, Perform at a professional level in diverse styles of dance including ballet, jazz and/or modern dance. (Performance emphasis) jazz and/or modern dance. (Performance emphasis)

Choreograph at a professional level in diverse styles of dance Choreograph at a professional level in diverse styles of dance (Choreography emphasis) (Choreography emphasis)

Understand and develop scholarly research techniques and be familiar Understand and develop scholarly research techniques and be familiar with the written and verbal skills necessary and appropriate for the field with the written and verbal skills necessary and appropriate for the field

Articulate a personal aesthetic Articulate a personal aesthetic

Teach a dance technique class as well as an academic class. Teach a dance technique class as well as an academic class. Solve problems creatively.Solve problems creatively.

Measures are harderMeasures are harder Dance performance is assessed through faculty feedback during technique classes, after Dance performance is assessed through faculty feedback during technique classes, after

performances & by individual meetings & is graded through studio classes & ensemble work. performances & by individual meetings & is graded through studio classes & ensemble work. Choreography is assessed through close faculty mentoring of graduate choreography projects Choreography is assessed through close faculty mentoring of graduate choreography projects

(DNC694A Masters Project in Dance & DNC909 Masters Report) as well as through classes (DNC694A Masters Project in Dance & DNC909 Masters Report) as well as through classes which explore the craft of choreography (DNC545A&B Advanced Choreography, which explore the craft of choreography (DNC545A&B Advanced Choreography, DNC596E Seminar in Music & Dance Collaborations). DNC596E Seminar in Music & Dance Collaborations).

Writing & verbal skills are developed & assessed through the formal research class Writing & verbal skills are developed & assessed through the formal research class (DNC596A Critical Issues in Dance), written proposals for Master's Projects & the Final Project (DNC596A Critical Issues in Dance), written proposals for Master's Projects & the Final Project Proposal & writing assignments in course work (DNC546A Careers in Dance, Proposal & writing assignments in course work (DNC546A Careers in Dance, DNC696A Graduate Forum, DNC555 Biomechanics for Dancers, DNC595A Teaching Methods DNC696A Graduate Forum, DNC555 Biomechanics for Dancers, DNC595A Teaching Methods of Dance, DNC596E Seminar in Music & Dance Collaborations & DNC591 001, 002 of Dance, DNC596E Seminar in Music & Dance Collaborations & DNC591 001, 002 Preceptorships). The large lecture preceptorship (DNC591 002) is particularly valuable to the Preceptorships). The large lecture preceptorship (DNC591 002) is particularly valuable to the student as it affords them the opportunity to mentor & grade undergraduate writing student as it affords them the opportunity to mentor & grade undergraduate writing assignments. assignments.

The student's personal aesthetic is developed & assessed throughout their plan of study The student's personal aesthetic is developed & assessed throughout their plan of study through close mentoring by the faculty. The assessment of the personal aesthetic is further through close mentoring by the faculty. The assessment of the personal aesthetic is further evaluated through the final project & the final oral examination which are conducted by a evaluated through the final project & the final oral examination which are conducted by a committee of faculty. committee of faculty.

Students develop teaching skills & are assessed on their pedagogical knowledge through Students develop teaching skills & are assessed on their pedagogical knowledge through specific course work including DNC595A Teaching Methods of Dance, DNC591 001 specific course work including DNC595A Teaching Methods of Dance, DNC591 001 Preceptions in Teaching Methods of Dance & DNC591 002 Preceptorship in a Large Lecture Preceptions in Teaching Methods of Dance & DNC591 002 Preceptorship in a Large Lecture class. Teaching Methods of Dance gives the student fundamental knowledge in teaching ballet, class. Teaching Methods of Dance gives the student fundamental knowledge in teaching ballet, jazz & modern dance at the beginning level. The Teaching Methods Preceptorship enables jazz & modern dance at the beginning level. The Teaching Methods Preceptorship enables them to develop advanced knowledge & skills in a specific dance discipline & to apply this them to develop advanced knowledge & skills in a specific dance discipline & to apply this knowledge & these skills in a classroom setting. DNC591 002, the large lecture preceptoship knowledge & these skills in a classroom setting. DNC591 002, the large lecture preceptoship enables the student to develop skills & knowledge for teaching a large academic lecture class. enables the student to develop skills & knowledge for teaching a large academic lecture class. in addition graduate students are given the opportunity to teach at least one course (studio or in addition graduate students are given the opportunity to teach at least one course (studio or academic) during their time of study. academic) during their time of study.

From Masters of Accounting From Masters of Accounting Program:Program:

Gainful EmploymentGainful Employment - The student will: - The student will: Exhibit professional demeanor in all classroom and Exhibit professional demeanor in all classroom and

community affairs community affairs Be engaged before graduation in pursuit of employment Be engaged before graduation in pursuit of employment

opportunitiesopportunities

Good outcomes, but who measures? Who collects the Good outcomes, but who measures? Who collects the data?data?

Oral & Written CommunicationOral & Written Communication - The student can: - The student can: Provide constructive and supportive feedback to peersProvide constructive and supportive feedback to peers

How are these data collected?How are these data collected?

Direct measurement is goodDirect measurement is good

Student performance such as exam or Student performance such as exam or projectproject

Set of criteria by which to evaluate the Set of criteria by which to evaluate the performanceperformance

Analysis and interpretation of the resultsAnalysis and interpretation of the results Feedback loop for improvement/decision Feedback loop for improvement/decision

makingmaking

Make your criteria explicit Make your criteria explicit Expert judgment is OK: make criteria explicitExpert judgment is OK: make criteria explicit A rubric can helpA rubric can help

Only collect data that someone will useOnly collect data that someone will use Build on what you’re doing nowBuild on what you’re doing now Results should lead to actionResults should lead to action

A simple gridA simple grid

Goals: Goals: 1. publishable 1. publishable researchresearch2. ethical 2. ethical principlesprinciples3. teach3. teach

Results should lead to ACTIONResults should lead to ACTION

Don’t bother with data that won’t Don’t bother with data that won’t lead to actionlead to action

Data for assessment: So easy a Data for assessment: So easy a caveman can do itcaveman can do it

Better information for Better information for decision makingdecision making

Richer dataRicher data Real time dataReal time data Consistent dataConsistent data Easy to use/gather/disseminateEasy to use/gather/disseminate

Dashboards Dashboards Instantly UpdatedInstantly Updated

Canned ReportsCanned Reports APR infoAPR info Accreditation dataAccreditation data Standard survey dataStandard survey data

Or unique for your needsOr unique for your needs We can help you build them through Analytics We can help you build them through Analytics

(BI)(BI) Got your own database? You can add to BIGot your own database? You can add to BI

Drill down capabilitiesDrill down capabilities

Sample Program DashboardSample Program DashboardApplications, Admits, Matrics

0

50

100

150

200

250

Applications Admits Matrics

2007

2008

2009

Median Time to Degree: 6.5 yrs

Completion Rate: 36%

Reports from Student ProfileReports from Student Profile

Any of the hundreds of attributes can be Any of the hundreds of attributes can be made into unique reports for your programmade into unique reports for your program

All sorts of bio/demographic dataAll sorts of bio/demographic data What was is SIS and Matrix What was is SIS and Matrix

PLUS:PLUS:

Some of MY FavoritesSome of MY Favorites

Years to degree (from first enrollment)Years to degree (from first enrollment) Enrollment term count (# of terms student Enrollment term count (# of terms student

has been enrolled)has been enrolled) Student milestonesStudent milestones

Grad college’s: POS, committee Grad college’s: POS, committee appointment, Comps, Adv to Candidacy, IRB, appointment, Comps, Adv to Candidacy, IRB, etcetc

YOURS: internships, core courses, qualifying YOURS: internships, core courses, qualifying exams, finger print, etcexams, finger print, etc

OthersOthers

Any & all tests (SAT, GRE, TOEFL, GMAT, etc)Any & all tests (SAT, GRE, TOEFL, GMAT, etc) HS GPA, rankHS GPA, rank Transfer infoTransfer info Financial awardsFinancial awards All majors/minors/optionsAll majors/minors/options Academic loadAcademic load GPA GPA credits/semestercredits/semester First and last enrollmentFirst and last enrollment Catalog yearCatalog year HUNDREDS moreHUNDREDS more

You Can Add Your Own MilestonesYou Can Add Your Own Milestones

Acad Advising module will go live in Acad Advising module will go live in AUGUST for graduate programsAUGUST for graduate programs

Grad Coordinators now going to ‘testing’ to Grad Coordinators now going to ‘testing’ to learn and to provide info to set up learn and to provide info to set up milestones and other aspects of Acad milestones and other aspects of Acad Advising.Advising.

We can add your unique milestone such We can add your unique milestone such as qualifying exams, internships, etc. as qualifying exams, internships, etc.

Comparison and National dataComparison and National data

Will continue to post: Will continue to post: http://grad.arizona.edu/faculty-and-staffhttp://grad.arizona.edu/faculty-and-staff

Data Data UA Data UA Data Graduate Programs Assessment Graduate Programs Assessment National Data Sources National Data Sources Point-in-Time Application, Admit and Point-in-Time Application, Admit and

Enrollment Data Enrollment Data

MyGradColl ResourcesMyGradColl Resources

Student RosterStudent Roster Training DocumentationTraining Documentation Signup for Testing or Training (grad Signup for Testing or Training (grad

college)college)

Plus Tom & Dianne happy to consult with Plus Tom & Dianne happy to consult with you on grad program assessmentyou on grad program assessment

Mosaic ResourcesMosaic Resources

http://mosaic.arizona.edu/Mosaic_Traininghttp://mosaic.arizona.edu/Mosaic_Training