The role of Mineralogy-Petrology-The role of Mineralogy-Petrology-Geochemistry courses in programmatic Geochemistry courses in programmatic learning outcomes and their assessment learning outcomes and their assessment

Darrell J. HenryDarrell J. HenryDept. of Geology and GeophysicsDept. of Geology and GeophysicsLouisiana State University, USALouisiana State University, USA

Fear and loathing … Fear and loathing … of assessmentof assessment

“This is just another bunch of paperwork that the administration wants that will do no good. It is a waste of time!”

“Tell us what you want us to write.”

“The results of assessment will probably be used against the faculty.”

“Our program is perfectly fine.”

“Once I have done it, do I have to change it?”

Student learning outcomesStudent learning outcomes

Specific statements that describe the required learning achievement that is intended to be met upon attaining: •Competency in a course

•The goals of a degree program

Typically a snapshot at/near the end of a course or degree program

AssessmentAssessmentThe systematic and continuous method of gathering, analyzing and using information from measured outcomes as a feedback to improve student learning.

Four levels of assessment•Individual student assessment (grade)•Course assessment (specific course)•Program assessment (academic and support programs related to a degree)•Institutional assessment (campus-wide characteristics)

Institutional assessmentInstitutional assessmentThe goal – to retain accreditation, a voluntary, independent review of educational programs to determine that the education provided is of uniform and sound quality. Six US regional accreditation agencies for college and university higher education programs: •The Middle States Association of Colleges and Schools.•The New England Association of Schools & Colleges.•The North Central Association of Colleges and Schools.•The Northwest Association of Schools And Colleges.•The Southern Association of Colleges and Schools.•The Western Association of Schools and Colleges.

Common format - an assessment Common format - an assessment matrix for each degree programmatrix for each degree program

Learning Outcome

(Kinesiology)

1. To demonstrate an understanding of the subject matter for kinesiology that reflects a variety of concepts from history, anatomy, biomechanics, motor learning, psychology of physical activity, and exercise physiology.

Assessment Process

This outcome is assessed indirectly using exit surveys,job placement of graduating students, focus group discussions, curriculum/syllabus analysis, and certification and licensure exams. The direct assessment is a multiple choice content knowledge test administered annually to a group of senior students. A pool of 120 mastery questions (20 from each of the six areas, designed to assess major concepts from the courses)has been constructed by the course coordinators. A sample of seniors completes a 30 question exam (5 randomly selected questions from each area)to assess mastery of the content.

Schedule of Assessment

Implementation

Objective is assessed annually.

Use of Results to Improve Program

The curriculum was changed. Based on assessment results and systematic evaluation of course analysis two required courses in the kinesiology core have been changed. KIN 2503 was deleted from the core and KIN 2501 has been revised to include a personal training component to more adequately meet program objectives.) New faculty have been hired to better meet instructional needs.

Typical University-level Criteria for Typical University-level Criteria for Evaluation of Degree Program Assessment Evaluation of Degree Program Assessment

 

Quality of core learning outcomes statements • Do they comprehend the program? • Are the core outcomes measurable?

 

Measures for determining levels of achievement of learning outcomes• What are direct and indirect measures of assessment of learning

outcomes? • Who is responsible for implementation of assessment procedures?• When are they measured ?  (They are best if they are ongoing.)

 

Use of results• Has entire faculty discussed results of assessment?• Have results been used to lead to program improvement (e.g., changes in

curricular structure, teaching and learning strategies, etc.) or indicate aspects of curriculum that appear to be working well?

What is direct assessment and how can What is direct assessment and how can we measure it?we measure it?

Direct assessment – assessment based on an analysis of student behaviors or products in which they demonstrate how well they have mastered learning outcomes*

* Information modified from: Mary Allen (2008) Strategies for Direct and Indirect Assessment of Student Learning. SACS 2008 Annual Meeting

Strategies for direct assessment of student learning1. Published tests (e.g. former Geology GRE)2. Locally-developed tests (e.g. exit exams)3. Embedded assignments and course activities (more later)4. Portfolios5. Collective portfolios

What is indirect assessment and how What is indirect assessment and how can we measure it?can we measure it?

Indirect assessment – assessment based on an analysis of reported perceptions about student mastery of learning outcomes.* This category is most useful in program review.

* Information modified from: Mary Allen (2008) Strategies for Direct and Indirect Assessment of Student Learning. SACS 2008 Annual Meeting

Strategies for indirect assessment of student learning1. Surveys (e.g. student exit survey)2. Interviews (e.g. student exit interviews)3. Focus groups near time of completion of degree4. Grades (generally)

Properties of Good Assessment Properties of Good Assessment TechniquesTechniques

* Information modified from: Mary Allen (2008) Strategies for Direct and Indirect Assessment of Student Learning. SACS 2008 Annual Meeting

•Valid – directly reflects learning outcome being assessed

•Reliable – especially true for inter-rater reliability

•Actionable – results help identify what students are learning well and what requires more attention

•Efficient and cost-effective

•Engaging to students and other respondents – more likely to demonstrate the extent of their learning

• Interesting to faculty – they should care about results and be willing to act on them

•Triangulation – multiple lines of evidence point to same conclusion

LSU student learning outcomes – BS LSU student learning outcomes – BS in Geologyin Geology

Learning outcome 1. Students will understand how fundamental processes (chemical, biological, and physical) have affected Earth.

Learning outcome 2. Students will develop the ability to conduct and to analyze field-based geologic problems by learning field methodology for multiple types of geological investigations and by using these methods to practice the scientific method in a field setting.

Learning outcome 4. Students will demonstrate their ability to solve complex geologic problems.

Focus on a single learning outcomeFocus on a single learning outcome

LSU Geology BS learning outcome 3.LSU Geology BS learning outcome 3.““Students will develop the ability to effectively Students will develop the ability to effectively communicate geologic concepts and material to the communicate geologic concepts and material to the public and professional colleagues in written and oral public and professional colleagues in written and oral formats”formats”

2009 – Igneous and Metamorphic Petrology is designated means of 2009 – Igneous and Metamorphic Petrology is designated means of assessmentassessment

Course-embedded research projects –Course-embedded research projects –Potential benefits Potential benefits

• Individual student grades

• Introduction into culture of research

• Venue for developing scientific communications skills

• Relatively noninvasive way to address degree program learning outcomes

Igneous and Metamorphic Petrology:Igneous and Metamorphic Petrology:Class-based Petrologic Research in an Class-based Petrologic Research in an

Outcrop-challenged State?Outcrop-challenged State?

You need to adapt!You need to adapt!

Pet Rock Project: 1996-presentPet Rock Project: 1996-presentSemester-long project

(Igneous and Metamorphic Petrology) integrated in fabric of course

• “Field experience” - random selection of rock in a box

• Research experience - processing pet rock sample, petrographic and analytical data acquisition, and petrologic interpretation

• Communication skills - professional-level written report and oral presentation

Samples – “Field work”Samples – “Field work”

Archean rocks from Beartooth Mtns, MT• Great diversity of rocks• Geologic complexity

Virtual field context• Field photos• Google Earth

Ownership and Responsibility of Pet RockOwnership and Responsibility of Pet Rock

Each sample is given a personal name and introduced to class• Examples: “Pierre”, “Bubba”,

“Freckles”, “Big-Al”…

Each student processes sample for thin sectioning

Introduction to Tools Available for Introduction to Tools Available for ProjectProject

Simple tools• Optical microscopyOptical microscopy• Optical CLOptical CL

Advanced tools• SEM imaging - BSE• EDS (mineral ID), electron

microprobe spot analyses and data handling

Beartooth Mtns Sample – Example Beartooth Mtns Sample – Example Petrographic Information Petrographic Information

Peraluminous migmatite

• Assemblage: Sil + Grt + Crd + Bt + Qtz + Pl + Kfs + Rt

• Clustering of Sil and Bt

• Interpreted as a metatexite

Beartooth Mtns Sample – Example Beartooth Mtns Sample – Example Analytical data and Interpretation Analytical data and Interpretation

Peraluminous migmatite

• Quantitative spot analyses

• Mineral normalization schemes

• Application of TWQ - peak metamorphic conditions

• Consistency with biotite dehydration melting scenario

• Possible retrograde path

Communicating the Science Communicating the Science

How do we make the science

known?

CxC mission: Enhancing learning experiences for students and improving their written, spoken, visual presentation, and technological communication skills

LSU CxC Initiative – Communication-LSU CxC Initiative – Communication-intensive certificationintensive certification• communication for learning and for formal

sharing of ideas publicly

• Emphases on at least 2 of the CxC components: written, spoken, visual, or technological communication

• Student/faculty ratio of 35:1

• Genres appropriate to discipline or profession

• Class time spent on communication (and revision required)

• Faculty member involvement in evaluation

• 40% of course grade based on communication

Igneous and Metamorphic Petrology CI Igneous and Metamorphic Petrology CI components (oral and written)components (oral and written)

Oral presentationsOral presentations• Preparation: effective oral presentation by

CI studio coordinator (week 1)

• Volcano Presentation (week 3)

• Pet Rock Project Presentation – embedded research project (week 13 and 15)

Written contributionWritten contribution• Pet Rock Project – 10+ page research

project written in the style of a professional petrology paper (week 13 and 15)

• Day-long Igneous and Metamorphic Petrology Symposium

• Symposium program• 12 minute talk with 3 minute

question period• Student-moderated• Student evaluated with

evaluation forms used previously

Professional-style oral presentation expectationsProfessional-style oral presentation expectations

Pet Rock – Speaking at the Igneous and Pet Rock – Speaking at the Igneous and Metamorphic Petrology SymposiumMetamorphic Petrology Symposium

presentation elementspresentation elements1. Cohesive design of background and clear, readable text 2. Effective use of space on slides3. Appropriate use of color4. Images are illustrative and have the correct citations5. Use of allotted time6. Linking ideas/transitions 7. Diction (enunciation, volume, clarity)8. Correct usage of words/terms without misspellings/grammatical errors 9. General style, liveliness, body language10. Appropriateness of dress for a professional presentation

content expected in a professional oral presentationcontent expected in a professional oral presentation11. Organization and structure of content 12. Introduction - too short/too long/just right 13. Material presented at the appropriate level of the audience14. Relevance of data to conclusions15. Comprehension and knowledge of field

Rubric for evaluation of oral Rubric for evaluation of oral communications communications (instructor/students)(instructor/students)

Components/content expected in Components/content expected in professional geology paperprofessional geology paper

5. Title6. Abstract 7. Introduction8. Geologic Setting9. Sample preparation and analytical

procedures 10. Results11. Discussion/conclusions 12. Tables 13. Acknowledgements 14. References cited 15. Appendices

Practice of writingPractice of writing1. Formatting requirements

2. Figures

3. Paragraph construction

4. Mechanics of writing – Grammar, spelling and punctuation

Rubric for evaluation of written Rubric for evaluation of written communications communications (instructor)(instructor)

Individual student assessment

Each category weighted depending on relative importance

LSU Geology BS learning outcome:LSU Geology BS learning outcome:““Students will develop the ability to effectively Students will develop the ability to effectively communicate geologic concepts and material to the public communicate geologic concepts and material to the public and professional colleagues in written and oral formats”and professional colleagues in written and oral formats”

2009 – Igneous and Metamorphic Petrology is designated means of 2009 – Igneous and Metamorphic Petrology is designated means of assessmentassessment

Transforming student scoring rubrics Transforming student scoring rubrics into degree program assessmentinto degree program assessment

Repurpose the individual scoring rubric to consider each category Repurpose the individual scoring rubric to consider each category • Goal: direct assessment of learning outcome • Renormalize on a 5-point basis• Investigate the group statistics• Develop threshold levels (and then re-examine)

Assessment results and feedbackAssessment results and feedbackInitial information/basis for improvementInitial information/basis for improvement

Inter-rater reliabilityInter-rater reliability

• Common projects (9 students) in two classes

• Two independent evaluations of writing from two faculty members

• Carefully define function of learning outcome(s) (e.g. course vs. degree program).

• Be proactive with on-campus assessment groups to establish what is right for your program

• Build course-embedded projects – i.e. use resources you have available

• Have enough detail in rubric to examine essential components of learning outcome(s)

• Transform from individual student scoring to a rubric useful for assessment

• Use results to improve course/program or assessment procedure

Lessons learnedLessons learned

• Fava, C.H. and Henry, D.J. (2009) Professional Communications Projects: Training science students to communicate. Pedagogy in Action, Science Education Resource Center (SERC) website. http://serc.carleton.edu/sp/library/communications_curricula/index.html.

• Henry, D. J. (2009) Rubrics in a communications-intensive geology-majors course: Their roles in departmental assessment. Workshop of “Assessing Geoscience Programs: Theory and Practice” "On the Cutting Edge – Geoscience Programs: Developing Pathways to Strong Programs for the Future" series. http://serc.carleton.edu/departments/program_assessment/program.html

• LSU Communications across the Curriculum Program. http://www.cxc.lsu.edu/Home.html

ResourcesResources