Is there a better model for P&T preparation and evaluation of geoscience education research (GER) in geoscience

departments?

Kristen St. JohnJames Madison University

stjohnke@jmu.edu

Motivation & Context:Personal Experience

St. John and Leckie, http://serc.carleton.edu/NAGTWorkshops/earlycareer/tenure/preparing.html

St. John (pseudonym Jones; 2007), http://chronicle.com/article/Tenured-Twice/46472/

1940 1950 1960 1970 1980 1990 2000 2010 20200

20

40

60

80

100

120

140

160

0

100

200

300

400

500

600

700

800

# ar

ticle

s

Year

# pa

ges

…there appear[s] to have been a noticeable increase in the frequency of research articles within the last 10 years. There is also an increase in both the JGE and the wider literature in research papers on the subject of cognitive issues in teaching and learning. – Pilburn, van der Hoeven Kraft, and Pacheco, (2011), A New Century for Geoscience Education Research. A commissioned study to inform the 2012 DBER Report.

Journal of Geoscience Education publication statistics

Motivation & Context:Publication Trends

Data compiled by Erica Zweifel

Change in pub.

standards/types of articles

Geoscience Education Research (GER): Graduate Programs by InstitutionProgram MS/PhD College Approximate

timeline*Arizona State MS/PhD Science 1990

Mississippi State MS/PhD Science 2007

Michigan State MS/MACT/PhD Science & Education 1979

North Carolina State MS/PhD Science 2005

Northern Colorado MA Science 1990

Purdue MS/PhD Education 1990

Texas A&M MS/PhD Science 1992

University of South Florida MS/PhD Science 1971

Western Michigan MA Education 2005

Western Washington MS Science 2000

University of Northern Iowa MS, MA Education 2005

University of Wyoming MS/PhD Education 2009

University of Nebraska MS/PhD Science 2011

Northern Illinois MS/PhD Education 2013

*based on approximate hire dates of lead

facultyData compiled by Karen McNeal; primary source: Libarkin, https://www.msu.edu/~libarkin/about_programs.html

Motivation & Context: Formal DBER Training Opportunities

Motivation & Context: More Geoscience Education Tenure-Track Positions

Motivation & Context: Shared concerns about P&T evaluation by DBER journal editors

EducationalPsychology

Geoscience

Teaching Practice

X

Source: Lukes et al., JGE, 2015

Real and time-sensitive issue for tenure-track geoscience education researchers in traditional geoscience programs. • P&T evaluation of GER

faculty is largely conducted by non-GER geoscience faculty and administrators.

Motivation & Context:Potential for GER marginalization/isolation in the geosciences.- Lukes et al., 2015; Feig 2013; and DBER report 2012, see esp. Dave Mogk’s summary http://serc.carleton.edu/NAGTWorkshops/DBER.html

P&T and GER Challenges: Appreciating the scope and methods of geoscience education research

Feig and Stokes, 2011, GSA Special Paper v. 474.

Doberneck et al., 2012. Beyond Activity, Place, and Partner: How Publicly Engaged Scholarship Varies by Intensity of Activity and Degree of Engagement, Journal of Community Engagement and Scholarship, v. 4.

Publicly Engaged Scholarship

P&T and GER Challenges: Measuring Impact

See Julie Libarkin’s Blog on Journal Impact Factors, and references therein:https://geocognitionresearchlaboratory.wordpress.com/2014/08/16/citations-and-impact-who-says-your-research-is-valuable/

Thomson Reuters Science Citation Index (SCI) selected 2014 Journal Impact Factors

• The impact factor of an academic journal is a measure reflecting the average number of citations to recent articles published in that journal.

• Different playing fields can lead to misleading comparisons (e.g., expectation of external citations for DBER fields).

• Not all journals are indexed in Thomson Reuters (none in geoscience education), thus no SCI-impact factor.

P&T and GER Challenges: Measuring Impact

One of the more important goals of DBER is translating research into practice. -DBER report, 2012.

• P&T Deliverable programmatic reform of teaching/student learning.

–Feig, 2013

• This is a P&T deliverable that does not have parallels in basic traditional geoscience research. How can this impact be demonstrated?

Geoscience education research results

Reform of geoscience education

teaching and student learning

institutional level

national level

P&T and GER Recommendations:From the Community

• Emphasize high quality of DBER [GER] work -DBER report 2012, Mogk’s summary http://serc.carleton.edu/NAGTWorkshops/DBER.html

• Clarify expectation for DBER [GER] faculty positions

-DBER report 2012, Mogk’s summary http://serc.carleton.edu/NAGTWorkshops/DBER.html

• Targeted marketing to non-GER colleagues – Feig, 2013

• Be an active part of a GER community of practice – Lukes et al, 2015

Improve communication & define metrics of GER success

Tenure-track GER Faculty• Talk with department colleagues early and

often about what you do, how you do it, and why it is important - St. John and Leckie, 2009, http://serc.carleton.edu/NAGTWorkshops/earlycareer/tenure/preparing.html

– Avoid jargon.– Maintain respect for colleagues.

• wisdom on geoscience teaching and learning isn’t exclusive to those with geoscience education PhDs.

– Find common ground.• Geoscience itself.• University mission, program goals

P&T Committees• Make informed decisions on what “counts”.

– Go beyond Impact Factors (e.g., download statistics)– How to measure “translation of research into practice”?

P&T and GER Recommendations:Additional Suggestions

P&T and GER Recommendations:Evolving the P&T Preparation and Evaluation Model

Traditional Boyer (1990, Scholarship Reconsidered)

Teaching

Research

Service

Teaching Research(Discovery)

Service(Application)

Integrationof Knowledge

Integration of knowledge “making connections across disciplines, placing specialties in a larger connect, illuminating data in a revealing way, often to educate non-specialists too.”

http://www.schev.edu/AdminFaculty/OFA/2016OFAGuidelines.pdf

P&T and GER Recommendations:Evolving the P&T Preparation and Evaluation Model

An Integrated (modified Boyer) Approach

Teaching

Service(Application)

Research(Discovery)

Integrationof Knowledge

Research results should advance the GER field• Publications• Grant success

Apply expert GER knowledge to• department (e.g., curriculum com.)• institution (e.g., Center for Teaching)• GER community (e.g., NAGT GER

Division)• Geo community (e.g., Broaden

Impacts on geo. grants, lead professional development workshops)

Use GER best practices • Evidence-based

teaching/ model “translating research into practice.”)

• Impact on student learning

• In Career Planning• In P&T Narrative• In P&T Evaluation

Criteria

Metrics for Success