Conducting Qualitative Research Remotely:
Investigating Conceptual Understanding of Chemistry
Outreach Practitioners
Justin M. Pratt & Ellen J. Yezierski
Methods in Chemistry Education Research - Session 2
June 11, 2020
Getting on the Same Page – Let’s Talk about Terms
• Outreach
• Science Outreach
• Informal Science Education
• Chemistry Outreach
• Informal Chemistry Education
• Communicating Chemistry in Informal Environments
• Working definition: Chemistry Outreach is any event that occurs outside of a normal classroom setting designed to increase interest, understanding, and/or involvement in chemistry. Targeted audiences are wide ranging and can include the general public, children/students, and/or teachers.
2
Wang, L. Highlights from National Chemistry Week 2019. Chemical & Engineering News, 2019, 97(48). https://cen.acs.org/acs-news/programs/Highlights-National-
Chemistry-Week-2019/97/i48
Examples of Chemistry Outreach
• Public Lectures
• Science Festivals
• Informational Videos
• Demonstration Shows (sometimes
referred to as “magic shows”)
• Afterschool Activities
3
Wang, L. Highlights from National Chemistry Week 2019. Chemical & Engineering News, 2019, 97(48). https://cen.acs.org/acs-news/programs/Highlights-National-
Chemistry-Week-2019/97/i48
Unique Population of Outreach Practitioners• College students involved in extracurricular student organizations that conduct
outreach voluntarily during personal time
• Two major collegiate chemistry organizations:
• Student Chapters of the American Chemical Society
• Collegiate Chapters of the Alpha Chi Sigma Fraternity
• Reach approximately 1 million outreach event attendees each year
• National Survey (N = 206)
• Goal: Audience Learning
• Common Activities: Elephant Toothpaste, Liquid Nitrogen Ice Cream, Making Slime
• Unanswered Question: How accurate and in-depth is their chemistry content knowledge?
Connelly, T. M. (2015) Why Is Chapter Community Outreach So Important? inChemistry, 24 (1), 3.
Pratt, J. M. (2017) Alpha Chi Sigma and Chemistry Outreach: Promoting the Second Object. The HEXAGON, 108 (1), 8–9.
Pratt, J. M., & Yezierski, E. J. (2018). Characterizing the landscape: Collegiate organizations’ chemistry outreach practices. Journal of Chemical Education, 95(1), 7-16.
4
Qualitative Data Collection in an Online Environment
• Need access to research participants
• Gatekeepers
• Need to elicit desired data
• Need to develop rapport with participants
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
Access Elicitation
Rapport
5
Accessing Our Research Participants – Story of Flexibility
• Goal of Maximum Variation Sampling
• Multiple recruitment techniques with varied success
• Gatekeepers were primary contacts
• Email Faculty Advisor to student organization
• Email Department Chair at institution
• Testing for data saturation with targeted recruitment
from awarded chapters
Patton, M. Q. Qualitative Research and Evaluation Methods, 3rd ed.; Sage Publications, Inc.: Thousand Oaks, CA, 2002.
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
6
Diversity of Sample
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
Year in School n Major n
Sophomore (Second Year) 5 Chemistry or Biochemistry 22
Junior (Third Year) 11 Science (Non-Chemistry) 11
Senior (≥ Fourth Year) 19 Non-Science 2
Graduate Student 2 Chemistry Graduate Student 2
Sex n
Males 17
Females 20
School Type n
Public School 17
Private School 5
22 different universities
7
Eliciting Desired Data
• Multimedia-based semi-structured interviews
• Average duration = 1.5 hours
• Transcribed verbatim
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
8
Eliciting Conceptual Understanding
Research Question: For this sample of college student outreach practitioners,
how accurate and in-depth is their chemistry content
knowledge?
Elephant
Toothpaste
9
What is the Elephant Toothpaste Reaction?
Elephant Toothpaste – Tulsa City County Library. Tulsa Children’s Museum Discovery Lab. https://events.tulsalibrary.org/event/83253
10
What is the Elephant Toothpaste Reaction?
30% H2O2 KI Dish Soap
11
What is the Elephant Toothpaste Reaction?
30% H2O2 KI Dish Soap
https://www.youtube.com/watch?v=p1eG2y2mn54
12
Pilot Study Results
Participants had prior experience facilitating the demonstration with an audience prior to the interview.
What chemicals go into elephant toothpaste?
“I don't remember exactly.”
Barbara (Senior/PUI/Chem)
“Hydrogen peroxide and…something else that you mix together”
Marie (Senior/Large/Bio)
“I don't remember that one.”
Lois (Senior/PUI/Chem)
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
13
Developing a Novel Critiquing Task
Oral Prompt:
We’ve received different explanations from other students, and they contain a mixture of accurate and inaccurate ideas.
We want you to go line by line and critique these explanations for:
1) accuracy of the content, is it correct or incorrect?
2) age appropriateness for intended audience, is it appropriate or not?
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
14
General Chemistry Prompt: Elephant Toothpaste
This reaction involves the catalytic decomposition of hydrogen peroxide into
water and hydrogen gas. This acid-base reaction is an exothermic reaction
because bonds are broken and heat is released. A catalyst is used because the
decomposition is not spontaneous. The catalyst allows the reaction rate to
increase because the mechanistic pathway changes. The catalyzed mechanism
has two steps with higher activation energies. Overall, the catalyst decreases
the overall enthalpy change of the reaction. The reaction starts off slow because
the first step is the rate limiting step. Soap is used to help break down the
hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the
reaction dramatically speeds up as noted by the increase in foam being
produced. Since the products are gas, the foam expands as the gas molecules
inside the foam spread out.
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
15
General Chemistry Prompt: Elephant Toothpaste
This reaction involves the catalytic decomposition of hydrogen peroxide into
water and hydrogen gas. This acid-base reaction is an exothermic reaction
because bonds are broken and heat is released. A catalyst is used because
the decomposition is not spontaneous. The catalyst allows the reaction rate to
increase because the mechanistic pathway changes. The catalyzed mechanism
has two steps with higher activation energies. Overall, the catalyst
decreases the overall enthalpy change of the reaction. The reaction starts
off slow because the first step is the rate limiting step. Soap is used to help
break down the hydrogen peroxide. Once all of the catalyst is converted to
the intermediate, the reaction dramatically speeds up as noted by the increase
in foam being produced. Since the products are gas, the foam expands as the
gas molecules inside the foam spread out.
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
16
General Chemistry Prompt: Elephant Toothpaste
This reaction involves the catalytic decomposition of hydrogen peroxide into
water and hydrogen gas. This acid-base reaction is an exothermic reaction
because bonds are broken and heat is released. A catalyst is used because
the decomposition is not spontaneous. The catalyst allows the reaction rate to
increase because the mechanistic pathway changes. The catalyzed mechanism
has two steps with higher activation energies. Overall, the catalyst
decreases the overall enthalpy change of the reaction. The reaction starts
off slow because the first step is the rate limiting step. Soap is used to help
break down the hydrogen peroxide. Once all of the catalyst is converted to
the intermediate, the reaction dramatically speeds up as noted by the increase
in foam being produced. Since the products are gas, the foam expands as the
gas molecules inside the foam spread out.Pilot study
Goal of very obvious inaccuracies
Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569.
Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.
17
General Chemistry Prompt: Elephant Toothpaste
This reaction involves the catalytic decomposition of hydrogen peroxide into
water and hydrogen gas. This acid-base reaction is an exothermic reaction
because bonds are broken and heat is released. A catalyst is used because
the decomposition is not spontaneous. The catalyst allows the reaction rate to
increase because the mechanistic pathway changes. The catalyzed mechanism
has two steps with higher activation energies. Overall, the catalyst
decreases the overall enthalpy change of the reaction. The reaction starts
off slow because the first step is the rate limiting step. Soap is used to help
break down the hydrogen peroxide. Once all of the catalyst is converted to
the intermediate, the reaction dramatically speeds up as noted by the increase
in foam being produced. Since the products are gas, the foam expands as the
gas molecules inside the foam spread out.Pilot study
Goal of very obvious inaccuracies
Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569.
Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.
18
Eliciting Research Participant Ideas
• Initial ideas about elephant toothpaste:
‘‘I don’t know what’s goin’ on’’ – Shayera
• Response to critiquing task:
‘‘Uhm...‘overall the catalyst decreases the overall enthalpy change of the
reaction’...yes? Yes, but only [as a] result...so...the catalyst decreases the
activation energy which leads to a decrease in overall enthalpy, but a catalyst
does not directly change enthalpy. The enthalpy changes as a result of the
decreased activation energy.’’ – Shayera
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
19
General Chemistry Prompt: Elephant Toothpaste
This reaction involves the catalytic decomposition of hydrogen peroxide into
water and hydrogen gas. This acid-base reaction is an exothermic reaction
because bonds are broken and heat is released. A catalyst is used because
the decomposition is not spontaneous. The catalyst allows the reaction rate to
increase because the mechanistic pathway changes. The catalyzed mechanism
has two steps with higher activation energies. Overall, the catalyst
decreases the overall enthalpy change of the reaction. The reaction starts
off slow because the first step is the rate limiting step. Soap is used to help
break down the hydrogen peroxide. Once all of the catalyst is converted to
the intermediate, the reaction dramatically speeds up as noted by the increase
in foam being produced. Since the products are gas, the foam expands as the
gas molecules inside the foam spread out.Pilot study
Goal of very obvious inaccuracies
Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569.
Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.
20
Eliciting Research Participant Ideas
• Initial ideas about elephant toothpaste:
‘‘It’s been a long time since chem two!...It’s not knowledge I deem important
enough to keep in my brain on a daily basis.’’ – Sue
• Response to critiquing task:
‘‘The line a couple of sentences from the end ‘once all of the catalyst is
converted to the intermediate’ uhm…catalysts don’t change so that’s super
wrong.”– Sue
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
21
Summary of Method
• Access
• Requires flexibility and multiple strategies
• Leads to greater sample diversity, adds to transferability
• Elicitation
• Requires flexibility and multiple strategies
• Data is commensurate with in-person interviews
• Rapport
• Evidence that rapport can successfully
be established online
“If we’ve learned anything today is that...[Max]
doesn’t remember chemical reactions’!’ – Max
“Obviously I’m a girl, a female in science
so…[laughter] I totally get that one!’’ – Lois
22
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
Technology Limitations and Considerations
• Freezing and/or dropped connections (n = 12 / 37)
• Audio/video skipping (N = 37)
• No evidence of hindered rapport
• Flexibility and participant choice
• Skype, Google Hangouts, Facebook Video
Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying
chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.
23
Audience Participation!
Breakout Rooms
1. Introduce yourself (Name & Institution).
2. Open chat window to see discussion questions.
3. Pick a discussion question and think about access, elicitation, and/or rapport for remote qualitative research:
a. What types of research questions align with these methods?
b. What is important to consider if using these methods during the COVID-19 pandemic?
4. Identify 1 person to summarize your group’s conversation into 1-2 sentences (or a question) to share when we get back to main Zoom room.
5. Discuss!!
Participants will have ~10 minutes to discuss
24
Report Out!
• Group Summarizers: Please summarize your group's discussion or pose a
question in the chat. Please keep it short!
• Think about access, elicitation, and/or rapport for remote qualitative research:
a. What types of research questions align with these methods?
b. What is important to consider if using these methods during the COVID-19 pandemic?
25
Acknowledgments
• Michael Seery and Aishling Flaherty
• Research participants from ACS and ΑΧΣ chapters
• ACS Undergraduate Programs Office
• ΑΧΣ Supreme Council
• Yezierski & Bretz Research Groups at Miami University
• Raker Research Group at the University of South Florida
26
1. How accurate and in-depth are collegiate students’
chemistry content knowledge as related to
common chemistry outreach activities?
2. How well do collegiate students adjust their
explanations of chemistry outreach
activities/demonstrations based on the audience?
3. How do these college students develop skills and
expertise to facilitate chemistry outreach events
for varying audiences?
Conceptual Understanding
Pratt, J. M., Yezierski, E. J. (2018).
J. Chem. Educ. 95(12), 2091-2102.
Pratt, J. M., Yezierski, E. J. (2019).
J. Chem. Educ. 96(2), 203-212.
Pratt, J. M., Yezierski, E. J. (2019).
J. Chem. Educ. 96(3), 414-422.
27
Findings for Chemistry Outreach Study