Upload
rosemary-stevens
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Working Together and Seeking Generalized Results: Secondary Analyses and Syntheses*
Larry D. YoreUniversity of VictoriaTaipei, Taiwan March 6, 2008
(*see Rossman & Yore, chapter in progress, Springer)
2
Key IdeasBased on medical and health sciences research and some work done in the social sciences.Clear Definitions of Target GoalsAssessment of Alignment and Selection CriteriaQuilting as Quantitative Secondary Analyses
Meta-analysesSecondary Reanalyses
Crazy Quilting as Qualitative SynthesesSystematic ReviewsSecondary ReanalysesCase-to-case ComparisonsMetasyntheses
3
Looking Outside of Science Education (also see Yore & Boscolo, in press)
NCLB and Gold Standard for Educational Research
Partially based on the public’s disappointment with education research to address pressing issuesPartially based on the skepticism that politicians and bureaucrats have toward education research
Advances in seeking and finding generalizations in medicine, health care, and social sciences
4
Clarifying Targets
Mathematics, Scientific, or Technological LiteracyNature of Mathematics, Science, and TechnologyLanguage in Doing, Learning, and Teaching ScienceLearning Outcomes, Measures, and Methods
5
Interacting Senses of Mathematical Literacy — Cognitive Symbiosis (Yore, Pimm, & Tuan, 2007)
Fundamental Sense Derived Sense
Cognitive and Metacognitive Abilities
Understanding the Big Ideas, Strands, and Substrands of Mathematics
Mathematical Thinking and Quantitative Reasoning
Nature of Mathematics
Emotional Dispositions/ Habits of Mind
Knowledge about Problem Solving
Language of Mathematics (including proofs as arguments)
Real-world Problems
Information Communication Technologies (ICT)
6
Interacting Senses of Scientific Literacy — Cognitive Symbiosis (Yore, Pimm, & Tuan, 2007)
Fundamental Sense Derived Sense
Cognitive and Metacognitive Abilities
Understanding the Big Ideas and Unifying Concepts of Science
Critical Thinking/Plausible Reasoning
Nature of Science
Habits of Mind Scientific Inquiry
Scientific Language (including mathematical language)
Technological Design
Information Communication Technologies (ICT)
Relationships among Science, Technology, Society, and Environment (STSE)
7
Interacting Senses of Technological Literacy — Cognitive Symbiosis (Pacific Crystal Group, in progress)
Fundamental Sense Derived Sense
Cognitive and Metacognitive Abilities
Understanding the Big Ideas of Technology: Design Principles, Real-world Problems, and Constraints
Critical and Creative Thinking
Nature of Technology
Emotional Dispositions/ Habits of Mind
Knowledge about Research and Development, Design, and Creative Problem Solving
Language of Technology Technology and Engineering in Daily Life and Democratic Society
Information Communication Technologies (ICT)
8
Modern View of Science: Naïve Realist Ontology and Evaluativist Epistemology(Yore, Hand, & Florence, 2004)
Science knowledge is a temporary explanation that best fits the existing evidence, established knowledge, and current thinking about reality as we know it.
9
Modern View of Science (continued)
Science knowledge claims are open to repeated public evaluation
Language is not a exact transcription of scientific inquiry — Science reports are not records of actual actionsLanguage shapes as well as reports science ideas — Scientists use metaphors to capture their mental images, and the metaphor starts to influence their thinkingLanguage must reflect scientific inquiry and the tentative and temporary nature of claims — Science does not ‘prove’; it only rejects or supports hypotheses
10
Language of ScienceThe 3–Language Problem: Home language (L1), school language (L2), and science language (L3) frequently do not match(Yore & Treagust, 2006)
Metalanguage of Science and scientific epistemology and ontology: Theory, hedges, physical causality, explanations, etc.Instruction must start with the student’s home language and move toward the languages of instruction and science
11
Alignment of Outcomes and Methods
Learning OutcomesMeasures: Items, questions, and tasks
CommonUnique
MethodsInterviewsWork samples, artifacts, etc.Observations
12
Learning Matrix (Shymansky, n.d.)
Meaningful
Level ofLearning
Rote Drill Argument
Process of Learning
13
Oral Language that Promotes Cognitive Symbiosis
Learning to talk/argue and talking/arguing to learnLittle meaningful oral discourse occurs in most science classrooms:
Most oral discourse in classrooms does not reflect scientific discourse and metalanguageIt is social, not focused and purposeful
14
Talking Science
Patterns of Verbal Interactions(Flanders, 1964; Shymansky, 1978)
Traditional Science Lesson• One-way: lecture• Two-way and unidirectional: teacher to
student (t-s)
Inquiry Science Lesson• Two-way and multi-directional: t-s, s-t, s-s
15
Classroom QuestioningTeacher questioning needs to reflect the phase and purpose of inquiry
Wait-time: 3 seconds among question, response, and further questions (Rowe, 1996)
Use specific types of questions for specific purposesChained Questions: Response and rationale
Debating Science, Technology, Society, and Environment IssuesArgumentation: The process of argument
16
Producing and Interpreting Science Text
Writing to readWrite and represent ideas first (knowledge construction), reading what is written (making sense of text) in early grades
Writing-to-learn and Reading-to-learn Science
Science Writing HeuristicVisual Representations and Transformation between Modes of RepresentationMetacognitive Awareness and Executive Control of Science Reading and Making Sense of Text
17
Debates and Arguments Involving Science, Technology, Society, and Environment Issues
STSE issues provide ill-structured problems, multiple solutions, and rich contemporary contexts involving trade-offs among science, technology, and societal valuesStructured Controversy: Apply authentic debating procedures (Johnson & Johnson, 1985)
King’s College London Project (Osborne, Erduran, & Simon, 2004; Simon, Erduran, & Osborne, 2006)
18
Structured Controversy(Johnson & Johnson, 1985)
DebateEvaluateChange Position and Debate AgainDraw Consensus and Plan Action
19
Scientific Arguments(Osborne, Erduran, & Simon, 2004)
Elements of ArgumentationClaimsEvidenceWarrantsBackingsCounter-claimsQualificationsRebuttals
20
Extended Pattern of Argumentation (Toulmin, 1958)
Evidence Qualifiers and Claims Counter-claims
Warrants Rebuttal
Backings
21
Example of an Extended Argument
Examination of:AIDS and HIV in HIVhealthy some causespatients people AIDS
HIV was found Peoplein all AIDS with weakpatients and some immunehealthy patients systems
22
Research into Language and Science Understanding (Tippett, in press)
Links between argumentation ability, quality of arguments, and science understandingUse of software scaffolds for improving argumentation and learningInterpretative frameworks and scoring rubrics to evaluate arguments and explore their effects
23
Developing Research Results and Reports to Persuade Politicians, Policy Makers, and Decision Makers
Academic demands of universities and funding agencies appears to influence research approaches toward short ‘one-shot’ inquiriesPopular beliefs that learning and teaching is situated and total unique has encouraged case studies and contextual focused studiesPoliticians and bureaucrats view these studies as not applying to their problems and constituents
24
Selection CriteriaAvailable studies focus on similar research questions, outcomes, measures, methods, etc.
Look for common tests, items, interview questions, performance tasks, instruction artifacts.
Collect published and unpublished reports of studies from journals, ERIC, theses, dissertations, client reports, etc.
Screening for quality should not be left to publication status.Follow-up via email with authors to clarify quality issues and elaborations of results.
25
Quantitative Secondary Analyses as Quilting
Cat Patternhttp://www.nmia.com/~mgdesign/
26
Quantitative Meta-analysis (Shymansky, Hedges, & Woodworth, 1990)
Summarization of results: Differential effects between control-treatment studies producing an average effectSummation of individual standardized differences in experiment-control designs: Unweighted or weighted sum of difference in gains/standard deviation (1 to n) divided by the number of studies (n)
Effect size #1 + Effect size #2 + … + Effect size #n divided by number of studies (n) orN1 x ES #1 + … + Nn x ES #n / N1 + … + Nn
27
Quantitative Reanalysis(Gunel, Hand, & Prain, 2007)
Combined datasets for similar individual studies focused on same outcome, using similar designs, or anchored measuresStandardize outcome measuresDevelop a collective dataset of the standardized dataAnalyze the combined dataset with combined sample size
28
Qualitative Summarization as Crazy Quilting
http://www.nmia.com/~mgdesign/qor/styles/crazy/crzayqlt.htm
29
Qualitative Systematic Review
Value AddedSelection CriteriaCritical Summary and Interpretation
New developmentsTheory and theoretical frameworkFuture lines of research
Summary is recursive, grounded in the data and interpretative framework
30
Case-to-case ComparisonAnalytical Generalization
Theory-driven predictions systematically verified against the results of several qualitative case studies to demonstrate valid applications and to map the problem space (Academies for Young Scientists)
Case-to-case SynthesisMultiple case studies of co-authoring (Florence & Yore, 2004)
Multiple case study of science teachers’ beliefs, instructional decisions, and implementation of literacy events in science (Dillon, O’Brien, Moje, & Stewart, 1994)
31
Qualitative ReanalysisCombined qualitative information from similar studies focused on same outcome, using similar designs, or anchored measures (Hand & McDermott, 2008)
Develop a collective text file or video file of these qualitative dataAnalyze the combined text or video files with a new or unified interpretative framework or method (discourse analysis, text analysis software, video analysis systems)
32
Qualitative Synthesis (Finfgeld, 2003)
Using the results of qualitative studies on similar issues, using similar data sources and collection techniques, and anchor itemsSystematically combine the results of selected studiesNational Literacy Panel (August & Shanahan, 2006)
33
References (not included in handout)Flanders, N. A. (1964). Some relationships among teacher influence, pupil attitudes, and achievement. In B. J. Biddle & W. J. Ellons (Eds.), Contemporary research on teacher effectiveness (pp. 196-231). New York: Holt, Rinehart & Winston.
Johnson, R. T., & Johnson, D. W. (1985). Using structured controversy in science classrooms. In R. W. Bybee (Ed.), Science technology society: 1985 yearbook of the National Science Teachers Association (pp. 228-234), Washington, DC: National Science Teachers Association.Norris, S. P., & Phillips, L. M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87, 224-240.
34
References (continued)
Novak, J. D., & Gowin, B. D. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press.Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science, Journal of Research in Science Teaching, 41, 994-1020.Rowe, M. B. (1996). Science, silence, and sanctions. Science and Children, 34(1), 35-38.
35
References (continued)
Shymansky, J. A. (1978). Assessing teacher performance in the classroom: Pattern analysis applied to interaction data. Studies in Education Evaluation, 4, 99-106.
Toulmin, S. (1958). The uses of argument. Cambridge, UK: Cambridge University Press.Wellington, J., & Osborne, J. (2001). Language and literacy in science education. Philadelphia, PA: Open University Press.
36
References (continued)
Tippett, C. (in press). Argumentation: The language of science. Journal of Elementary Science Education.Rossman, G.B. & Yore, L.D. (in progress). Stitching the pieces together to reveal the generalized patterns: Systematic research reviews, secondary reanalyses, case-to-case comparisons, and metasyntheses of qualitative research studies. In M. Shelley, B. Hand, & L.D. Yore (Eds.). Education Research Meets the “Gold Standard”: Evaluation, Research Methods, and Statistics after No Child Left Behind (pp. tba), Springer.
37
References (continued)
Yore, L.D. & Boscolo, P. (in progress). Why “Gold Standard” Needs Another “s”: Results from the Gold Standard(s) in Science and Literacy Education Research Conference. In M. Shelley, B. Hand, & L.D. Yore (Eds.). Education Research Meets the “Gold Standard”: Evaluation, Research Methods, and Statistics after No Child Left Behind (pp. tba), Springer.Yore, L.D. & Lerman, S. (in press). A call for systematic reviews, secondary analyses, generalizations, and meta-syntheses. International Journal of Science and Mathematics Education.
38
References (continued)
Yore, L.D., Hand, B.M., & Florence, M.L. (2004). Scientists’ views of science, models of writing, and science writing practice. Journal of Research in Science Teaching, 41, 338-369.Yore, L.D. & Treagust, D.F. (2006). Current realities and future possibilities: Language and science literacy--empowering research and informing instruction. International Journal of Science Education, 28, 291-314.Yore, L.D., Pimm, D., & Tuan, H-L. (2007). The literacy component of mathematical and scientific literacy. International Journal of Science and Mathematics Education, 5, 559-589.