Kilgo Research Model - HISD Board

Kilgo Research Model for Mathematics, Science, Social Studies, and English/Spanish Language Arts/Reading

Interpreting and Developing a Deeper

Understanding of State and National

Curriculum Standards from the

Viewpoint of Standards-Based

Assessments

Margaret KilgoKilgo Consulting, Inc.2011-2012

Kilgo Research Model

Research Focus and PurposeStudy and research state curriculum standards from the viewpoint of state standards-based assessments.

Research Process RequirementsResearch the relationship between state curriculum standards (TEKS/SEs) and the test items/questions and correct answers only on the assessment without considering multiple-choice wrong answer options.

Research process is based on the Open-Ended Response (OER) or Constructed-Response Assessment Process, NOT the selective or multiple-choice process.

Using the OER Process for research removes the false assumptions of test-taking strategies including the elimination strategy and correctly focuses on thinking, comprehension skills, concepts, context, and vocabulary.

2(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved

Kilgo Research Model – cont.

Research and Interpret State Curriculum Standards from Assessment Viewpoint

Test Item/ Correct Question-----------------------Answer

TEKS/SE OER Research Process No Multiple Choice Review

What are the steps in thinking required to answer the test item/question

correctly?3(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved

Kilgo Research Process

Research State Curriculum Standards, in Texas the TEKS/SEs, From the Viewpoint of the State Standards-Based Assessments:

1st question for state curriculum standards and assessment alignment

What standard (TEKS and Student Expectation) is the test item/question and correct answer testing?

2nd question for cognitive (thinking) alignment Does the verb in the Student Expectation according to the

verb’s dictionary definition align to the steps in thinking required in the test item/question and correct answer relationship?


Kilgo Research Process – cont.

3rd question for concept alignment Is the concept identified in the Mathematics, Science, or the

Social Studies Student Expectation the actual concept tested in the test item/question and correct answer?

Is the reading comprehension skill in the English/Spanish Language Arts/Reading Student Expectation the actual reading comprehension skill tested in the test item/question and correct answer?

4th question for context alignment Does the context in the Student Expectation align with the terms

required in the test item/question and correct answer relationship?

5th question for vocabulary alignment What vocabulary in the test item/question and correct answer

aligns with vocabulary in the Student Expectation? 5(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved

Kilgo Research on SAT and National Council of Teachers of Mathematics (NCTM) Standards

1992-2005: SAT is 94% aligned with the NCTM Standards for Grade 8.

2005-ongoing: Added more Algebra I standards. Currently 70% aligned with the NCTM Standards for Grade 8.

Grade 8 Standards in Measurement are 20% of SAT, ACT, GRE, and GMAT.

Texas Mathematics Curriculum Standards: 100% aligned with NCTM Standards.


Kilgo Research on Time for Student Mastery According to the TEKS/SEs Grade Level Scope

Research Grade Level TEKS/SEs from the Viewpoint of Concepts/Skills Introduced and Tested to Mastery in the Same Year

Research Procedure Review mathematical skills/concepts at each grade level by

TEKS strand.

Count the total number of curriculum standards, i.e., Student Expectations (skills/concepts) at each grade level.

Compute the percentage of Mathematics skills and concepts embedded in the TEKS/SEs introduced and tested in the same year.


Kilgo Research Findings on Mastery Time

Research Findings on Relationship Between TEKS/SEs from the Viewpoint of Mathematics Skills and Concepts Introduced and Tested to Mastery in the Same Year

Grade% Introduced and Tested to

Mastery3 61%4 63%5 47%6 71%7 36%8 38%


Kilgo Research Findings

The concepts in the standard identify “what” is tested on the assessment.

The verbs in the standard in relation to the concept in the standard determine the thinking required to answer the test item correctly.

The verb-concept cognitive alignment is based on the dictionary definition of the verb and is not based on Bloom’s Taxonomy.

Many of the verbs in the standards require much higher-level thinking than the thinking required of students in teaching activities for specific concepts presented in commercial programs and materials.

The standards often specify the terms of the context of exactly how the concept will be tested. 9(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved

Kilgo Research Findings – cont.

The concept-context alignment is often more complex as written in the standard than the context in which the concept is presented to teachers and students in commercial programs and materials.

The vocabulary in the standard is often used in test items. The state standardized assessments align 100% with the exact

wording in the state standards, but not all verbs, concepts, and context in the standard are tested in one test item.

If there are different levels of thinking with 2 or more verbs aligned to a concept in a standard, there may be fewer test items on the higher-level thinking verb.

30-60% of Mathematics concepts in grades 3-8 are introduced and tested to mastery in the same year.

70% of the Mathematics concepts tested on SAT are based on the Grade 8 standards. 10(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved

Applying the Kilgo Research Model in Mathematics

TEKS Statement: 3.7 Patterns, relationships, and algebraic thinking. The student uses lists, tables, and charts to express patterns and relationships. The student is expected to:

SE 3.7 (B): identify and describe patterns in a table of related number pairs based on a meaningful problem and extend the table.

Study and research the TEKS/SE:Verbs: dictionary definition of “identify” and “describe”

Identify: to recognize or establish as being a particular person or thing; verify the identity of

Describe: to give an account of in words; to tell in words what something or someone is like

ConceptContextVocabulary


Applying the Kilgo Research Model in Mathematics – cont.


Research TEKS/SE 3.7 (B) from the Assessment Viewpoint

Identify the verbs tested for the cognitive alignment. Identify the concept tested for the concept alignment. Identify the context requirements tested for the context alignment. Identify the vocabulary in the question.


Research TEKS/SE 3.7 (B) from the Assessment Viewpoint – cont.

What are the critical components of mathematics aligned lesson(s) in teaching this TEKS/SE to the depth and complexity of STAAR and EOC?



TEKS Statement: 4.7 Patterns, relationships, and algebraic thinking. The student uses organizational structures to analyze and describe patterns and relationships. The student is expected to:

SE 4.7 (A): describe the relationship between two sets of related data such as ordered pairs in a table.

Study and research the TEKS/SE:

Verb: dictionary definition of “describe” Describe: to give an account of in words; to tell in words what

something or someone is like

Concept

Context

Vocabulary 15(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved



Research TEKS/SE 4.7 (A) from the Assessment Viewpoint

Identify the verbs tested for the cognitive alignment. Identify the concept tested for the concept alignment. Identify the context requirements tested for the context alignment. Identify the vocabulary in the question. Critical components of an aligned lesson.



TEKS Statement: 7.9 Measurement. The student solves application problems involving estimation and measurement. The student is expected to:

SE 7.9 (B) connect models for volume of prisms (triangular and rectangular) and cylinders to formulas of prisms (triangular and rectangular) and cylinders.

Study and research the TEKS/SE:Verb: dictionary definition of “connect”

Connect: make a logical or causal connection; to join, link, or fasten together; unite or bind)

ConceptContextVocabulary 18(c) Margaret Kilgo, Kilgo Consulting, Inc., all rights reserved



Research TEKS/SE 7.9 (B) from the Assessment Viewpoint

Identify the verbs tested for the cognitive alignment. Identify the concept tested for the concept alignment. Identify the context requirements tested for the context

alignment. Identify the vocabulary in the question. Critical components of an aligned lesson.



TEKS Statement: A.1 Foundations for functions. The student understands that a function represents a dependence of one quantity on another and can be described in a variety of ways. The student is expected to:

SE A.1 (A) describe independent and dependent quantities in functional relationships.

Study and research the TEKS/SE:Verbs: dictionary definition of “describe”

Describe: to give an account of in words; to tell in words what something or someone is like

ConceptContextVocabulary




Research TEKS/SE A.1 (A) from the Assessment Viewpoint

Identify the verbs tested for the cognitive alignment. Identify the concept tested for the concept alignment. Identify the context requirements tested for the context

alignment. Identify the vocabulary in the question. Critical components of an aligned lesson.


Kilgo Research Findings in Reading Comprehension

Personal Schema Reading ComprehensionBased on more than 25 years of research and development of theories of describing and defining the thinking required for reading for meaning based on the critical role of bringing the personal schema of prior knowledge and experiences to the reading selection and reacting to the text based on what the words mean to the reader from the reader’s viewpoint, prior knowledge, and experiences. In this type of reading comprehension the answers to questions vary and the reader’s opinion often can overrule the author’s opinion. Kilgo research refers to this as “Constructivism Reading” in which readers “construct” their own meaning. “C” Reading


Kilgo Research Findings in Reading Comprehension – cont.

Text-Dependent Reading Comprehension (“T” Reading)Standardized assessment must be focused on the requirements of standardization and equity for all students and aligned with the reading comprehension skills in the National Reading Standards. In order to develop a reading comprehension assessment that is fair and equitable for all students, regardless of their background, personal experiences, and academic experiences, all answers must be based on the text, NOT personal experiences and prior knowledge not connected to the text. Closed/text-dependent reading comprehension is the real-life comprehension of the world of work in terms of:

functioning as a responsible citizen of a democratic society by interpreting the U.S. Constitution and Congressional and State legislation and court rulings,

following protocols in engineering, medicine, law, plumbing, food service, etc., making decisions based on considering expert and other opinions and the

facts, and reading and interpreting deductively without adding any information.

Kilgo research refers to this type of comprehension as “Text-Dependent Reading Comprehension.” “T” Reading


Documents

Kilgo Research Model - HISD Board