PowerPoint Presentation
Recontextualizing Blooms Taxonomy: Quantitative Measures in
Formative Curriculum Assessments and Program Evaluationspresented
by Anthony ClemonsStudent, Master of EducationTeachers College,
Columbia University
Aaron Smith, Ph.D.Lecturer, Dept. of MathematicsUniversity of
Central FloridaPresented March 12, 201628th International
Conference on Technology in Collegiate Mathematics
AGENDAPurpose of Our ResearchSupporting LiteratureModified
Delphi Assessment ModelConclusionReferences
RESEARCH PURPOSEWe wanted to find a way to:Formatively evaluate
the quality of an academic course through its curriculum,Using
cognitive achievement as the measurement vehicle,While quantifying
our results
RESEARCH PURPOSEWhat would this evaluation require?A reasonable
surrogate (Horner et al, 2005)That assimilates assessment dataTo
provide a comprehensive evaluation of both a course and its
curriculum
RESEARCH PURPOSEHow did we arrive at a finding a reasonable
surrogate?
SUPPORTING LITERATUREThis evaluation is possible using,
SUPPORTING LITERATUREWhat is Blooms Taxonomy?A hierarchal scheme
for identifying a continuum of six specific cognitive demands in
learning, ranging from simple (knowledge, comprehension, and
application) to more complex (analysis, synthesis, and evaluation)
(Bloom et al.,1956; Armbruster & Ostertag, 1989).
SUPPORTING LITERATURE
Higher-Order Thinking Skills (HOTS)?The taxonomys top three
cognitive domainsRequires students to grasp a deep understanding of
what they are learning and be more critical and creative instead of
merely recalling information
SUPPORTING LITERATURE
Lower-Order Thinking Skills (LOTS)?The taxonomys lowest three
cognitive domains Emphasis on instruction versus facilitation
Involves repeated responses to recurrent stimuli for the purpose of
adjusting cognitive tendencies so individual knowledge might be
thwarted to action (Dewey, 1916).
SUPPORTING LITERATUREWhy Use Blooms Taxonomy?Research concedes
it as a valid framework for quantitatively measuring the
consistency and quality of HOTS and LOTS, it can also be an input
for categorizing and comparing cognitive skillsIt defines the
academic quality of courses in terms of the level of academic
demands or rigor expected of the students (Nordvall & Braxton,
1996, p. 486)
10
MODIFIED DELPHI ASSESSMENT MODELHow do we use Blooms as a
formative assessment tool?NOTE: Nature of formative curriculum
assessments is that they are conducted prior to curriculum being
implemented as an instructional instrument (i.e. before the
coursework is taught in class).By modifying Rowe and Whites (1999)
characterization of the classical Delphi method, this need can be
met through redefining the methods key features via an iterative
Delphi-based model.
11
Tri-Phased Delphi Evaluation Process for Programmatic Curriculum
Assessments
MODIFIED DELPHI ASSESSMENT MODEL
12
MODIFIED DELPHI ASSESSMENT MODELDelphi 1: Coding Learning
Objectives Learning objectives are assessed for validity and coded
according to the taxonomical domain with which each objective is
aligned as follows:
Taxonomical LevelTaxonomical
CodeKnowledge1Comprehension2Application3Analysis4Synthesis5Evaluation6
13
MODIFIED DELPHI ASSESSMENT MODELDelphi 1: Coding Learning
Objectives Using table below, synchronizing the taxonomical domain
coding for a courses learning objectives can be facilitated:
14
MODIFIED DELPHI ASSESSMENT MODELQuantitative ModelDelphi 2:
Quantitative Curriculum Assessments Statistical
MeasurePurposeArithmetic MeanEstimate of the sampled populations
central tendencyWeighted Distributive MeanEnsures the percentage of
time spent in a class session on each learning objective is
accurately captured using a weighted percentageVarianceProvides a
quantitative basis for consideration as to whether to narrow or
widen the variance in how curriculum and learning objectives foster
learning
15
MODIFIED DELPHI ASSESSMENT MODELQuantitative ModelDelphi 2:
Quantitative Curriculum Assessments Statistical
MeasurePurposeStandard DeviationProvides a quantitative basis for
consideration as to whether to narrow or widen the variance in how
curriculum and learning objectives foster learning. One-to-one
relationship with the variance. Used according to
preferenceMedianAllows for researchers and practitioners to
determine the quality of distribution through knowing what the
measure for central tendency is for all of the learning outcomes
within a course.SkewnessMeasures whether there are observable
extreme values following a common direction along either the first
and fourth quartiles, incurring an asymmetric form to the central
tendency along that direction.
16
MODIFIED DELPHI ASSESSMENT MODELWhat are we looking for?
Cognitive Progression and an emphasis on HOTS!
17
MODIFIED DELPHI ASSESSMENT MODELQuantitative ModelDelphi 3:
Content Analysis, Content Revisions, Learning Objectives Recoded,
and a Quantitative ReassessmentLessons endure a another review for
learning objectives and lesson content alignmentSome lessons must
be recomposed by: Adjusting the lesson content to meet the actual
cognitive level of the objective, Adjusting the lesson learning
objective(s) to the taxonomically sufficient level that meets the
actual cognitive level of the content, or Removing the objective
and content from consideration entirely
18
MODIFIED DELPHI ASSESSMENT MODELDelphi 3: Content Analysis,
Content Revisions, Learning Objectives Recoded, and a Quantitative
ReassessmentOnce all appropriate revisions successfully instituted,
the actions within Delphis 1 and 2 must be re-engaged This ensures
the most accurate data is accumulated giving all stakeholders a
decisively precise snapshot of the state of a courses
curriculum
Achieved through using this model and Blooms taxonomy as lesson
assessment and course evaluation vehicles=
19
CONCLUSIONThis model allows institutions to have a structured,
measurable body of curriculum for its courses, while leaving the
content creativity to the instructor and department
This model provides a quantitative assessment methodology for
accreditation agencies to use in determining the level of critical
thinking being achieved in every course, department, and
institutionThis can provide consumer data to prospective students
who want to effectively achieve HOTS in their coursework
We acknowledge the need for additional research, using various
data sets, but the existence of course learning outcomes and lesson
learning objectives must exist to do so
20
REFERENCESBloom, B., Engleheart, M., Furst, E., Hill, W., &
Krathwohl, D. (1956). Taxonomy of Educational outcomes: The
Classification of Educational Goals. Handbook I: Cognitive domain.
New York: Longman.
Dewey, J. (1916). Democracy and education: An introduction to
the philosophy of education. New York: Macmillan
Horner, R., Zavodska, A., & Rushing, J. (2005). How
challenging? Using Bloom's Taxonomy to assess learning outcomes in
a degree completion program. Journal of College Teaching &
Learning, 2(3), 47-51. http://dx.doi.org/10.19030/tlc.v2i3.1785
Rowe, G. & Wright, G. (1999). The Delphi technique as a
forecasting tool: Issues and analysis. International Journal of
Forecasting, 15(4), 353-375
