Embed Size (px)
Citation preview
Chapter 8Developing Written Tests and
Surveys
PhysicalFitnessKnowledge
Test Planning
Types
Mastery
Achievement
Table of Specifications
Content Objectiveshistory, values, equipment, etiquette,safety, rules, strategy, techniquesof play
Educational Objectivesknowledge, comprehension,application, analysis, synthesis,evaluation
Table 8.1 Table of Specifications for a 60 Item Written Test on
Badminton
Educational objectives
Con
ten
t ob
ject
ives
Test Characteristics
How to measure
When to test
How many questions?
What format should be used?
What types of questions should be used?
Essay Questions
When to use
Weaknesses
Scoring
Objectivity
Construction recommendations
Scoring recommendations
Semi-Objective Questions
When to use
Weaknesses
Construction recommendations
Scoring recommendations
Objective Questions
True/False
MatchingMultiple Choice
Construction recommendationsScoring recommendations
CompetenciesCompetency A
correctB
correct
Ccorrect
D correct
Total
Motor development and motor leaning – Competency 1 23 16 13 9 61
Fitness and fitness development/maintenance – Competency 2 21 14 13 14 62
Lifetime individual, dual, and group activities – Competency 3 6 7 10 12 35
Fitness-related health, nutrition, and safety – Competency 4 16 13 15 10 54
Affective development – Competency 5 8 7 6 5 26
Social development – Competency 6 4 2 6 5 17
Cognitive development – Competency 7 8 8 8 6 30
Physical education program – Competency 8 5 6 9 10 30
Learner assessment – Competency 9 4 13 8 18 43
Managing physical education classes – Competency 10 11 12 9 13 45
Legal, ethical, medical, and safety concerns – Competency 11 10 9 14 18 51
Total correct response 116 107 111 120 454
% Correct response by foil 26% 24% 24% 26% 100%
Figure 8.1 The difference Between Extrinsic and Intrinsic Ambiguity
Administering the Written Test
Before the test
During the test
After the test
Characteristics of Good Test Items
Leave little to "chance"ReliableRelevantValidAverage difficultyDiscriminate
Gotten correct by more knowledgeable studentsMissed by less knowledgeable students
Time consuming to write
Some Test-Taking SkillsReview these important skills on pages 171-172
Preparing for the testGetting started and taking the testAfter taking the test
Quality of the Test
Reliability and validity
Overall test qualityIndividual item quality
Alpha Coefficient
Recall the alpha coefficient from chapter 6
total
trials
S
S*
K
K2
2
11
KR20
Notice the Similarities
total
trials
S
S*
K
K2
2
11
KR20 =
alpha =
These two ARE the same.Use alpha with continuous data.Use KR20 with dichotomous data.
KR21
Notice the Similarities
total
trials
S
S*
K
K2
2
11
KR21 =
KR20 =
alpha =
Notice the Similarities
total
trials
S
S*
K
K2
2
11
KR20 =
alpha =
These two ARE the same.Use alpha with continuous data.Use KR20 with dichotomous data.
Notice the Similarities
KR21 =
KR20 =
KR21 is a conservative estimate ofthe reliability. KR20 will ALWAYS begreater than or equal to KR21 but KR21 is much easier to calculate.
Item Analysis
Used to determine quality of individual test items
Item difficultyPercent answering correctly
Item discriminationHow well the item "functions“Also how “valid” the item is based on the total test score criterion
One Way to Organize Data for Item Analysis
1) Score the tests2) Arrange answer sheets from high to low3) Separate answer sheet into three subgroups
a) upper 27%b) middle 46%c) lower 27%
4) Count and record responses per foil in upper group5) Count and record responses per foil in lower group6) See figure 10.3
Item Difficulty
100*nn
cc
LU
LUDifficulty
Item Discrimination
100*U
LUationminDiscri
n
cc
Item Difficulty
100*nn
cc
LU
LUDifficulty
Item Discrimination
100*U
LUationminDiscri
n
cc
Figure 8.3—Item #5
Responses A B C D* E Omit Diff. Net D
Upper 27% = 50 28 2 1 19 0
36% 4%Lower 27% = 50 24 8 1 17 0
Figure 8.3—Item #25
Responses A B C D* E Omit Diff. Net D
Upper 27% = 300 69 10 5 216 0
53% 37%Lower 27% = 300 89 52 54 104 1
Figure 8.4 The Relationship Between Item Discrimination and Difficulty
Maximum, Minimum, and Desired Item Difficulty and Item Discrimination
Difficulty Discrimination
Maximum 100 100
Minimum 0 -100
Desired 50 100
Sources of Written Tests
Professionally constructed tests
Textbooks
Periodicals, theses, and dissertations
McGee & Farrow
Item Analysis Practice27% = 50 people
Item # Upper Lower Difficulty Discrimination
1 32 21
2 18 12
3 40 21
4 50 40
5 36 42
6 25 25
7 15 35
8 35 15
Item Analysis Practice27% = 50 people
Item # Upper Lower Difficulty Discrimination
1 32 21 53% 22%
2 18 12 30% 12%
3 40 21 61% 38%
4 50 40 90% 20%
5 36 42 78% -12%
6 25 25 50% 0%
7 15 35 50% -40%
8 35 15 50% 40%
Each Item Was Completed By the Same Number of People
Complete the chart below by fillingin the empty boxes
Item Uppercorrect
Lowercorrect
Difficulty Discrimination
1 10 100%
2 5 50%
3 5 65%
Each Item Was Completed By the Same Number of People
Complete the chart below by fillingin the empty boxes
Item Uppercorrect
Lowercorrect
Difficulty Discrimination
1 10 10 100% 0%
2 10 5 75% 50%
3 5 8 65% -30%
Questionnaires
Determine the objectivesDelimit the sampleConstruct the questionnaireConduct a pilot studyWrite a cover letterSend the questionnaireFollow-up with non-respondentsAnalyze the results and prepare the report
Constructing Open-Ended Questions
AdvantagesAllow for creative answersAllow for respondent to detail answersCan be used when possible categories are largeProbably better when complex questions are involved
DisadvantagesAnalysis is difficult because of non-standard responsesRequire more respondent time to completeCan be ambiguousCan result in irrelevant data
Constructing Closed-Ended Questions
AdvantagesEasy to codeResult in standard responsesUsually less ambiguousEase of response
DisadvantagesFrustration if correct category is not presentRespondent may chose inappropriate categoryMay require many categories to get ALL responsesSubject to possible recording errors
Factors Affecting the Questionnaire Response
Cover letterBe brief and informative
Ease of returnYou DO want it back!
Neatness and lengthBe professional and brief
InducementsMoney and flattery
Timing and deadlinesTime of year and sufficient time to complete
Follow-upAt least once
The BIG Issues in Questionnaire Development
ReliabilityConsistency of measurement
ValidityTruthfulness of response
Representativeness of the sampleTo whom can you generalize?
