Measuring AcademicAchievement in Career-

Technical Education

Presented by:

Sandra Pritz, Senior Consultant

National Occupational Competency Testing Institute

Rationale for Standardized Assessments—

technical and academic

• External objective data of student achievement

• Comparative data for schools, districts, states, and across states

• Perkins and NCLB core measures

• May be used by employers and postsecondary institutions

Value Added To Change What Is Taught and How to Teach

• “The administrator sits down with teachers and goes over the results to determine changes that may be needed in the curriculum.” (Teacher Sandra Himes, PA)

• “I love the data and detail to be able to improve instructional strategies.The results are charted, show technical gains. If the group shows highs and lows, you can work on the lows. Teachers see where they missed and work on different approaches.” (Administrator Ed Haynor, MI)

Why should career-technical programs assess academic

skills?• State/national academic standards demand

attention.• Career-tech is accountable for academic gains.• High school students must pass academic tests

for graduation.• Academics in context tend to be student-friendly • Workplace demands for technical literacy are

increasing.

Technical Literacy: integrated academics and technical content

at basic, proficient,and advanced levels

• Read, understand, and communicate in the language of a career field

• Understand technical concepts and principles

• Use academic knowledge and skills to solve problems

• Use basic technology

Academic Skills Measured in Context

Academic Skills

+ Technical Skills

+ Employability Skills

Comprehensive Assessment

AcademicSkills

EmployabilitySkills

TechnicalSkills

How Should Career-Technical Programs Assess Academic

Skills?

• In the context of the students’ career-technical program

• In alignment with state and national academic standards

• In alignment with post-secondary program requirements

• Embedded In their career-technical tests, such as end-of-course or program

Testing Academic Skills in Context

• Math, science, and language arts are required to complete most occupational items and performance tasks.

• NOCTI tests occupational competencies, but also tests embedded academics.

• Embedded academics are correlated to national standards (NCTE, NCTM,NSES).

• Student scores can be isolated by discipline.

Academics Identified• Written tests contain embedded

academics that match standards in all three disciplines.

• Performance tests contain a higher percentage of embedded academics than written tests.

• Students tend to score lower on the written than on the performance test, and the items most often missed involve technical literacy.

Ways NOCTI can help

• By providing performance as well as written tests, NOCTI can help assess academics at higher levels.

• NOCTI can respond to the need to reinforce academic instruction through assessment, can customize tests and test reports.

• NOCTI can provide states the service of cross-correlating its tests to their state standards.

Agricultural Mechanics Example from the Agricultural Electrical Power & Processing Duty:

The completed path through which electricity flows is called the

A. fuse B. circuit C.circuit breaker D. ground wire

- NCTE Language Arts: Apply strategies to comprehend, interpret, evaluate, and appreciate texts. Use experience, interactions with readers/writers, meaning of words, word identification , and features, e.g., sound-letter correspondence, sentence structure, context, graphics.

- National Science Education (NSES) Standards:Transfer of energy

Workplace Readiness Example: You earn $8/hour and work 40 hours/week. You receive time and a half for overtime and you worked five overtime hours last week. What was your gross pay last week?

A.$350 B.$360 C.$380 D.$390

- NCTM Math: Select/use various types of reasoning and methods of proofs; Apply/adapt a variety of appropriate strategies to solve problems.

- South Dakota Math, grades 9-12:Use numbers in a variety of equivalent forms to solve problems; Use properties of the number system to solve problems.

- South Dakota Language Arts, grades 9-12 :Use various reading cues/strategies to comprehend text

PA Assessments…

• Math Standards Alignment– 8th, 11th, and National

Standards– Standardized paper

and pencil doesn’t fit everyone—use written and performance.

– Students show improvement using CTE methodologies.

• Criterion Referenced Scores– Advanced, Proficient,

Basic, Below Basic– Aligns with State

Assessment– Top 20 Tests & 200

Pennsylvania Employers

Other IA Assessments in demonstration projects…

• South Dakota – Workplace Readiness

and Carpentry– Language arts, math ,

and science– State and national

standards– 8th and 9-12th grade

standards– Written and

performance

• Oklahoma– 27 different tests on

11 campuses– 14 math and 10

science selected– State standards with

national as available– 9-12th grade

standards– Written at all sites,

performance at sites per their selection

What Do I Receive from NOCTI?

• Score Reports– Coordinator’s Report– Individual Student Reports– Customized Reporting Available—

e.g. embedded academics

• Statistical Information– Group Data– Site Data– State Data– National Data

• Student Certificates

• Personalized Contact Assistance

All results, including integrated

academics, are broken down into

duty, category and total scores.

Note:

Job Ready Welding Written Assessment

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

The

rmal

Cut

ting

Pro

cess

es

Hea

tT

reat

/Met

allu

rgy/

Dis

tort

ion

Con

trol

Wel

ding

Ter

ms/

Blu

eprin

tR

eadi

ng

Wel

ding

Insp

ectio

n

Wel

ding

Con

sum

able

s

Saf

ety

Pow

er S

ourc

e P

rinci

ples

Oxy

fuel

Wel

ding

&B

razi

ng

Shi

elde

d M

etal

Arc

Wel

ding

Gas

Met

al A

rc &

Flu

xC

ored

Arc

Wel

ding

Gas

Tun

gste

n A

rcW

eldi

ng

Mea

surin

g In

stru

men

ts

Wel

ding

Qua

lific

atio

ns &

Pro

cedu

res

Wel

ding

Cos

ts

Tot

al

Pre-Test Scores Post-Test Scores

Pre and Post Test Gain Analysis

Program Improvement

• Focus on industry standards – Plan program and curriculum changes– Assign “just-in-time” lab projects

• Expand emphasis on technical literacy at proficient levels– Reading career -technical material– Interpreting technical information– Calculating in technical problems

Student Achievement Improvement

• Identify gaps—group and individual-- and target instruction to them

• Heighten awareness of responsibility for learning per strengths/weaknesses

• Focus on academic and employability skills

Workplace Readiness Test

• NOCTI Workplace Readiness Test measuring employability skills is now up-scaled.

• A large and stellar Subject Matter Expert Team has culled and rated the critical competencies.

• Career Cluster Foundation Skills are assessed.

Workplace Readiness Assessment

Communications Systems

Information Technology Applications

Problem Solving and Critical Thinking

Safety, Health and Environmental

Leadership, Management and Teamwork

Ethics and/or Legal Responsibilities

Employability and/or Career Development

Workplace Readiness Assessment

82 item multiple-choice assessment; parallel forms with .94 internal consistency/reliability

41-item version to accommodate single-period schedules; no sub-scores; .96 reliability

Career Skills middle school 41-item version

Online delivery format available for all versions

To discuss your interests,visit

www.nocti.org, call 1-800-334-6283, or

write spritz@nocti.org

Thank You