JIM MARTIN & AMBER MCCONNELL

UNIVERSITY OF OKLAHOMAZARROW CENTER

The Transition Assessment and Goal Generator

TAGG DEVELOPED WITH A GRANT FROM THE NATIONAL CENTER

FOR SPECIAL EDUCATION RESEARCH AND OU ZARROW

CENTER FUNDS

Thanks to IES and National Center for Special Education Research

Quality Transition Education Produces Greater Outcomes

Transition Education Begins With Transition Assessment

The TAGG Will Assist Student To Answer This Question

What do I need to learn nowTo do the job I want after graduating

from high school?To learn where I want to further my

education after graduating from high school?

Results will facilitate writing I-13 compliant IEP and transition planning discussions

TAGG Overview

PurposeAssess non-academic skills associated with post-

school employment and further educationTo provide lists of student strengths, needs, a

written summary, and annual transition goals matched to common core standards to facilitate writing I-13 compliant IEPs

Designed to AssessSecondary-aged students with mild to moderate

disabilities who plan to be competitively employed and/or enrolled in higher education after graduation, their educators, and parents

Overview Continued

Versions 3 versions: student, family, and professional

FormatOn-line written English

May be printed and taken by hand, but item scores must be entered into website to produce results

In next few months TAGG versions in Spanish, Mandarin, and other languages will be added

Users may choose to listen to audio or watch ASL videos for each instruction and item

Overview Continued

Reading LevelsProfessional 10.4Family 5.7Student 4.8

TAGG Results Profile

The TAGG Will Provide Results to Copy and Paste Into IEP and To Use for Transition Planning Discussions Graphic results profile by constructsWritten summaryListing of strengthsListing of needsAnnual transition goals matched to

Common Core Standards

Development of TAGG Items

TAGG items derived from research studies that identified behaviors of former high school students with disabilities engaged in post-high school employment and/or further education

The research team initially used the research studies to develop10 construct definitions Items developed from constructs

15 or so iterative TAGG versions were created before testing began

Initial Structure: Ten Initial Constructs

Knowledge of strengths and limitations

Actions related to strengths and limitations

Disability awareness

EmploymentGoal setting and

attainment

PersistenceProactive

involvementSelf-advocacySupportsUtilization of

resources

First Effort to Establish TAGG Structure

Users from multiple states completed the initial test-version TAGG 349 high school students with disabilities 271 family members 39 professionals

Applied various factor analyses statisticsWent from 10 constructs to 8Went from 75 items to 34Construct structure confirmed by two more

year-long studies

Professional and Family Constructs After FA

Stayed1.Strengths and

Limitation

2.Disability Awareness

3.Persistence

4.Interacting with Others

5.Goal Setting and Attainment

6.Employment

7.Student Involvement in IEP

8.Support Community

Dropped1.Actions Related to

Strengths and Limitation

2.Utilization of Resources

TAGG-P: (c2=1043.62, df=499, RMSEA=.058, CFI=.92, TLI=.91,

RMSR=.0597) TAGG-F: (c2=862.74, df=499,

RMSEA=.057, CFI=.91, TLI=.90, RMSR=.058)

Student Version Constructs After FA

After FA Constructs

1.Strengths and Limitations & Support Community

2.Disability Awareness

3.Persistence

4.Student Involvement in IEP

5.Interacting with Others

6.Goal Setting and Attainment

7.Employment

Dropped Constructs

1.Actions Related to Strengths and Limitation

2.Utilization of Resources

Combined Constructs

3.Strengths and Limitations

4.Support Community

TAGG-S: (c2=819.00, df=505, RMSEA=.047, CFI=.89, TLI=.88, RMSR=.064)

Numbers of Participants to Validate TAGG Across Three Years

Professionals Families Students

Total 139 847 1, 537

EIGHT CONSTRUCTS DEFINED: SEE TAGG FACT SHEET

Final TAGG Constructs

Strengths and Limitations

Express personal strengthsDescribe personal limitationsExplain academic situations where assistance

is neededDescribe academic situations where success

is experienced

Disability Awareness

Report accurate information regarding one’s own disability

Express types of accommodations needed for success

View a disability as only one aspect of lifeExplain the special education services that

one receives to others

Persistence

Understand the importance of putting forth continued effort in school

Work toward a goal until it is accomplishedUtilize multiple strategies to stay on taskContinue to work toward a goal after facing

adversity

Interacting with Others

Maintain at least one good friendSuccessfully participate in small groups to

complete projectsSuccessfully participate in community

organizations such as sport clubs and social groups

Successfully interact with peers, teachers, and other adults

Goal Setting and Attainment

Participate in the systematic learning of goal setting and attainment

Understand of the importance of setting and striving for goals

Set post-school goals that match skills and interests

Attain at least one transition goal

Employment

Express the desire for a job that matches career interests

Demonstrate job readiness skills, such as being on time, completing work as assigned, and working cooperatively

Successfully participate in a career technology or job-training program

Obtain a paid job

Student Involvement in the IEP

Identify effective and ineffective accommodations

Request additional accommodations when encountered with one that is ineffective

Discuss post-school goals with the IEP teamActively lead one’s IEP meeting

Support Community

Recognize the difference between individuals who provide a positive source of support from those who do not

Identify situations when positive support people are needed

Use help from positive support people only when needed and necessary

Maintain a support network by showing appreciation or reciprocity

Summary of Psychometric Findings

See TAGG Technical Manual Available at OU’s Zarrow Center TAGG web site for more information and updates as more validity evidence becomes available.

Internal Reliability

Generally a score between .7 and .8 is considered “good”Each TAGG version has great overall

internal consistency and satisfactory subscale consistency (ranging from α =. 89 to α =. 95)

TAGG Internal Reliability Measures are . . .

Test-Retest Reliability

Scores of .7 or higher represent good or satisfactory test-retest reliability14 weeks after the first TAGG was

completed, same users completed the TAGG again.

A large correlation was found between the first and the second administration .80 for professional TAGG .70 for family TAGG .70 for student TAGG

The TAGG has what type of test-retest reliability . . . . .

Measurement Invariance

The three TAGG versions are appropriate for students regardless of: Students’ time in general education classes Number of transition education classes

completed High School grade level Disability category

This means TAGG results do not need to be adjusted for the

above factors TAGG users believe TAGG items are of equal

value

Fairness Validity Evidence: Gender

Do differences exist by gender? No overall difference by gender on

TAGG-P, TAGG- F, or TAGG-S Some construct differences exist

across all versions. For Instance: On TAGG-S, females rated themselves higher on student involvement than males

Fairness Validity Evidence: Disability Category

Do differences exist by disability category?No overall significance difference by disability

category Due to sample size, only students with Emotional

Disturbance, Intellectual Disabilities, Autism, Specific Learning Disabilities, and Other Health Impairments were included in this analysis.

In general, students with ED and Autism scored lower on the three TAGG versions than students who had SLD or OHI disability categories. In general, students who had SLD scored the

highest on the vast majority of constructs on all three TAGG versions.

Free/reduced lunch eligibility No significant differences for construct scores on

TAGG-P or TAGG-S. Only small differences for TAGG-F scores.

Family employment No significant differences for construct scores

Family education Significant differences due to family education

were found on the TAGG-F scores. Almost all of the differences occurred between

students whose family member was at the highest level of education (Master’s/Ph.D./other professional degree) and those at the lowest level of education (less than HS).

Fairness Validity Evidence: SES Effect

How Close Are Students, Professionals, and Family TAGG Scores?

How closely do the different TAGG versions assess the same student?Medium correlations across Parent,

Educator, and Student versions when assessing the same student.

This is considered excellent for this type of assessment

Concurrent Validity Studies

How Close Do the TAGG and AIR Self-Determination Assessment Match?Medium Correlation

This implies the TAGG addresses some self-determination skills and assesses other skills, too.What we would want in a concurrent

validity study.

ITEM RESPONSE THEORY

Scoring Using IRT

Advantages to Using IRT

Advantages of IRT include The ability to scale different item

types Provides a common metric for scales

with different number of items Weights items differentially by their

validity for assessing the construct of interest

We Used a Four-Step IRT Algorithm

1. Placed each scale onto a common score metric

2. Projected item characteristics (e.g. item difficulty) onto the scale score metric

3. Conducted a within-student comparison of scale scores across constructs to determine relative strengths and weaknesses

4. Conducted a within-construct comparison of a student’s scale score to item responses (e.g. difficulty) to generate appropriate goals for identified weaknesses

Stanine Scores

The stanine transformation places scale scores into one of nine categories.

Each of the nine categories has a width corresponding to a half of a standard deviation on the normal curve, with the mean lying at the center of the stanine scores (i.e., score of 5)

Scale Score to Stanine Conversion for Reporting Purposes

Students placing in the green area are in the average range

Overall Score

The overall scale score is a weighted combination of all items

Future TAGG Development

More To Come

Now Conducting a second round of follow-up studies of former high school students with disabilities who took the TAGG

We will determine the relations between their high school TAGG profile and post-school outcomesThis will predict constructs associated with

positive employment or further education outcomes

Results will be included into the TAGG profile to prioritize importance to skills that once learned will be most likely associated with positive post-school outcomes

Final TAGG Details

Minimal Cost $3 per set (Professional, Student, Family versions) Pay with Purchase Order or Credit Card All funds handled by the OU Office of Technology

DevelopmentTAGG profiles saved for 7 yearsData kept on OU high-speed secure cloud serversPurchased credits may be transferred to other

registered TAGG usersUnused credits refunded for one-year after purchase

TAGG Location

1. The OU Zarrow Center’s Web Pagehttp://zarrowcenter.ou.edu

2. The TAGG Section of the ZC Web Page

https://tagg.ou.edu/tagg

THE ON-LINE TAGG AND RESULT PROFILE

Sample TAGG Screen Shots

Disability Awareness Profile

Combined Score Profile

Greatest and Relative Strengths

Areas of Greatest and Relative Need

Summary Statement for IEP

Chad Bailey’s skills were assessed using the TAGG, a norm-referenced assessment with research-based items known to be associated with post-school employment and education. Compared to similar students, Chad’s scores are average. Results indicate greatest strengths are in the areas of Goal Setting and Attainment. Chad’s relative strengths include Disability Awareness and Student Involvement in the IEP. Greatest needs are in the area of Strengths and Limitations, with Employment being a relative need.

Suggested Annual Transition Goals

To prepare for success in employment, the student will write an essay describing three situations where the student used his or her strengths with 90% grammar and context accuracy by the end of the essay writing unit.

Contact Us At

Dr. Jim Martinjemartin@ou.edu

Dr. Amber McConnellambermcc@ou.edu

(405) 325-8951