TFA Regional OSAT

TFA Regional OSAT2011 Improvements and Introduction to Growth Platform

The Growth Tracker Platform Huge new feature set for the OSAT, based

on the work done for Growth Goals at institute, to track growth alongside and/or independently of mastery.

Platform includes tracker organization tool (Plan tab), data entry tool (Track tab), and growth-based reports (Explore tab).

This release includes 6 growth-based trackers: 6+1 Writing Traits, Dibels, DRA, NWEA RIT, Lexile Tracker, and SRI Tracker

Platform potentially supports many other trackers that fit similar design patterns.

General Design Goals

Standardize all trackers into a handful of useful formats and metrics that users could learn

Support ~80% of existing Excel-based trackers

Provide possibility of aggregation over normalized metrics (such as % Met/Achieved)

Allow for goals to be derived from a number of sources, including externally

Fit within existing OSAT course/user/data structures for easy integration and process management

Integrate with mastery tracking where possible

Common Properties

A number of common properties of trackers have been identified, and preliminary support has been built in to the OSAT for the 2011 regional pilot.

Across all properties, an effort has been made to standardize the data entry and reporting interface to allow for easy switching between trackers.

Property 1: Optional Manual Goals Goals can be entered for any tracker, but

they aren’t required. A tracker can also disallow goals entirely.

They can be entered manually, looked up via a table, or calculated statically.

When a goal is present, it informs the % Met metric, which is available in the Explore Growth Report.

When the goal is NOT present, it informs the Growth average metric, which is always available for a student if there is a pre and post score entered

Optional Goals

Property 2: Automatic AssessmentsGrowth trackers can optionally have

an automatic Pre and Post assessment aligned to all standards (if there are any).

These inform the final Growth metric, as well as the final % Met (achievement) metric in the case that a goal is present. The automatic assessments cannot be edited or deleted by the user.

Can be used for process (submission) reports.

Automatic Assessments

Property 3: Optional Standards In previous OSAT incarnations, all units

(trackers) required standards to operate correctly. With these updates, trackers (for example, the SRI tracker) can accept a growth value that is entered manually and does not require any standards-based (mastery-based) scores to be entered.

In cases where there are no standards, the scores will ONLY inform the Explore Growth report and will NOT inform the Explore Mastery reports.

Ex: With Standards

Ex: Without Standards

Property 4: Unlimited Mid-Assessments Trackers can have any number (zero or higher)

“mid-assessments” (their name can be customized).

These mid-assessments can be provided with the tracker (i.e. as it is cloned in) and/or entered by the user. They can also be locked down so that mid-assessments cannot be edited.

When present, they inform the mid-assessment columns of the Explore Growth report, which focuses on the Most Recent and Max Value mid-assessments seen (in order to limit “crazy columns,” and also informs the “Trend” column which provides a “spark line” graph of all growth so far.

Unlimited Mid-Assessments

Property 5: Mid-assessments Inherit Diagnostic Scores Optionally, any and all mid-assessments

can inherit (in gray) the pre-assessment scores for a given tracker on any standards for which the score has not been overridden (this is the same behavior given at institute, but can optionally be applied or not to any tracker).

Property only really makes sense for those which growth values that are calculated from standards-based scores, such as institute trackers, the 6+1, etc.

Mid Inherits Diagnostic

Property #6: Tracker-Specific “Growth Value” Header TextEach tracker can specify the header

text above the Growth Value (see definition below) column in order to prompt the user for the proper value (e.g. GLE for Grade Level Equivalent)

Tracker-Specific Growth Value Header Text

Property 7: Contextual HelpTracker-specific help text and link:

each tracker can specify a short (1-2 sentence) help text (visible in the top-left of the tracker) along with a link to a URL (preferably hosted on our support forum) with details on how to use the tracker (the text of the link can be specified as well).

Contextual Help #1

Contextual Help #2

Property #8: Locked Standards Each tracker can specific whether or

not standards (the columns to the right) can be edited or not.

When standards are locked, you cannot add, delete or modify the standards for a given tracker within the planning tool and the corresponding Add Standard links are removed

Useful when standards roll up to growth values and must be accurate

Locked Standards

Accessing Trackers

Trackers can be created and made available to CM’s at any time via the “TFA Trackers” section of the OSAT.

Once selected, they are cloned in to the user’s account and immediately made accessible for grade entry within existing courses.

Once scores are entered, they are immediately accessible to Growth Explore reports for CM’s and Staff

Selecting a Tracker: Step 1

Selecting a Tracker: Step 2

Selecting a Tracker: Step 3

Exploring Growth

Similar to the concept of Exploring Mastery, users can Explore Growth for any Growth tracker from the Explore tab.

All Growth trackers are standardized into a single, easy-to-learn reporting format.

While this format does present some limitations, I believe that it will be powerful to standardize metrics for teacher use and eventual aggregation.

The Report Format

Growth Report Ex. 1: 6+1 Traits

Growth Report Ex. 2: Dibels

Growth Report Ex. 3: DRA

Growth Report Ex 4: Lexiles

Growth Report Ex 5: Multiple

Zoom in on Trends

Integration with Mastery

Mastery data (and thus Mastery Explore-based reports) are automatically available for any tracker that is based on standards (like the 6+1 Writing Traits Tracker) or any tracker that simply includes standards columns (like the DRA Tracker).

To compensate for varying maximum score values, the mastery report adjusts bar widths and colors linearly.

Ex: Mastery over Growth Tracker

Ex. 2: Mastery for Entire Class

Other Features & ImprovementsGeneral speed and reliability

overhaul of the system.Login speed is much improved over

last year’s tracker.System is available from

tfaosat.com, instead of the crazy URL!

Easy Mode – still putting some finishing touches on this, and will walk you through soon.

CM Dashboard

Course/Enrollment Management

Manage Students from Track Tab

Improved Import Tools

Improved Export Tools

Help “Dots” Everywhere!

Improved Data Deletion

The Recycle Bin (!!)

Definitions (in my language, we can change the public word used)

Growth Value (or Growth Score?) – this is the primary number for any given tracker, given once per assessment. It can be out of any value (configurable) and can either be calculated (currently AVG or SUM) from standards-based scores or entered manually

% Met (or Achievement?) – this is the normalized percent of goal achieved value, calculated in the same way that it was at Institute. It is only present for growth tracker records that include a goal.

Goal (or Target?) – this is the goal of the total value to be achieved on the post-assessment. The Growth Goal is the amount that the student is expected to GROW from the pre-assessment, Goal is Pre + Growth Goal. Goals are always entered as Goals, not as a Growth Goals (as it stands), though they can potentially be looked up as Growth Goals rather than Goals in cases where they are looked up.

Definitions (page 2) Diagnostic – this is the pre-test. Summative – this is the post-test, one which all

authorative growth values are derived (until post-assessment score exists, things are in mid-assessment mode which means that the actual final % met has not yet been calculated. I think that this is good and will help with process/accountability, as opposed to always taking the most recent?

Assessment – any moment in time during which data is gathered related to the growth on the tracker. Pre and Post are required for the authorative values, and any number of mid-assessments are supported

Mid-Assessment – any assessment along the path towards Growth that is not the authorative Pre or Post assessment. These are “rolled up” in the Explore Growth reports to show the Most Recent, Max Value, and a trend sparkline. Currently, % Met is not calculated for mid-assessment. This could be a possible future improvement, or a possible source of confusion?

Things I’d like to improve (s0 far) Column averages for growth value and goal DRA should show sum instead of average of columns to

the right? Dynamic columns for mis-assessments within Explore

Growth report Entry of growth values for trackers which do not have

standards (i.e. a manual growth score) but with multiple assessments could be laid out horizontally for purposes of data entry and easy reporting (instead of cycling through assessments on the left)

Look up conversion of entered data (from letter code or number) to equivalent numeric value used for growth (i.e. DRA levels->Grade Level Equivalents)

Clicking the Trend column should pop up a window giving details on pre, all mid, and post assessment in comparison to goal (if available)

Things Pending

Mostly done with the staff tool, but holding my overview of it for a day or so while I finish a couple of things up.

Easy Mode hasn’t yet been “written up” or prepared for beta release because of the work necessary to create unique “brand new accounts” for each tester to show the full process of getting started – coming very soon.

Testing is ongoing on our end this week.

Thanks!