Training Evaluation

TAB Mtg Update: 3/21/14

Big Picture

• Training Evaluation Framework (in process; categories: Internal staff, relationships with agencies, eval curricula, trainers, trainees, and eventually, outcomes as a goal)

• Plan is to revise:• PSS forms to better reflect participants’

learning and potential for applying on the job• Trainer Eval Rubric as part of improving trainer

eval process

Part of the Framework: Common Core Evaluation

BAA• Demographics (from Nov 2013 semi-annual report, N=24):• Education Level: ~65% had MSWs as highest level• Title IV-E: ~66% participated• Ethnicity: Between 25-30% Latino/Hispanic, about 20% White,

about 20% African American, 10% MultiRacial, 0% API• Role: About 90% were county CWWs, other 10% indicated

working for nonprofit or “other” category• Have spvr/mentor: About 95%; of these, about 75% had

discussed training needs with their supe/mentor

Part of the Framework: Common Core Testing

BAA• Testing Knowledge (July 2012-June 2013):• CYD (Pre-post): N=49; Statistically significant gains from pre

to post. No significant demographic differences in testing this content.

• CPCM (Pre-post, Temp Discont…): N=45; Statistically significant gains from pre to post. Significant demographic differences in testing this content: ethnicity/race and gender.

• PP (Pre-post): N=42; Statistically significant gains from pre to post. No significant demographic differences in testing this content.

Part of the Framework: Common Core Testing

BAA• Testing Skill in Classroom (late Dec 2012-early 2013):• CMI1: N=18; In making a decision of whether or not physical abuse

occurred in test scenarios, 100% of BAA trainees passed using criterion of 3 out of 4 correct decisions. Participants were more likely to make two correct decisions on non-abuse than abuse scenarios (94.4% vs. 88.9%). However, N is small and this difference represents one response.

• CMI2: N=29; In making a decision of whether or not sexual abuse occurred in test scenarios, 89.7% of BAA trainees passed using criterion of 3 out of 4 correct decisions. Participants were more likely to make two correct decisions on non-abuse than abuse scenarios (89.3% vs. 79.3%). Appears to be due in part to a higher %age of missing responses on Warner (last of 4 scenarios tested), which is an abuse scenario.

Part of the Framework: Pilot Skill-Based Eval

• Art of Coaching Institute: March 24 & 25, 2014, plus Day 3 April 29, 2014

County-Based Training Evaluation

• San Mateo County• Visitation• TDM