1

Implementing Language Access Plans: What Works? What Counts?

Evaluation Results

Azadeh KhaliliOctober 5, 2011

2

Table of Contents

1. Evaluation Overview 2. Quick Snapshot of Participant Ratings3. Results from the Morning Session4. Results from the Afternoon Breakout

Sessions5. Considerations and Implications for Next

Steps

3

Evaluation Overview• AECF evaluation forms measured extent to which convening achieved 5

desired results:1. Understand how language access contributes to achieving agency’s mission2. Identify strategies and action steps to move language access forward3. Connect with colleagues from other agencies to learn and share ideas4. Use knowledge, skills, and networks gained today to strengthen my language access

work5. Know who to contact for TA going forward

• Also asked: 1. Effectiveness of morning speakers and roundtable discussions2. What participants found useful3. What would have been more useful4. Other comments

• Separate evaluations of morning session (116 respondents) and afternoon breakouts (77 respondents)

4

Snapshot of Participant Ratings:The Convening Was Productive Use of Time

Concurrent Afternoon Breakout Session B

Concurrent Afternoon Breakout Session A

Morning Session

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

5

Results from the Morning Session

6

Results of Morning Session

Understanding of LA & Agency Mission

Identified Strategies and Action Steps

Know Who to Contact for TA

Roundtable Discussions Enhanced Thinking

The Speakers Enhanced My Thinking

Will Use Knowledge, Skills, and Networks

Connected With Colleagues

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

7

What was most useful about the morning?

Note: Relative size of words reflects how often they were mentioned in participant responses.

8

What would have made the morning more useful?

Note: Relative size of words reflects how often they were mentioned in participant responses.

9

Additional comments from the morning session…

Note: Relative size of words reflects how often they were mentioned in participant responses.

10

Results from Afternoon Breakout Sessions

11

Aggregate Results of Afternoon Breakout Sessions

Learned New Skills to Move LA Forward

Connected with Colleagues

Helped Refine Action Plan

Productive Use of Time

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Helped Refine Action Plan

Connected with Colleagues

Learned New Skills to Move LA Forward

Productive Use of Time

Session A (n=76)

Session B (n=64)

12

Results of Each Breakout Session

Monitoring and Evaluating Plans B (n=24)

Interpretation: Outside Vendors A (n=10)

Monitoring and Evaluating Plans A (n=19)

Best Practices in Contracting B (n=10)

Interpretation: Testing & Training A (n=16)

Interpretation: Testing & Training B (n=18)

Best Practices in Contracting A (n=10)

Interpretation: Outside Vendors B (n=9)

Meeting Translation Needs A (n=20)

Meeting Translation Needs B (n=3)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

* Results are the aggregate responses for the 3 desired results: learned new skills; helped refine action plan; and connected with colleagues.

13

Results of Concurrent Breakout #1:Meeting Your Agency’s Translation Needs

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

Session A (n=20)

Session B (n=3)

14

Results of Concurrent Breakout #2:Interpretation Testing and Training

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

Session A (n=16)

Session B (n=18)

15

Results of Concurrent Breakout #3:Interpretation Using Outside Vendors

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

Session A (n=10)

Session B (n=9)

16

Results of Concurrent Breakout #4:Best Practices in Contracting

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

Session A (n=10)

Session B (n=10)

17

Results of Concurrent Breakout #5:Monitoring and Evaluation

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

strongly agree agree neutraldisagree strongly disagree

Productive Use of Time

Connected with Colleagues

Helped Refine Action Plan

Learned New Skills to Move LA Forward

Session A (n=19)

Session B (n=24)

18

Comments from afternoon breakout sessions

Note: Relative size of words reflects how often they were mentioned in participant responses.

19

Considerations and Implications for Next Steps• Design team set very ambitious goals for a single day

with a large, heterogeneous group• High demand among participants for continued

engagement:1. Examples of DOJ-approved LAPs2. Peer learning from federal agencies linked to specific

elements/challenges of their own LAPs3. More intentional networking (eg, all law-enforcement, mix-max from

multiple agencies) with more choice of content4. Responsive to concerns re: logistics (eg, room, handouts, etc)

• Opportunities to leverage greater role for federal LA task force members