PROGRAM PROGRAM MANAGEMENT AND MANAGEMENT AND ACCOUNTABILITYACCOUNTABILITY

Requirements and Considerations Requirements and Considerations Across Discretionary GranteesAcross Discretionary Grantees

Day One Plenary Session—Presentation Day One Plenary Session—Presentation

Accountability and Grants Management:Connecting the Dots

U.S. Department of Labor, ETA, Region 4 U.S. Department of Labor, ETA, Region 4 Discretionary Grantee Training ConferenceDiscretionary Grantee Training Conference

March 1-2, 2011March 1-2, 2011 11

ObjectivesObjectives

Establish a foundation for program management and related accountability requirements, irrespective of the source of the requirements or the employment and training program funded

Identify gaps in current plans, policies or operations and discuss ways to address

2

Approach (1)Approach (1)

3

Approach (2)Approach (2)

4

ConsiderationsConsiderationsAs You Develop Your Accountability Framework…As You Develop Your Accountability Framework…

Definitions—Don’t believe everything you think (a/k/a Webster’s dictionary doesn’t always apply)

Make sure you’re following all required parameters

You need systems to capture, maintain, and report all the information Policies, Procedures, Protocols

When there are partners, sub-awards, contractors . . .

5

Who you’re serving…Who you’re serving…General Grantee Requirements*General Grantee Requirements*

All grantees need policies and procedures to ensure compliance with all federal parameters governing who can participate in the program

6

*Although this section applies to grantees serving participants, policies and procedures to support operations is critical across

grantees and organizations.

Who you’re serving…Who you’re serving…Federal ParametersFederal Parameters

Citizenship Selective Service Veteran Status Ethnicity, Race, Disability Status SSN Characteristics (grant-specific although

many are common such as socioeconomic data, age, employment status at entry, level of education)

7

Who you’re serving…Who you’re serving…Data Sources and DocumentationData Sources and Documentation

Sometimes have same meaning / not always

Sometimes specified, or minimum requirements provided with details left up to grantees

Some grantees assume “documentation requirements” that don’t exist

What does your _________ say? What does your _________ say?

8

Who you’re serving…Who you’re serving…Citizenship RequirementsCitizenship Requirements

“Participation in programs and activities … shall be available to citizens and nationals of the United States, lawfully admitted permanent resident aliens, refugees, asylees, and parolees, and other immigrants authorized … to work in the United States.” WIA, Sec. 188(a)(5)

Documentation could include _________________ or ___________________ (formerly “________ card”)

Do you currently capture this information? How?

9

Who you’re serving…Who you’re serving…Selective Service RequirementsSelective Service Requirements

“The Secretary shall ensure that each individual participating in any program or activity… has not violated section 3 of the Military Selective Service Act (50 U.S.C. App. 453) by not presenting and submitting to registration as required…” WIA, Sec. 189(h)

Registration required for all males born on or after 1/1/60 within 30 days of 18th birthday with late registrations accepted until age 26

10

Who you’re serving…Who you’re serving…Selective Service Requirements (2)Selective Service Requirements (2)

From www.sss.gov –

“Some men may have failed to register during the time they were eligible to do so and may now find they are ineligible for certain benefits.”

“The Workforce Investment Act (WIA)…. is only open to those men who register with Selective Service.”

Documentation could include __________________ ___________________ noting the person registered

11

See Handout #1

Who you’re serving…Who you’re serving…Veterans’ Priority of Service RequirementsVeterans’ Priority of Service Requirements

“Grantees are required to provide priority of services for veterans and eligible spouses pursuant to 20 CFR part 1010, the regulations implementing priority of service for veterans and eligible spouses in [DOL] job training programs under the Jobs for Veterans Act…” Text from an SGA (ETA policies also referenced)

“New” policy – Training & Employment Notice (TEN) 15-10 dated 11/10/10 contains implementation protocol

12

See Handout #2

Who you’re serving…Who you’re serving…Veterans’ Priority of Service Requirements (2)Veterans’ Priority of Service Requirements (2)

Basically, __________ has to be eligible and, of those, “veterans and eligible spouses” receive priority

______ grantees must collect veterans’ characteristics and information to determine if priority of service (POS) applies

Documentation could include letter from veterans’ organization

______________ ok until documentation received

13

Who you’re serving…Who you’re serving…Non-Discrimination / Equal OpportunityNon-Discrimination / Equal Opportunity

Grantees must collect information on Ethnicity, Race, Sex, Age, Disability Status Requirements appear in many places including 29 CFR Part

37, Civil Rights Act of 1964, etc. Purpose of mandatory data collection is to ensure equal

opportunity

Data collected on participants and applicants You have to ask but they don’t have to tell Are your forms properly designed?

Documentation __________________ unless otherwise stated since info is based on __________ ____________

14

See Handout #3

Who you’re serving…Who you’re serving…SSNsSSNs

Collecting SSNs is usually required through the SGA and/or reporting instructions

Example: “Applicants will be required to collect participants’ Social Security numbers as part of individual level data collection…. Applicants must ensure that Social Security numbers will be maintained in a secure and confidential manner.” (ARRA HGEI 6/24/09 SGA)

Needed to calculate common measure outcomes

15

Who you’re serving…Who you’re serving…SSNs (2)SSNs (2)

Individuals cannot be ____________ ____________ for failure to provide a SSN

Documentation could include ________ _________ _________ or military ID

Do any of your policies or procedures [even

unintentionally] require a SSN?

Do your intake forms/documents unintentionally discourage the provision of SSNs?

16

See Handout #4

Who you’re serving…Who you’re serving…Characteristics Characteristics

Could include: Age Dropout status LEP Ex-offender Economically disadvantaged

Documentation can be specified in SGA and reporting instructions if populations are targeted Or sometimes a statutory reference is implied (e.g., WIA

defines adult as 18+, a target for CBJT grantees)

17

See Handout #5

Who you’re serving…Who you’re serving…‘Housing’ Considerations for the Data‘Housing’ Considerations for the Data

“An applicant must collect and report participant-level data from the following categories: Demographic and socioeconomic characteristics; Services provided; and Outcomes achieved.” This text is in every SGA involving the

enrollment of participants

So what should grantees do with all the data that’s collected?

18

Who you’re serving…Who you’re serving…‘Housing’ Considerations for the Data (2)‘Housing’ Considerations for the Data (2)

Participant data should live in both case files and the MIS Establish policy, procedures, protocols for

case files (e.g., __________?) Case files serve as a repository for information

and should be an ______________ of your MIS Example: Case file contains copy of certificate

whereas MIS just notes certificate was attained

Note: Grantees submit reports containing aggregate numbers but they’re based on ___________ ___________

19

Who you’re serving…Who you’re serving…Connecting the DotsConnecting the Dots

Do you have a written eligibility policy, associated forms (e.g., consent forms, intake forms) and protocols in place (especially if there are subs) that incorporate all of the foregoing?

Are there gaps that could lead to compliance issues?

Where are there opportunities for improvement or refinement?

Would you be comfortable with a monitoring visit from your FPO next week?

20

Sample/Handout #4

What they’re getting…What they’re getting…General Grantee RequirementsGeneral Grantee Requirements

Grantees are responsible for program design and delivery—including the design and delivery of services—within federal parameters that are general as well as those that are grant-specific.

21

What they’re getting…What they’re getting…Federal ParametersFederal Parameters

Participants Only

Services (and/or Deliverables)

Scope, Complexity ranges across grantees

Terms / Definitions

Management of / Coordination with subs, providers, partners

22

What they’re getting…What they’re getting…Data Sources and DocumentationData Sources and Documentation

“The applicant must collect participant-level data on . . . employment training and other services provided through the grant.”

Text from an SGA

23

What they’re getting…What they’re getting…Participants OnlyParticipants Only

Participant—Individual determined eligible to participate in the program and who receives a service funded by the program (TEGL 17-05)

‘Qualifying’ Services – Know the services that trigger participation

Example: Eligibility determination by itself does not trigger participation

Participation Date = Date of _________ __________

24

What they’re getting…What they’re getting…Scope/Complexity of ServicesScope/Complexity of Services

Generally speaking, the more complex the scope or range of services, the more “controls” you need

Key Consideration – Systems Needed

Would the decision-making process be easier, more fair, and/or more consistent if we had a policy in place?

25

What they’re getting…What they’re getting…Scope/Complexity of Services (2)Scope/Complexity of Services (2)

Policies – Administrative tools describing what has to be done/why to help staff attain organizational goals

Procedures – Actions comprising the acceptable manner of performance so policies are consistently implemented

Steps 1—Define the issue, task, problem; 2—Determine

responsibility for development; 3—Develop draft using agreed-upon template; 4—Review/revise/make final; 5—Adopt/approve; 6—Implementation (distribution and education); 7—Regular review (eventual archival)

26

What they’re getting…What they’re getting…Service Terms/DefinitionsService Terms/Definitions

Sometimes, services that are allowable are specified as unallowable in the SGA

Remember that Webster’s dictionary is not always applicable

Example: “Received Education or Job Training Activities” means something different for CBJT grantees, HGEI grantees, and YouthBuild grantees

27

What they’re getting…What they’re getting…Management/Coordination CriticalManagement/Coordination Critical

Generally speaking, the more complex the arrangement of partners, subs, providers, the more ____________ you need

Key Consideration – __________ NeededNavigating relationships is hard under any

circumstances (add distance, varied experience, different understandings, existing “controls”)

If you don’t say what you mean, people will interpret it for you

28

What they’re getting…What they’re getting…Connecting the DotsConnecting the Dots

Do you have a ‘participant flowchart’ or similar service design/delivery framework? Are the services provided consistent with the SOW (e.g., sequence, definitions, parameters)? Are protocols in place (especially if there are subs) that incorporate all of the foregoing? Are there gaps that could lead to compliance

issues? Where are there opportunities for improvement or

refinement? Would you be comfortable with a monitoring visit

from your FPO next week?

29

With what results…With what results…General Grantee RequirementsGeneral Grantee Requirements

All grantees need a management information system that, at a minimum, captures required data, performs necessary calculations, and reports information needed to manage the grant and meet federal reporting requirements.

30

With what results…With what results…Federal ParametersFederal Parameters

Exiters Exit vs. Completion Outcomes

Terminology; Common Measures; Other Measures

Unexpected Circumstances Gap in Service; Performance Exclusions

Quarterly Program Reporting Management of / Coordination with subs,

providers, partners

31

With what results…With what results…Data Sources and DocumentationData Sources and Documentation

“An applicant must collect and report participant-level data [including] outcomes achieved.”

“Selected applicants will have to include projections and track such information as ‘total number of participants who complete employment and training activities that receive a degree, certificate, or other type of credential’.”

Typical text (generally within the same SGA)

32

With what results…With what results…ExitersExiters

Exiter – A participant who hasn’t received a program or partner-funded service for 90 consecutive days and is not scheduled for future services (TEGL 17-05)

Exit Date is applied retroactively* to the last date of service (after 90 days of no service)

Exit Date = Date of ________ _________

Exit must be understood by all grantees Represents a trigger (e.g., inclusion in

common measure outcomes, beginning of youth follow-up)

33*Exception for YouthBuild grantees

With what results…With what results…Exit vs. Completion Exit vs. Completion

Exit is defined for you; Completion is usually defined within general parameters (grantees add the meat)

Completion often means successful completion

Everyone exits but not everyone completes (see your program reporting instructions)

34

With what results…With what results…Outcomes—TERMINOLOGY Outcomes—TERMINOLOGY

If it’s not clear, get clarity! You’re accountable

“Completion”

The one-time-only nature of completion often presents challenges

Completion of entire program vs. completion of one ‘step’ or ‘course’

35

With what results…With what results…Outcomes—TERMINOLOGY (2)Outcomes—TERMINOLOGY (2)

“Entered Employment” To be [the common measure] or not to be

Example: CBJTs and HGEIs are accountable for two different “entered employment” outcomes

Projected “Enrollments” Your projections were based on your

understanding of enrollment at the time you wrote the grant, which may not have been correct

36

With what results…With what results…Outcomes—COMMON MEASURESOutcomes—COMMON MEASURES

All discretionary grantees are responsible for some common measure outcomes Adult Entered Employment Rate Adult Employment Retention Rate Adult Average Earnings Youth Placement in Employment/Education Youth Attainment of Certificate/Credential Youth Literacy/Numeracy

ALL calculations are based on the Exit Date with exception of Youth L/N

37

With what results…With what results…Outcomes—OTHER MEASURESOutcomes—OTHER MEASURES

Note: Required program report formats will never fully account for what you have done with the funds you received!You could ‘stay within the box’

and still tell your story

38

With what results…With what results…Unexpected Circumstances (1)Unexpected Circumstances (1)

[Slide #33 said Exit is based on 90 days of no service…] – With one exception

A participant should not be considered as exited if there is a gap in service greater than 90 days based on one of three circumstances:

1. Delay before beginning of training2. Health/medical condition of participant or family member3. Temporary move that prevents participation

Grantees need to understand federal parameters regarding gaps in service

39

See Handout #6

With what results…With what results…Unexpected Circumstances (2)Unexpected Circumstances (2)

Certain conditions allow individuals to be excluded from common measure outcomes Institutionalization Health/Medical or Family Care Death Reservist Called to Active Duty Relocation to Residential or Non-Residential

Program (youth only) Exclusions apply if circumstance takes

place at exit or in the 3 quarters following

40

With what results…With what results…Quarterly Program ReportingQuarterly Program Reporting

“Grantees must agree to meet DOL reporting requirements.” (SGA Text)

OMB-approved Program reporting requirements vary across

discretionary grantees Across grantees. . .

What Do FPOs Look For? General Suggestions About Data Quality

41

What Do FPOs Look For?

1. _____________ – Reports due NLT 45 days after the end of the quarter (program and fiscal reports)

2. _____________ – All sections included, no pertinent information missing, referenced attachments are included

3. _____________ – cumulative numbers go up and not down, number of males and females equal total enrollments

42

With what results…With what results…Quarterly Program Reporting (2)Quarterly Program Reporting (2)

General Suggestions Work on it _____________ the quarter Maintain easy access to your _______ As they occur, check off/crosswalk activities,

products, and deliverables to your SOW Keep list of events, meetings and activities you

would report to the leadership of your organization (e.g., college president)

Schedule time before the date to enter data, talk with staff, etc.

…and make sure your subs/partners do the same

43

With what results…With what results…Quarterly Program Reporting (3)Quarterly Program Reporting (3)

About Data Quality Ensure data elements are well-__________ and

___________ by project staff (including contractors) Note: Original source of information is more reliable

than secondary information (e.g., info on ‘training completion’ from a community college should be used rather than second-hand info from participants through case managers)

Ensure duplication of effort does not occur in data collection (costly, lack of consistency)

Procedures needed (e.g., SOPs with timeframes and deadlines, responsibilities assigned)

44

With what results…With what results…Quarterly Program Reporting (4)Quarterly Program Reporting (4)

With what results…With what results…Connecting the DotsConnecting the Dots

Do you have policies and procedures defining outcomes (ours and yours) and associated data elements and how the data are gathered, processed, reported (especially if there are subs) that incorporate all of the foregoing? Are there gaps that could lead to compliance

issues?

Where are there opportunities for improvement or refinement?

Would you be comfortable with a monitoring visit from your FPO next week?

45

ConclusionConclusion

When you start connecting the dots you soon realize that, practically speaking, you cannot separate program design from program delivery, or either of those from program management, or any of those from program accountability. At least not without something important falling through the cracks.

46

U.S. Department of Labor, ETA, Region 4 U.S. Department of Labor, ETA, Region 4 Discretionary Grantee Training ConferenceDiscretionary Grantee Training Conference

March 1-2, 2011March 1-2, 2011

Accountability & Grants Management: Connecting the Dots

47