Partners in Stewardship:NSF, Monitoring, Audit

Resolution, and You

March 10, 2013

New Orleans, Louisiana

Ask Early, Ask Often

Division of Institution & Award SupportOffice of Budget, Finance & Award Management

• Mary Santonastasso, Director

• Charles Zeigler, Senior Advisor, Cost Analysis and Audit Resolution Branch / DIAS

• NSF Fundamentals

• Award Monitoring and Business Assistance Program (AMBAP)

• Audit Resolution

• Discussion

Agenda

NSF Act of 1950• To promote the progress of science…”

• NSB (24) and 1 director, appointed by the president

• Encourage & develop a national policy for the promotion of basic research and education in math, physical, medical, biological, engineering, and other sciences

• Initiate & support basic scientific research

• Evaluate scientific research programs undertaken by agencies of the Federal Government

• Provide information for S&E policy formation

NSF Characteristics

• Independent Agency• Supports basic research

& education• Uses grant mechanism• Low overhead; highly

automated

• Discipline-based structure

• Cross-disciplinary mechanisms

• Use of Rotators/IPAs• National Science

Board

Mathematical& PhysicalSciences

(MPS)

Geosciences(GEO)

Engineering(ENG)

Computer &Information Science &

Engineering(CISE)

BiologicalSciences

(BIO)

Office of theInspector General

(OIG)

DirectorDeputy Director

National Science Board(NSB)

Office of Diversity & Inclusion

Office of the General Counsel

Office of International & Integrative Activities

Office of Legislative &Public Affairs

Social, Behavioral

& EconomicSciences

(SBE)

Education & Human

Resources(EHR)

Budget, Finance & Award

Management(BFA)

Information& Resource Management

(IRM)

NSF Organizational Chart

Chief Financial Officer & Director, BFA

Large Facilities Office

Budget DivisionDivision of Acquisition & Cooperative Support

Division of Institution & Award Support

Division of Grants & Agreements

Division of Financial Management

Budget Operations & Systems Branch

Program Analysis Branch

Contracts Branch

Cooperative Support Branch

Cost Analysis &Audit Resolution Branch

Policy Office

Systems Office

EHR/BIO/OIA Branch

MPS/GEO/SBE Branch

ENG/CISE/OCI/OPP/OISE Branch

Accounting Operations Branch

Cash Management Branch

Financial Systems Branch

Office of Budget, Finance & Award Management

• $7.373 billion• Consistent with

Administration’s commitment to doubling NSF and basic research agencies

• Emphasizes ways that fundamental research contributes to addressing national challenges

FY 2013 Budget Request

DOD, $71.2

HHS (NIH), $31.4

NASA, $9.6

DOE, $11.9

NSF, $5.9

USDA, $2.3DOC, $2.6

All Other, $5.9

Total R&D =$140.8 billion

FY 2013 Request: Total R&D by AgencyBudget Authority in Billions of Dollars

• $27.8 billion in total award funding

• 45,052 active awards– Standard and continuing

grants– Cooperative agreements– Graduate research

fellowships– Other awards

• 3,157 awardee Institutions– Universities / 4-year

colleges– Non-profit organizations– For-profit organizations– Community colleges– Other awardees

Award information as of June 14, 2012

86%

2.2%2.5%

<1%8%

Type of Awardee Organizations Universities / 4-year Colleges

Non-profit Institu-tions

For-profit Institutions

Community Col-leges

Other Awardees

63%

33%

1%<1% 2%

Type of Award InstrumentStandard Grants

Continuing Grants

Cooperative Agreements

Other Awards

Fellowships

NSF Award Portfolio

Risk

Asses

smen

t Monitoring

Activities

Feedback

Oversight and Monitoring

Evolution of NSF Post-Award Monitoring Processes

2002 2004 2010 2012

Formalized monitoring

program piloted

Created the Division ofInstitution and Award

Support

Adjusted risk assessmentbased on findings

Piloted virtual site visits

From Awards To Awardees

Risk-based Awardee Ranking

Prioritize monitoring on: - Highest risk points - Highest dollars - Number of awards

Risk-Based Award Ranking

Category A~7% of AwardeesRisk Points ≥ 32Total Obligation > $500K

42,192 AwardsRanked by risk points

Category B~23% of Awardees

16-32 Risk Points

Total Obligation > $500K

Category C~70% of Awardees

NSF not Cognizant

Risk Points < 16 or

Total Obligation < $500K

3,197 Awardees Ranked by risk points

1

Risk Adjustment Screens1. Institutional factors2. Prior monitoring activities

and results3. Award administration and

program feedback

Risk Adjustment Criteria

Awardee Risk Categories

NSF Award Portfolio

1 2 3

Annual Risk Assessment

Percentage of Portfolio

AdvancedMonitoring

BaselineMonitoring

Incr

easi

ng

ly f

ocu

sed

an

d t

arg

eted

Category

B *

Category A

* Category B selected for advanced monitoring on resource-available basis

Category C

Oversight and Monitoring

Desk Reviews

Federal Financial Report(FFR) Transaction Testing

Grants and Agreements Monitoring

Automated Report Screening

SiteVisits BSRs

Advanced Monitoring

• Desk Reviews – Assess general management environment, review selected accounting and financial management policies and procedures, and obtain financial information submitted by awardees

• Site Visits - Conduct onsite review of selected higher risk award administration areas and follow up on desk review results as needed

• BSRs – Combine desk and onsite reviews of large facility business systems to determine whether the operation of those facilities meets NSF’s expectations for business and administrative management

Site Visits and Business System Reviews

Site Visit Modules

Consultants

Cost Sharing

Final Project Reports

Fringe Benefits

Indirect Costs

Participant Support Costs

Procurement

Program or Award-Related Income

Property and Equipment

Special Terms and Conditions

Subawards and Subrecipient Monitoring

Time and Effort Records

Travel

Business System Reviews

General Management

Award Management

Budget and Planning

Financial Management

Financial Reporting

Human Resources

Procurement

Property and Equipment

Desk Reviews

General Management

Accounting and Financial Management

FFR Reconciliation

Keys to Success for Awardees

• Know requirements• Good Accounting Practices• Know the Scope of the Project• Document approvals and conversations between the

awardee and NSF

Ask Early and Ask Often!

The Future of AMBAP• Maintain comprehensive coverage • Further integrate post-award monitoring activities • Enhance management systems to better track

monitoring data • Develop knowledge base of lessons learned • Share best practices with other agencies • Virtual Site Visits through collaborative technologies

Virtual Site Visit Approach Use the risk assessment outputs to prioritize the pool of awardees

eligible for advanced monitoring activities Develop target number of virtual site visits based upon staff and

resource availability Select specific awardees for virtual site visits based upon the

following criteria:o Geographic Proximityo Awardee Amenabilityo Perceived Capabilityo Prior Monitoring Results

Implement the virtual site visit using a multi-phased approach:

19

Logistics and DocumentCollection

(Pre-Virtual Site Visit)

Virtual Site Visit

Analyses and Resolution

(Post-Virtual Site Visit)

Virtual Site VisitClose-Out

VSV Benefits

20

Resource Savings Travel and per diem costs Personnel time

Focused Interaction Clear agenda; more targeted discussion Document review in advance of meetings

Access to Resources Reach back capability at NSF Awardee access to internal information and experts WebEx to look directly at awardee systems/documents real time

Flexibility in Process Delaying the exit interview On the fly meeting date and time adjustments

VSV Challenges

Determining the Right Organization for a VSV It’s not for everyone Technology capability, straightforward issues, willingness

NSF and Awardee Relations Lack of onsite touch and feel, no hallway conversations Harder for NSF team to get close to the science Cannot put a name to a face as easily as onsite visit

NSF Team Bonding Lost opportunity for cross-division bonding

21

Pyramid of Monitoring

Audit Resolution

Overview• How Audits Originate

• Complementary Roles of NSF Management and the NSF OIG

• Types of Audits

• Audit Resolution Flow Chart

• Questioned vs. Sustained Costs FY 2012

• Audit Resolution Time Frame

How Audits Originate• NSF Office of Inspector General Annual Audit

Plan– NSF Office of Inspector General Staff– NSF Office of Inspector General contract audit firms

• A-133 Audits – Independent Auditors hired by the awardee

How NSF and the Inspector General Work Together• Office of the Inspector General = Independent Audit

Function

• NSF Management Resolves Audit Reports– NSF Management coordinates resolution with the auditee

and the Office of the Inspector General

• Agency Follow-up Official makes the final agency determination

Types of AuditsInternal• Performance and Programmatic Reviews• NSF Financial and Administrative Operations• NSF Financial Statement (Annually)

External• Pre-award• Incurred Costs• Compliance• Internal Controls

Audit Resolution Flowchart

Questioned vs. Sustained Costs FY 2012

Total Questioned Costs

Total Costs Sustained by NSF

$9,342,049 $4,080,970

Why the Difference?• Awardee provides additional/secondary documentation

to NSF that was not available to auditors• NSF makes equity determination based on demonstration

that the project goals have been accomplished• Professional judgment

Audit Resolution Time Frame• Resolution must occur within six months of

issuance to NSF Management• Audit reports unresolved after six months are

reported in the OIG Semi-annual Report to Congress– Complex– Large Amount of questioned costs– Circumstances beyond an institution’s control

Questions and Discussion

Background Materials

• Cost Analysis & Audit Resolution Branch: http://www.nsf.gov/bfa/dias/caar/index.jsp

• Division of Institution & Award Support: http://www.nsf.gov/bfa/dias/index.jsp

• General: http://www.nsf.gov

• Policy Office: http://www.nsf.gov/bfa/dias/policy/index.jsp

Online Resources

Current Advanced Monitoring Activities

34

Key Characteristics of AMBAP Monitoring Activities

Characteristics Desk Reviews Site Visits

Completed Annually • 110 - 150 • 26 - 30

Completion Time• Approximately 110 days and is conducted from a

remote location• Approximately 126 days, including 2-3 days on site

Staff Resources

• Contractor analysts gather information to conduct analysis and prepare the desk review workpapers and report

• NSF Team Lead reviews and approves desk review workpapers and report

• Team of 2 - 3 NSF staff gathers information, completes the on-site visit, and prepares the report

• The Site Visit team often relies heavily on desk review workpapers

Content Review Areas • Assess 3 - 4 Core Review Areas

• Assess 3 - 4 Core Review Areas or follow-up on desk review concerns

• Select 4 - 6 Targeted Review Areas, including mandatory review areas

Awardee

Engagement

Written Communications

• Gather and collect data via email or submitted awardee documentation

• Exchange correspondence with awardees to clarify responses to areas of concern or supporting documentation

• Gather and collect data via email or submitted awardee documentation

• Exchange correspondence with awardees to clarify responses to areas of concern or supporting documentation

Audio Communications • Discussion calls• Discussion calls• Face-to-face interactions

Visual Interactions

• N/A

• Engage face-to-face with awardees• Review award administration systems “in-person” • Observe non-verbal cues related to award administration

practices, such as institutional culture

Technology Considerations

35

 Technology Considerations

Technology Options

DescriptionNSF-Supported Collaboration

ToolsTechnology Requirements Audio Visual

Desktop

Sharing

File Sharin

g

File Storag

e

Video-conferencing

• Collaborate and conduct small meetings utilizing TV screens, webcams, or desktop systems

Yes• Network(s) for the remote site(s) • Compatible equipment brand and internet transmission

speedYes Yes No No No

TelePresence• Provides high definition

videoconferencing which mimics an in-person meeting

Yes• Connection with an institution that has TelePresence

technology• Connection and integration with collaboration tools

Yes Yes No No No

WebEx

• Combines desktop sharing through a web browser with telephone and video conferencing capabilities

• Collaborate through whiteboards, PowerPoint, and desktop sharing

Yes

• Computer or wireless device with an Internet connection• Audio connection is available through the computer or

telephone • Webcam (optional)• Plug-ins may be required (i.e., WebEx)• Licenses may be required• Microsoft (MS) operating system or server may be required

(i.e., MS Lync)

Yes Yes Yes Yes

No

AdobeConnect

Yes

Microsoft Lync

Yes

Video in aBox

• Provides awardee institutions with basic IT collaboration tools (e.g., webcam and instructions to set-up)

No• Assessment of awardee institution’s technological

capabilities (e.g., internet speed, and operating system)Yes Yes No No No

SharePoint

• Allows for collaboration, document sharing, and file storage for NSF and non-NSF staff on an external facing intranet site

Yes• Utilizes internet browser• Requires prior approval from NSF to access external

SharePoint siteNo No No Yes Yes