Embed Size (px)
Citation preview
FY 2017 Competition Peer Reviewer Orientation
VeteransUpward Bound
Program
Orientation Presenters:
Ken Foushee
Tameika Holly
Resource Documents:
Sample Technical Review Form (TRF)
VUB Program Regulations
http:/reviewgrants.com/vub/
Greetings and Introductions
2
Reviewer’s Role and Responsibilities
ED Staff ’s Role and Responsibilities
Overview of the Peer Review Process
VUB Legislation and Regulations
VUB Selection Criteria
Purpose of the Orientation
3
One Review Session:
July 10-21, 2017
Reading, writing , and paneling calls at home.
Peer Review Sessions
5
Provide Knowledge and Experience
Ensure Confidentiality
Notify ED Staff of Conflicts of Interest
Read and Provide Thorough Comments
Participate Actively in Panel Discussions
Make Corrections and Edits to Reviews in a Timely Manner so
that All Panels can Clear on Time
Role and Responsibilities of the Peer Reviewer
6
Provide Panel Logistics
Monitor the Progress of Individual Reviews
Foster Panel Discussions
Facilitate a Fair Review Process
Role and Responsibilities of ED Staff
7
Writing Your
Technical Review Forms (TRFs)
8
• TRIO programs have very specific selection criteria, and you are
asked to comment in-depth on each of them:
• Provide an evaluative assessment (use adjectives)
• Include data or examples from the application narrative
• Each sub-criterion comment should be 2-4 sentences long
• One line comments or bulleted lists are not be acceptable
• Each review may take up to a few hours each, which will likely
decrease as you learn the process
Paneling Process
9
• Panels will consist of 3 reviewers, reviewing 10 applications
• Panel Monitor (PM) will facilitate discussions of each application
• Each reviewer must have a completed draft TRF for each application before the
scheduled panel meeting
• Panel discussions occur to iron out misconceptions or contradictory findings
• (Not seeking unanimity; however, variance can be no more than 15 points)
• PM review and edit Draft TRFs prior to sending to Control Room
• Control Room provides second-level quality control, additional edits
• PM provides one set of edited drafts back to reviewers for final corrections and clearance
of applications
• As you can see, this is an involved process and requires every participant to be fully
engaged and respectful of deadlines and expectations.
Additional Review Guidance
Reviewers’ comments and numerical scores
Evaluating Selection Criteria
Writing Tips
Allowable variance
10
Reviewers complete and submit first draft of TRF for application to PM
PM reviews, sends feedback, if needed
Panel conference to discuss application (no more than 1 hour per app)
Reviewers make post-panel revisions within 1 hour
PM prints TRFs, marks up with comments, corrections
PM gives 3 marked up TRFs to CR. CR adds additional comments
CR gives TRFs back to PM; PM reviews and gives TRFs back to reviewers
If necessary, PM has reviewers re-panel conflicting areas
Reviewers make all corrections/changes as indicated on marked TRFs, returns to PM ASAP
PM reviews for corrections; if good, prints clean copies, signs signature page
PM gives completed package to CR; CR reviews and clears
From First Draft to Clearance: The Basic Flow of an Application through Review
11
Veterans Upward Bound • Regulations and Legislation
• Selection Criteria
13
How Did We Get Here?
The Federal TRIO Programs are educational
opportunity outreach programs designed to motivate
and support students from disadvantaged backgrounds.
TRIO consists of 7 outreach and support programs targeted
to serve and assist low-income, first generation college, and
disabled students to progress through the academic pipeline
from middle school to post-baccalaureate programs.
Under the Economic Opportunity Act of 1964, Upward Bound
was the first TRIO program created. In 1976, Upward Bound
expanded to serve returning veterans and thus
Veterans Upward Bound Program was formed.
In 1990, the US Department of Education
created the Upward Bound Math and Science
Program to address the need for specific
instruction in the fields of math and science.
What is the Veterans Upward Bound Program?
A program that provides Federal grants
to projects designed to assist veterans
in the development of skills and
motivation necessary for success in a
program of postsecondary education.
14
Regulations and Legislation
15
34 CFR Part 645, amended to implement changes made to
the Higher Education Act(HEA) of 1965 by the Higher
Education Opportunity Act(HEOA) of 2008.
http://www2.ed.gov/policy/highered/leg/hea08/index.html.
Final regulations amending Part 645 were published in the
Federal Register on October 26, 2010
Laws, Regulations, and Guidance at the following address:
http://www2.ed.gov/programs/triovub/index.html.
Veterans Upward Bound
Structure
16
Determined by the
Project
Participants complete a
prescribed education
program.
17
Academic tutoring
Advice and assistance in secondary and postsecondary course selection
Preparation for college entrance exams and completion of college
admission applications
Information on federal student financial aid programs:
Federal Pell grant awards
Loan forgiveness
Scholarships
Assistance completing financial aid applications
Free Application for Federal Student Aid (FAFSA)
Veterans Upward Bound
Required Services (34 CFR 645.11)
Required Services cont.
Guidance on and Assistance in:
Secondary school reentry;
Alternative education programs for secondary school dropouts
that lead to the receipt of a regular secondary school diploma:
Entry into postsecondary education; and
Education or counseling services designed to improve the
financial and economic literacy of students or the students’
parents, including financial planning for postsecondary education.
18
Required Services cont.
19
Any project that has received funds under this part for at
least two years must include as part of its core curriculum,
in the next and succeeding years, instruction in-
Mathematics through Pre-Calculus
Laboratory Science
Foreign Language
Composition and
Literature
Veterans Upward Bound
Additional Services Required
Intense basic skills development in those academic subjects required for
successful completion of a high school equivalency program and admission
to postsecondary programs.
Short-term remedial or refresher courses for veterans who are high school
graduates but who have delayed pursuing postsecondary education.
Assist veterans in securing support services from other locally available
resources such as Veterans Administration, State Veterans agencies, veteran
associations and other State and local agencies that serve veterans.
Provide special services, including mathematics and science preparation, to
enable veterans to make the transition to postsecondary education.
20
Allowable Costs (34 CFR 645.40)
In-service staff training
Rental of space (if not available at host institution)
Participant room and board
Room and board for staff providing dormitory supervision
Educational pamphlets and material
Student activity fees
Admissions fees, transportation, t-shirts and other costs
One banquet or ceremony
21
Allowable Costs (34 CFR 645.40) (Continued)
Tuition costs (summer bridge component)
Accident and Medical insurance
Courses in English language instruction
Transportation cost for participants for regularly scheduled activities
Transportation, meals, and overnight accommodations for staff
members accompanying participants on project activities
Purchase, lease, or rental of computer equipment, hardware and
software, and service agreements, supplies that support the delivery of
services to participants, technology used by participants in a rigorous
secondary school program of study. 22
Allowable Costs (34 CFR 645.40) (Continued)
Fees for college admissions applications or
entrance examinations if:
Waiver of the fee is unavailable
Fees paid by the grantee to a third party
Tuition cost for course that is part of a rigorous
secondary school program of study.
Project pays for no more than the equivalent of
two courses for a participant each school year.
23
Unallowable Costs (34 CFR 645.41)
Research not directly related to the evaluation or improvement of
the project.
Meals for staff except as provided in 645.40(d).
Room and board for administrative and instructional staff who do
not have responsibility for dormitory supervision of participants.
Construction, renovation or remodeling of any facilities.
Tuition, stipends, or any other form of student financial aid for
project staff beyond that provided to employees of the grantees as
part of its regular fringe benefit package.
24
Veterans Upward Bound
Stipends (34 CFR 645.42)
The following rules govern the amounts of stipends a
grantee is permitted to provide:
The stipend may not exceed $40 per month.
25
Selection Criteria
Section Maximum Points
Need for the Project 24
Objectives 9
Plan of Operation 30
Applicant and Community Support 16
Quality of Personnel 8
Budget and Cost Effectiveness 5
Evaluation Plan 8
Quality of Project Design 5
Total Selection Criteria 105
26
Need for the Project Maximum Points: 24
Sub-criteria:4
Criteria for Demonstrating
Need for the Veterans Upward Bound
The proposed target area lacks the services for eligible veterans
that the applicant proposes to provide;
The large number of veterans who reside in the target area are
low income and potential first generation college students;
A large number of veterans who reside in the target area who
have not completed high school or, have completed high school
but have not enrolled in a program of postsecondary education;
Other indicators of need for a Veterans Upward Bound project,
including the presence of unaddressed academic or socio-
economic problems of veterans in the area 28
Objectives Maximum Points: 9
Objectives
There are four Standardized Objectives:
Standardized Objectives are found on the Program Profile forms.
All four Standardized Objectives must be in the application.
The applicant should have provided the percentages to achieve for each
Standardized Objective.
The Objectives may not be rewritten, restated or reworded.
Applicants could have proposed additional objectives but were
not required to.
30
Objectives
(Continued)
An applicant must develop outcome measures for each objective based on the baseline data.
You must assess if the applicant has proposed outcome measures that are ambitious and attainable.
Use the rubric to score the Objectives
31
Academic Improvement
Standardized Test
_____% of participants who completed
their VUB educational program during
the project year will improve their
academic performance as measured by
a standardized test taken before and
after receiving services from the
project.
32
Education Program Retention and
Completion
_________% of participants served
during the project year will complete
their VUB educational program by the
end of the project year or remain
enrolled in the program.
33
Postsecondary Enrollment
__________% of participants who
completed their prescribed VUB
educational program in the previous
project year (e.g., 2011-12) will enroll in
a program of postsecondary education
by the end of the next project year
(e.g., 2012-13).
34
Postsecondary Completion
_______% of participants who completed
the VUB educational program in one
project year (e.g., 2012-13) and who enrolled
in a program of postsecondary education by
the end of the next project year (e.g., 2013-
14) will complete a program of
postsecondary education within six project
years (e.g., by project year 2018-19). 35
Plan of Operation Maximum Points: 30 Sub-criteria: 10
Plan of Operation
(645.31)(c)
1. The plan to inform the faculty and staff at the
applicant institution or agency and the interested
individuals and organizations throughout the target
area of the goals and objectives of the project;
2. The plan for identifying, recruiting and selecting
participants to be served by the project;
3. The plan for assessing individual participant needs and
for monitoring the academic progress of participants
while they are in Veterans Upward Bound; 37
Plan of Operation
(645.31)(c)
(Continued)
4. The plan for locating the project within the applicant’s
organizational structure;
5. The curriculum, services and activities that are planned for
participants in both the academic year and summer
components;
6. The planned timelines for accomplishing critical elements of
the project;
7. The plan to ensure effective and efficient administration of the
project, including, but not limited to, financial management,
student records management, and personnel management;
38
8. The applicant’s plan to use its resources and personnel to
achieve project objectives and to coordinate the Veterans
Upward Bound project with other projects for
disadvantaged students;
9. The plan to work cooperatively with parents and key
administrative, teaching and counseling personnel at the
target schools to achieve project objectives; and
10. A follow-up plan for tracking graduates of Veterans
Upward Bound as they enter and continue in
postsecondary education. 39
Plan of Operation
(645.31)(c)
(Continued)
Applicant and Community
Support Maximum Points: 16
Sub-criteria: 2
Applicant and Community Support
(645.31)(d)
The applicant is committed to supplementing the project
with resources that enhance the project such as:
Space
Furniture
Equipment
Supplies
Time and effort of personnel other than those
employed in the project
41
Resources secured through written commitments from
community partners.
Institution of higher education must include in its application
commitments from the target schools and community
organizations;
Secondary school must include commitments from
institutions of higher education, community organizations, as
appropriate other secondary schools and the school districts;
Community organizations must include commitments from
target schools and institutions of higher education.
42
Applicant and Community Support
(645.31)(d)
(Continued)
Quality of Personnel Maximum Points: 8
Sub-criteria: 3
Quality of Personnel
(645.31)(e)
The qualifications required of the project director.
The qualifications required of other personnel.
The quality of the applicant’s plan for employing
personnel who have succeeded in overcoming
barriers similar to those confronting the project’s
target population.
44
Budget and Cost
Effectiveness
Maximum Points: 5
Budget
(645.31)(f)
The budget for the project is
adequate to support planned project
services and activities; and
Costs are reasonable in relation to the
objectives and scope of the project.
46
Evaluation Plan
Maximum Points: 8
Sub-criteria: 2
Evaluation Plan
(645.31)(g)
Is appropriate and includes both qualitative and
quantitative evaluation measures; and
Examines in specific and measurable ways the
success of the project in making progress
toward achieving its objectives.
48
49
Plans should clearly indicate:
Types of data to be collected
When data will be collected
What data collection methods will be used
How the data will be analyzed
When reports and outcome data will be available
What information, on a prescribed schedule (monthly, weekly) will be
developed in a manner that meets its goals and objectives
Who is responsible for making sure information is available in a timely
manner and is influencing the ongoing management of the project
Evaluation Plan
(Continued)
Quality of Project Design Maximum Points: 5
50
Quality of Project Design
• In considering the quality of the design of the proposed project, the
Secretary considers the extent to which the proposed project is
supported by strong theory (as defined in the notice)-- a logic model.
• For the purposes of this grant program, components of a logic model
should include the following;
▫ Resources- inputs
▫ Activities- related to program implementation
▫ Results- observable products as a result of the activities
▫ Outcomes- tangible effects or impact of the outputs
51
Quality of Project Design
A logic model (aka a theory of action) means a well-specified conceptual framework that
identifies key components of the proposed process, product, strategy, or practice (i.e., the active “ingredients” that are hypothesized to be critical to achieving the relevant outcomes)
describes the relationships among the key components and outcomes, theoretically and operationally
52
Quality of Project Design
The logic model will help projects plan and monitor their measured outcomes
A logic model should reveal the relationships among the 4 components. The model should include short, mid and long term outcomes
Each component will affect the success or challenges of the other.
53
Quality of Project Design
Reviewers will evaluate to what extent the applicant has demonstrated the quality of the design of the proposed project and the extent to which the proposed project is supported by strong theory.
54
Example of a Program Logic Model (from http://ies.ed.gov/ncee/edlabs/regions/pacific/pdf/REL_20140007.pdf)
55
Summary of Presentation
Peer Reviewer Roles and Responsibilities
ED Staff Roles and Responsibilities
Review Process
Highlights from the VUB Program Regulations
VUB Selection Criteria
56
57
