The Best Practices in Program Portfolio Evaluation - Running an Effective Program Portfolio Evaluation Process

Confidential www.GrayAssociates.com 1 Confidential

Academic Program Selection and Assessment

How to Choose Which Programs

to Start, Stop, Sustain, or Grow

April 2017

Confidential www.GrayAssociates.com 2

Goal for Today’s Session

§  Share an overall approach, and best practices, for an integrated program portfolio evaluation process –  Participants –  Process –  Analysis –  Integration


Agenda

1.  Where do you start?

2.  From Data to Evaluation

3.  Key Take-Aways


But first…


Make sure there is nothing major blocking the path to a decision-making process.

Before You Begin…

Photo source: http://i.imgur.com/wsRm02x.jpg


Before You Begin…

It is important to understand:

§  Programs have life-cycles

§  You are ready to identify where each of your programs are

§ Doing this once is helpful, doing this systematically creates strength

§  This is not something to be done ‘behind closed doors’

Check out the organization’s assumptions surrounding program evaluation.

No process will overcome unidentified assumptions.


So, where do you start?


Thousands of Potential Programs… Everyone has a favorite program…


Limitless Possibilities Everyone has a favorite market metric and source of data.


Integrating Data and Knowledge

A program portfolio occupies two dimensions, and both are important to its evaluation.

Inside the Four Walls

§  Standards §  Faculty §  Facilities §  Cost

Outside the Four Walls

§  Student Interest §  Employment

Demand §  Competitive Intensity §  Your market area

Evaluating a portfolio requires more than just data. It requires a process.


First: Identify the Markets You Serve Where do your students come from?


Schools often serve more than one market, state or region.

Are You National or Local…or Both?

Campus A Campus B


Second: Confirm Strategy

Check it out - Have you drifted?

Anchor the portfolio evaluation in your institution’s strategy and mission.


Third: Decide What is Important

What are your priorities?

What market factors are key for your institution?

Student Demand

Employment Opportunities

Degree Fit Competitive

Intensity


Fourth: Walk Through The Data

Comparisons to other programs help you interpret the data, find opportunities, see problem areas.

100,000 Program X 8,690 9.4% 39,721 -664


Agenda





From Data to Evaluation: Process This is a well-tested and successful process for looking at a portfolio and deciding what programs to “Start, Stop, Sustain, or Grow.”

1. Data and Scoring 2. Workshop 3. Follow-up

§  Uses facts and data effectively

§  Incorporates judgment of key stakeholders

§  Identifies the best new programs, not just “good enough” programs

§  Earns the understanding and buy-in of key stakeholders

§  Positions the organization for next steps

–  Creating an action plan for teaching out, sustaining, fixing, or growing existing programs


From Data to Evaluation: Process Why does it work?

Inside the Four Walls

Outside the Four Walls

Image Source: http://applesofgold.com/Merchant2/wedding-bands/POL52-24-Set1C.jpg

§  The process is transparent

§  It is cross-discipline, cross-functional

§  Uses quantitative data to inform, Academic and Operational knowledge to decide

§  It creates alignment


Where Do You Want to Spend Your Energy?


Cooperative and Transparent Process Good program decisions take more than numbers…they take intellectual capital and teaming.

Workshop participants should include:

•  Academic Leaders and Partners

•  Enrollment/Admissions

•  Finance

•  Marketing

•  Operations

•  Institutional Effectives

•  Student Services

•  Career Services

•  ….and others with a stake and knowledge


The agenda will vary depending on the number of campuses, the number of award levels, and whether the focus is on existing programs, new programs, or both.

From Data to Evaluation: Workshop Agenda

Sample Workshop Agenda

Day 1 Review Workshop Objectives Explain Approach to Program Selection

Decide how to Decide Refine Program Scoring Select New Programs

Day 2 Match New Programs to Campuses/Online

Discuss Current Programs: Grow/Sustain/Fix/Stop Wrap Up



Collaboration and Decision Approach Collaboration foundations:

The degree to which people feel a process, decision or outcome is theirs The degree to which people see and understand the problem in the same way The trait of believing in the honesty and reliability of the other

Decision Models: A Sample

Person-in-Charge Decides Without

Group Discussion

Person-in-Charge Decides After

Group Discussion

Majority Vote

Unanimous Agreement

Red Yellow Green

§  Ownership: §  Alignment: §  Trust:


The workshop will produce an agreed upon view of your current portfolio, and identified new program opportunities.

From Data to Evaluation: The Output

The Master Scorecard


The Workshop is Just the Start…. There will be plenty of work after the workshop.


§  Gap analysis for existing programs: For existing programs that are underperforming in attracting or placing students, what are the reasons, and are they fixable?

§  Deep dives on specific potential new programs –  Employment opportunities: Who are the likely employers? What skills and credentials are

most important in their hiring processes? –  Competitor programs: What content do they cover? How do they position the program in

their marketing materials? Do they target specific types of students with specific credentials? Do they indicate what they expect their students to do with their new credential? How long are those programs, and how are they priced?

–  Financial analysis –  Regulatory analysis –  Accreditation analysis –  Identification of clinical/externship sites (if needed)

§  Planning, development, and launch of new programs


The Goal: A Vibrant Institution


Agenda





Key Take-Aways

§  Don’t ignore any of the big market elements when you’re evaluating programs. –  Student demand –  Job opportunities –  Competitive intensity –  Degree fit

§  Each data element has some flaws or limitations. That’s why we have people. J

§  Don’t ignore institutional knowledge, but don’t allow it to dictate either.

§  Analyzing programs requires a process that respects and engages key stakeholders.

§  Data can inform, and should, but it is management and academic judgement that is required to reach informed choices.

Leverage institutional knowledge and expertise – Engage it!

The most important take-away is the simplest: you have the ability to consistently, comprehensively and cooperatively evaluate programs based on market conditions.


For More Information…

To contact Gray Associates:

Mary Upchurch Senior Partner

Gray Associates, Inc.

[email protected]

480-390-7230

Education

The Best Practices in Program Portfolio Evaluation - Running an Effective Program Portfolio Evaluation Process