Event Evaluation - FEI.org

Event Evaluation

FEI Sports Forum, 8-9 April 2013, IMD Lausanne

• For the past four years, we have conducted evaluations on World Cup Qualifiers in North America.

• These evaluations differ from the usual procedures completed for the FEI because they examine all aspects of the event, not just what would affect the competitor.

Event Evaluation


• The system proved successful in demonstrating areas of strength and weaknesses to Organizing Committees, so we looked at developing a similar method that could be used more widely to evaluate events, including perhaps in the other disciplines.

Event Evaluation


• Using a system based on that used in North America, we completed test evaluations of three events in Western Europe this winter/spring. For the purpose of this presentation they will be known as Example A, Example B and Example C.

Event Evaluation


• In N.A. scores for the 2011 season ranged from 61.7% to 84.5%.

• In the 2012 season many shows improved their scores. For Example: – WEF improved their scores nearly 10% – Toronto improved 4.5% – The Hampton Classic improved 4% The general trend over the four years was that shows would maintain (within +/- 2%) or improve.

Event Evaluation in North America


• In N.A. we used an independent evaluator at each show.

• To avoid this additional expense, we modified this process for our test events by utilizing individuals already present at events, such as spectators, judges, riders, etc.

• Each receives a portion of the questionnaire that is within their knowledge to answer successfully. (Foreign Rider Survey, Judge Survey, Course Designer Survey, Steward Survey, etc)

Event Evaluation in North America


• Questions are tailored to be fact based, rather than opinion. The completed questionnaires are combined into a score sheet, which showcases each event’s strengths and areas for improvement.

(SEE FOLLOWING SAMPLE QUESTIONS)

Differences in N.A. and Europe Evaluations


Here are some examples of questions used: • Were timely, accurate & complete results for

all horses made available… 1. Within ½ hour after competition? (1 point)

2. In hard copy? (1 point) 3. Via Internet? (1 point) • Was the World Cup competition… 1. Televised? (2 points) 2. Live streamed? (2 points)



• What is the seating capacity of the arena? 1. More than 10,000 (4 points) 2. 7,501 – 10,000 (3 points) 3. 5,001 – 7,500 (2 points) 4. 2,501 – 5,000 (1 point) 5. Less than 5,000 (0 point)



• What percentage of the seating was filled during World Cup/Grand Prix?

1. 76% - 100% (3 points) 2. 51% - 75% (2 points) 3. 26% - 50% (1 point) 4. Less than 26% (0 point)



• What mandatory fees were charged? 1. No fees (1 point) 2. Manure disposal ( - 1 point) 3. Electricity ( - 1 point) • Was their adequate staff in the show office 1. All of the time? (3 points) 2. Most of the time? (2 points) 3. Some of the time? (1 point)



• Approximately how many vendors were present in the trade area?

1. More than 25 (4 points) 2. 15 to 24 (3 points) 3. 14 to 5 (1 point) 4. Less than 5 (0 point)



• Was there a sufficient number of food vendors for the public?

1. Yes (1 point) • Was the announcer… 1. Easy to understand? (1 point) 2. Explaining the competition? (1 point) 3. Audible to all in the arena? (1 point)



• Was there a first aid station that was visible and centrally located?

1. Yes, visible (1 point) 2. Yes, centrally located (1 point) • Was the awards ceremony safe for horses

and riders from a welfare point of view? 1. Yes (1 point) 2. No ( - 1 point)



• For the purposes of scoring, areas of the results are weighted for importance. – For example, rule compliance and

footing quality are both weighted heavier than printed material or press facilities.

Event Evaluation


Criteria Group Criteria for Scoring Example A Example B Example C A.: Technical and Compliance A1. FEI Compliance 100.0% 95.1% 97.6%

A2. Medical and Equine Emer. Plans 92.2% 100.0% 92.2%

A3. Footing Quality 91.7% 55.0% 90.0%

A4. Quality of the Course 93.8% 100.0% 62.5%

A5. Class Value for Rankings 100.0% 100.0% 100.0%

Etc.

Total for Compliance/Technical Combined 95.9% 93.4% 93.8% B.: Show/Event - Building the Brand B1. Announcing the Event 100.0% 100.0% 100.0%

B2. Ceremonies & Awards 100.0% 100.0% 100.0%

B3. Exhibition/Trade Area Incl. Food 75.0% 91.7% 75.0%

B4. Organization and Show Office 94.1% 94.0% 94.1%

B5. Attendance 85.7% 100.0% 71.4%

Etc.

B8. Housing/Other Arrangements 75.0% 75.0% 100.0%

Total for Building the Brand 87.2% 98.1% 99.0%

SCORES FOR A,B,C


Criteria Group Criteria for Scoring Example A Example B Example C

C. 1-6: Marketing and Media C1. Sponsors 93.8% 93.8% 93.8%

C2. VIP Amenities: Owners/Enthusiasts 87.5% 80.0% 100.0%

C3. Program 83.3% 100.0% 50.0%

C4. Advertising/Media 100.0% 100.0% 100.0%

C5. Press Facilities 100.0% 100.0% 87.5%

C6. Website 27.8% 44.4% 38.9%

Total for Marketing/Media 65.8% 72.6% 69.3%

C. 7: Fan Experience C7. Amenities for Fans 91.7% 91.7% 91.7%

Total Non-Technical/Compliance 82.5% 88.8% 87.9%

Total Weighted Score 88.0% 90.6% 90.2%

SCORES FOR A,B,C


• Looking through Example A’s scorecard, we are able to quickly identify areas for improvement for future years in the following categories: – Exhibition/Trade Show including Food (75%) – Program (83%) – Website (28%)

Note: while scores above 70% are “passing” , a few 70-80%

scores have been marked as areas for improvement for a show looking to be at the top of the market in all areas.

Closer Look at Test Event Scores: Example A


• We are also able to see areas of strength for Example A – FEI Compliance (100%) – Footing Quality (92%) – Organization and show office (94%) – Advertising and Media (100%)

The overall final weighted score for Example A was 88%.

Closer Look at Test Event Scores: Example A


• Areas for improvement for Example B – Footing Quality (55%) – VIP Amenities: Owners/Enthusiasts (80%) – Website (44%)

Note: while scores above 70% are “passing” , a few

70-80% scores have been marked as areas for improvement for a show looking to be at the top of the market in all areas.

Closer Look at Test Event Scores: Example B


• Areas of Strength for Example B – Quality of the Course (100%) – Attendance (100%) – Program (100%) – Press Facilities (100%)

The final score for Example B was 90.6%.

Closer Look at Test Event Scores: Example B


• Areas for improvement for Example C – Quality of course (63%) – Attendance (71%) – Program (50%) – Website (39%)

Note: while scores above 70% are “passing” , a few

70-80% scores have been marked as areas for improvement for a show looking to be at the top of the market in all areas.

Closer Look at the Test Event Scores: Example C


• Areas of Strength for Example C – Footing Quality (90%) – Housing/Other Arrangements (100%) – VIP Amenities: Owners/Enthusiast

(100%) – Advertising/Media (100%)

The overall score for Example C is 90.2%.

Closer Look at the Test Event Scores: Example C


• Let’s look at Example B: – Perhaps the Management of Example B is

hoping to increase the number of top riders bringing their top horses in the following season and looking for ways to entice them to their event.

– A look at their scores identifies that the riders currently have a concern with their footing and gives management a tangible item to address to help meet their goals.

How does this help the events?


• Let’s look at Example C: – The only test event to score 100% on

Amenities for VIPS and Enthusiasts, but also the lowest score for Attendance at 71%.



• Let’s look at Example C: (continued) – Advertising score is strong, but low

website and press facilities scores might hint to Management that word about their strong event is not being well communicated to possible attendees.

– This gives Management a real reading on areas to improve in the future to help increase attendance numbers.



• The tests we’ve done on the European events have been helpful in bringing to light necessary tweaks to the weighting values on different areas, as well as the questions themselves.

• As we continue to fine tune the process, we will send the questions to all departments of FEI for feedback.

Future


• For a rider, sponsor or fan considering attending or partnering with an event, these numbers may steer one in a direction that matches their goals.

• For events looking to improve, this gives valuable feedback moving forward and will help to strengthen the FEI brand as well as the sport itself.

How does this help riders, sponsors, media, etc?


• The major differences from what has been happening in prior years with evaluations that go to the FEI: – The questions can be more objective

and complete and they can be graded as opposed to pass/fail.

– The other important difference is that we are able to generate a number score for the show and thus a ranking for all events.

Conclusions


• With further input, honing the weighting including prize money, strength of the field, mandatory fees, the scores will be able to be used to solve calendar disputes, assign star levels and can be tied to the rider ranking list for point awarding.

Conclusions


• Officials will be more motivated to fill out the forms when they know their input offers real influence on scores. They are able to give input on a show that offers suggestions for improvements, but doesn’t necessarily damage the show.

Conclusions


THANK YOU

FOR YOUR ATTENTION

Event Evaluation

Documents

Event Evaluation - FEI.org