101/26/10

U.S. Competition Scoring Analysis

Jan 2011 SSA ConventionG. Frank Paynter, PhdEE

201/26/10

Motivation

• Current Scoring system is ‘opaque’, especially for AAT/TAT

• With few exceptions, nobody understands the scoring system – but it matters!

• Scoring errors are difficult to detect, especially if the day is devalued

• Although it is the conventional wisdom that ‘getting speed points’ is always more advantageous than a long landout, I couldn’t tell that by looking at the formulas (this may no longer be true – film at 11).

301/26/10

Scoring Parameter Definitions*

• SCR = Scored Completion Ratio (~ Finishers/Starters)• STF = Short Task Factor (less than std min time deval)• SPEED = Scored Speed (Finishers)• BESTSPD = Best value of SPEED for any finisher• UTFACTOR = Under-Time Factor• UF = Under-time Finishers• TOC = Time On Course• STOC = Scored Time on Course• MSP = Maximum Speed Points• MDP = Maximum Distance Points• And many more!

* 2009 FAI Regional Rules

401/26/10

TAT Scoring Equations• Speed score is largest value from three ‘simple’ equations: *

– MSP * SPDRATIO– MDP + 30 + MSP .* 0.2 * (SPDRATIO - 0.4)– MDP + 30

• But …– MSP = STF * (600 + 500 * SCR) (but not > STF * 1000)

– MDP = MSP * (0.8 - 0.2 * SCR)– SCR = (#Finishers – 0.75 * UF) / #Contestants– SPDRATIO = SPEED/BESTSPD– SPEED = DIST/STOC– STOC = TOC for over-time finishers, or STOC = MINTIME – (MINTIME

– TOC) * UTFACTOR for under-time finishers– UTFACTOR = 0.1 + 6 * ((DIST/MAXDIST) – 0.85) (but not < 0.1 and not > 1)

– Yikes!

* See 2011 proposed rule changes

501/26/10

TAT Scoring Equations (cont)

• So, MDP + 30 + MSP .* 0.2 * (SPDRATIO - 0.4) becomes...– S = MDP + 30 + MSP .* 0.2 * (SPDRATIO - 0.4)– S = STF * (600 + 500 * SCR) * (0.8 - 0.2 * SCR) + 30

+ 0.2 * STF * (600 + 500 * SCR)(SPDRATIO – 0.4)– And so on and so on...

• To get to an expression in terms of SCR, STF & SPDRATIO

• S = 510 + 280*SCR – 100*SCR2 + SPDRATIO*(120*STF + 100*STF*SCR) – 48*STF – 40*STF*SCR

601/26/10

So, What Now, Coach?

• Humanly impossible to understand the detailed behavior of such complex expressions (and believe me, I tried!)

• Reduced to parametric analysis (setting most factors to a constant, and varying just one or two to see what happens)

• Even this is very complex and difficult to understand, but…

• Choose the important factors:– STF (Short Task Factor)– SCR (Scored Completion Ratio)– SPDRATIO (your scored speed/BESTSPD)

• Use some common scoring scenarios• Then explore some ‘edges’

701/26/10

AAT/TAT Scoring Scenarios

• A good day: Min Time > SMTT, no landouts, no under-time finishers. SCR = STF = 1, UF = 0

• A not-so-good day: Min Time > SMTT some landouts, some under-time finishers: SCR < 1, STF = 1, UF > 0

• A poor day: Min Time < SMTT (Devalued task), landouts, under-time finishers: SCR < 1, STF < 1, UF > 0

• A really bad day – Devalued task, no finishers: SCR = 0, STF < 1, UF = 0

801/26/10

How Max Speed Pts (MSP) vary with SCR & STF

901/26/10

How Max Distance Points (MDP) vary with SCR & STF

1001/26/10

Combined MSP & MDP vs SCR with STF from 0.5 to 1

~360

~150

400

1101/26/10

Last-day ‘Bump’ Scenario*• Pilot A has a 350-point lead over Pilot B in a 5-glider class on the last day of the competition, and an AAT or MAT

is called. Pilot A can ensure an overall win by flying the minimum possible distance ( > standard min distance for the class) and coming home as soon as possible, even if Pilot B wins the day with 1000 points. For example, a 3 Hour MAT with one mandatory turnpoint 30 miles away. Pilot A flies 60 mph to tp1 and returns immediately, for a total distance of 55 sm @60 mph = 55 min MT – 125! Pilot B flies 3 hours at 60 mph, covering 180 sm, winning the day. Pilot B gets MSP = 1000. Pilot A gets:

• Speed score is largest value from:– MSP * SPDRATIO– MDP + 30 + MSP * 0.2 * (SPDRATIO - 0.4)– MDP + 30

• So,– SCR = (#Finishers – 0.75 * UF) / #Contestants = (4 – 0.75 * 1) / 5 = 4.25/5 = 0.85– STOC = MINTIME – (MINTIME – TOC) * UTFACTOR = 180 – ((180 – 55) * 0.1) = 180 – 12.5 = 167.5– SPEED = DIST/TOC = 55 / 167.5 = 32.83 mph– SPDRATIO = SPEED / BESTSPEED = 32.83 / 60 = 0.55– MSP = STF * (600 + 500 * SCR) = 1 * (600 + 500 * 0.85) = 600 + 425 = 1025 1000

– MSP * SPDRATIO = 1000 * 0.55 = 550– MDP = MSP * (0.8 - 0.2 * SCR) = 1000 * (0.8 – 0.2 * 0.85) = 1000 * 0.63 = 630– MDP + 30 + MSP * 0.2 * (SPDRATIO – 0.4) = 630 + 30 + 1000 * 0.2 ( 0.55 – 0.4) = 30 + 1000 * 0.03 = 630 + 30 + 30 = 690– MDP + 30 = 660

• Pilot B gains 1000 – 690 = 310 points on Pilot A, but Pilot A wins overall by 40 points• Situation is worse if day is devalued, as MSP – MDP range gets narrower.

*see 2011 proposed rule changes

1201/26/10

What about ‘MT15’ finishes?

1301/26/10

What about ‘MT’ finishes? (1 of 2)Pilot’s raw speed assumed = BESTSPEED

1401/26/10

What about ‘MT’ finishes on an under called day?Pilot’s raw speed assumed = BESTSPEED

1501/26/10

TAT/MAT and ‘Under-time’

• For TAT, each MT15 finish is equivalent to 0.75 land-outs. For a class size of 7 gliders, each MT15 finish causes an SCR reduction of ~0.1

• MT (<15) finishes aren’t that expensive – between 4 and 7 points/minute on a 1000pt day, less otherwise.

• Under time arrival penalties are reduced for pilots who fly distances approaching MAXDIST (penalty reduction starts to be significant at about 0.8 * MAXDIST).

1601/26/10

Scoring quiz: what’s wrong with this picture?

1701/26/10

“The rest of the story”

18561852

18571656

1801/26/10

But wait!.....

• MSP = STF*(600 + 500*SCR) (MSP <= 1000)

• SCR = #Finish/#Starts = 6/7 = 0.857• MSP = STF*1028.571• So, for MSP = 1000, STF = 0.972• STF = TOC for BESTSPEED/SMTT• TOC (for MSP =1000) = STF*SMTT = 1.944hrs = 1:56:67

So, with just one land-out, I actually didn’t have to circle at all – could have come in 4 (well, 3.5) minutes under time and still gotten 1000! However, had two or more landed out (or the equivalent in MT15s), then it would have paid to wait out the SMTT.

1901/26/10

Combining FAI Classes

• 15m & std: std gets 2% handicap• 15m & 18m: 15 gets 2% handicap• Std/15/18m: 15 gets 2%, std gets 4%• But, in Sports class, relative to ASG-29/V2C

– V2bx gets 3.4% hdcp– D2bx gets 8.3% hdcp (4.8% over V2)

• 15m guys are better off in Sports class!• Note: 2011 rule changes reflect this – can now

have handicapped class with ballast allowed, but handicaps are for dry weight

2001/26/10

Proposed 2011 Rule Change

• 2010: Speed score is largest value from:– MSP * SPDRATIO– MDP + 30 + MSP * 0.2 * (SPDRATIO - 0.4)– MDP + 30

• 2011: Points for finishers - larger of:• MSP * SPEED / BESTSPD• MDP * DIST/BESTDIST + 30 (But not greater than

MDP + 30.)

2101/26/10

Proposed 2011 Scoring Rule (1 of 3)

Max Dist Pts + 30

2201/26/10

Proposed 2011 Scoring Rule (2 of 3)

Max Dist Pts + 30

2301/26/10

Proposed 2011 Scoring Rule (3 of 3)

2401/26/10

Conclusions (1 of 2)• Speed points vs Spd Ratio curve no longer has a

MDP+30 ‘floor’. 2011 scoring rule change for ‘short finishers’ means you have to earn speed points the old-fashioned way – by actually flying the distance

• 600 pts is *not* the highest score for a long land-out! On a day with just one or two land-outs (SCR ~0.8), it can go as high as 640 (665 with airport bonus), squeezing the speed point range even more than normal.

• MT15 = 0.75 land-out. MT cost is 4-7pts/minute on a 1000pt day.

2501/26/10

Conclusions (2 of 2)• For CD’s – no real reason to call a short task (less than

standard minimum time), causing the day to be devalued from the outset. If the day won’t support the min time, then under-time finishes will go up and have the same effect. If the day turns out better than expected, so much for the better.

• Combining FAI classes sort-of works for std & 15m (same span), but doesn’t work well for 15m/18m span. In very weak weather, the 18m guys can stay up, but 15m cannot. A 15m or std pilot is better off going into Sports class, where the handicap over 18m is much better. 2011 rule change allows ballasted handicap class (but handicap uses dry weights)

• It pays to read the rules!

2601/26/10

Questions?