Automated Assessment of Kinaesthetic Performance in RowingSimon Fothergill

Ph.D. student

Digital Technology Group, Computer Laboratory, University of Cambridge

SeSAME Plenary Meeting, 2nd September 2010, Cardiff

Can assessment of kinaesthetic performance be automated?

Feedback is fundamental pedagogical mechanism is sport

Sense and Optimise

Automate to supplement.

Rowing is a novel domain for well known algorithms

• Capturing Kinetics

• Collection of Corpora

• Stroke similarity

• Identifying Improvements

• Useful feedback

Synchronised capture of multiple forms of kinetics

Simple, real-time feedback helps fatigued athletes

Post-workout feedback

Rich, flexible source of data

Dataset

• Real and uncontrived

• Large

• Representative of the performance

• High fidelity

• Synchronised

• Segmented

Data capture system

• Compatible

• Equipment augmentation

• Annotation

• Security

• Portable

• Cheap

• Physically robust

• Extensible platform

Reliable, real-world deployment for over 1 year

Stroke Similarity is an important form of feedback

Basic and sophisticated forms of feedback

Questionnaire (GB Rowing news feed), observations of deployment and coaching sessions, coaches

comments

Analysis of kinematic trajectories impacts many areas

Movement variability profiles as diagnostic tool, could suggest fatigue, higher variability can

reduce injury

Training may become inefficient if consistency drops off, abnormal behaviour can be detected, similarity to ideal (coach defined) targets can be measured, consistency is a good coarse grain performance metric (for novices).

A definition of is arbitrary and subjective

Characteristics of motion trajectories

Overall or individual aspect

Different populations of strokes, such as inter and intra athlete

Collection of Corpora is logistically challenging!

The number of unsupervised, unselfconscious, and curious athletes with range of skills is limited

An online system was used to collect performance annotations from national coaches due to their availability.

Judgements for overall performances and the handle trajectories were collected

A B relative comparison considered better than scale Video quality considered acceptable given commentsOverlay considered better side by side

1000's of strokes were captured

20 expert coaches (national and international GB Rowing and CU(L)(W)BC) each gave from about 30 minutes to 3 hours.

Capturing expert opinions on forms of similarity

Evaluate known trajectory and shape similarity metrics

Classes of algorithms:

Difference in distanceDifference in duration

Difference in momentsDifference in outline distance

Accumulative error

Euclidean distance (binary chop)Hausdorff distanceHausdorff with temporal constraintsLCSSDTW (2D, shape matching, truncated)

d2d1

E.g.

d1 < d2

Evaluation with limited, subjective annotations

Evaluate known trajectory and shape similarity metrics

d2d1

E.g.

d1 < d2

Algorithm weighting += (0.8 * c)

Results : Overall performance, inter-athlete

Weighting Algorithm

15.79 DurationDifferenceMetric14.62 AccumulationOfError14.59 NoEndShapeMatching2DTWMetric2D_2.0_5_2014.53 WearingOutDTWMetric_2.0_0.9999 14.53 WearingOutDTWMetric_2.0_0.999 14.53 DTWMetric_2.0 19 / 7214.23 ShapeMatching2DTWMetric2D_2.0_5 13.95 NoEndShapeMatching2DTWMetric2D_2.0_5_10 13.77 ShapeMatchingDTWMetric2D_2.0_10.0 13.34 NoEndShapeMatching2DTWMetric2D_2.0_5_2 13.05 ShapeMatching2DTWMetric2D_2.0_10 13.00 EuclideanDistanceMetric 12.97 LCSSMetric_1.0_2.0 Percentage agreement with trusted consensus of best algorithm: 76%

Results : Overall performance, intra-athlete

Weighting Algorithm

12.39 EuclideanDistanceMetric 11.83 LCSSMetric_1.0_2.0 9.58 DurationDifferenceMetric 8.24 AccumulationOfError 5.15 Hausdorff2Metric 5.15 Hausdorff1Metric 2.22 NoEndShapeMatching2DTWMetric2D_2.0_5_20 2.00 ShapeMatching2DTWMetric2D_2.0_10 1.89 NoEndShapeMatching2DTWMetric2D_2.0_5_2 1.62 DistanceDifferenceMetric 1.44 SpeedInvariantEuclideanDistanceMetric 1.26 ShapeMatching2DTWMetric2D_2.0_5 1.05 NoEndShapeMatching2DTWMetric2D_2.0_5_10

Percentage agreement with trusted consensus of best algorithm: 82%

Results : Handle trajectory, inter-athlete

Weighting Algorithm

38.73 NoEndShapeMatching2DTWMetric2D_2.0_5_1237.76 NoEndShapeMatching2DTWMetric2D_2.0_5_1036.91 ShapeMatchingDTWMetric2D_2.0_10.036.91 ShapeMatchingDTWMetric2D_2.0_12.035.97 NoEndShapeMatching2DTWMetric2D_2.0_5_535.87 ShapeMatching2DTWMetric2D_2.0_534.21 DTWMetric2D_2.034.04 DTWMetric_2.032.32 Hausdorff2Metric31.33 Hausdorff1Metric30.56 ShapeMatching2DTWMetric2D_2.0_230.20 AccumulativeErrorMetric29.47 LCSSMetric_1.0_2.0

Percentage agreement with trusted consensus of best algorithm: 77%

Results : Handle trajectory, intra-athlete

Weighting Algorithm

11.63 ShapeMatching2DTWMetric2D_2.0_510.91 DurationDifferenceMetric10.63 DTWMetric_2.010.38 NoEndShapeMatching2DTWMetric2D_2.0_5_510.30 LCSSMetric_1.0_2.09.86 NoEndShapeMatching2DTWMetric2D_2.0_5_129.70 Hausdorff1Metric9.24 DTWMetric2D_2.08.89 MomentsDifferenceMetric8.71 NoEndShapeMatching2DTWMetric2D_2.0_5_108.64 Hausdorff2Metric7.82 ShapeMatchingDTWMetric2D_2.0_10.07.82 ShapeMatchingDTWMetric2D_2.0_12.06.42 EuclideanDistanceMetric

Percentage agreement with trusted consensus of best algorithm: 57% (Duration difference = 59%)

Summary & Discussion

Overall Performance similarity Inter-athlete: DurationDifferenceMetric (76%) Intra-athlete: EuclideanDistanceMetric (82%)

Handle trajectory similarity Inter-athlete: DTW (NoEndShapeMatching2DTWMetric2D) (77%) Intra-athlete: DTW (ShapeMatching2DTWMetric2D) (57%)

Rate is an important aspect of the overall technique

Explain no reduction for overall intra-athlete case

Euclidean distance – spatio-temporal, (bias towards time)

DTW – spatio-temporal, 2D, bias towards shape (sections)

Conclusions

The length of the warping path between two handle trajectories from the Discrete Time Warping algorithm is the best of the algorithms

investigated to approximate expert coaches judgements of similarity of technique between the corresponding rowing strokes with a reliability

of ~60/70%.

The overall, summary measures of similarity between whole performances can be told from video recordings can be approximated

with reliability of ~70/80%.

Sensor systems ; devil in the detail!

Collection of large corpora and expert annotation is fraught!

Basic and sophisticated forms of feedback have started to be provided using pervading sensors.

Other Work

Stroke similarity:

More careful consideration of the influence of the trajectory characteristics on similarity to further refine algorithms. Use of more than 3D motion trajectories.

Identifying Improvements:

Evaluate algorithms based on HMMs using annotations provided using a 4 value Lickert scale of importance an individual aspect of technique is addressed, where

the consensus is modelled as a Normal distribution with high disagreement. “Importance addressed” and the aspects of technique were carefully chosen using free, natural english comments on performances provided by expert

coaches.

Acknowledgements

GB Rowing

CUWBC

Jesus College Boatclub

Jesus College BoatClub Trust

Cantabs Boatclub

ISEA

DTG

Rainbow group

SeSAME

Computer Laboratory

Jesus College

Andy Hopper

Sean Holden

George Coulouris

Rob Harle

Andy Riice

Brian Jones

Marcelo Pias

Salman Taherian

Richard Gibbens

Andrei Breve

Alan Blackwell

Joe Newman

Andrew Lewis

Relevant Calls for papers

Mobisys 2011

CHI 2011

Data Mining Journal

ICVNZ 2011 (27th Sept 2010)

Pattern Recognition

(Interdisciplinary research struggles against too generic or too broad calls?)

Questions

Thank you for your attention.

Comments and questions, please!