Upload
kevin-mayles
View
243
Download
0
Tags:
Embed Size (px)
Citation preview
Riding the tiger: dealing with complexity in the implementation of institutional strategy for learning analytics
Kevin Mayles, Head of Analytics, Open University
The Open University Mission
3
Student profile
Nearly 30% of new OU undergraduates are under 25
The average age of our new undergraduate students is 30
Only 9% of our new students are over 50
42% new undergraduates have 1 A-Level or lower on entry
Over 17,400 OU students have disabilities
11,000 OU students are studying at postgraduate level
p.5
A clear vision statement has been developed to galvanise effort across the institution on the focused use of analytics
Analytics for student success vision
VisionTo use and apply information strategically (through specified indicators) to retain students and progress them to complete their study goals
MissionThis needs to be achieved at :● a macro level to aggregate information about the student learning experience at an
institutional level to inform strategic priorities that will improve student retention and progression
● a micro level to use analytics to drive short, medium and long-term interventions
Vision in action
The OU recognises that three equally important strengths are required for the effective deployment of analytics
Underpinning organisational strengths
Adapted from Barton and Court (2012)
The OU recognised three equally important strengths are required for the effective deployment of analytics
Underpinning organisational strengths
We need to ensure we have the right architecture and processes for collecting the right data and making them accessible for analytics – we need a ‘big data’ mind-set
The OU recognised three equally important strengths are required for the effective deployment of analytics
Underpinning organisational strengths
The university needs world class capability in data science to continually mine the data and build rapid prototypes of simple tools, and a clear pipeline for the outputs to be mainstreamed into operations
The OU recognises that three equally important strengths are required for the effective deployment of analytics
Underpinning organisational strengths
Benefits will be realised through existing business processes impacting on
students directly and through enhancement of the student learning
experience – we will develop an ‘analytics mind-set’ in
these areas
For/in/on-action adapted from Schön (1987)
The OU is developing its capabilities in 10 key areas that
build the underpinning strengths required for the effective deployment of analytics
Analytics enhancement strategy
12
Development of predictive indicators
Application of a student number forecasting model to trigger interventions with vulnerable students
Calvert (2014)
13
Development of predictive indicators
The 30 variables identified associated with success vary in their importance at each milestone
Student
(Demographic)
Student – previous study/motivation
Student progress in previous OU
study
Student – moduleQualification /
module of study
Calvert (2014)
14
Application of predictive indicators
15
Application of predictive indicators
16
Application of predictive indicators
Technology showcase
4.30pm Thursday
17
Learning design link to success
Rienties et al (2015)
18
Learning design link to success
Rienties et al (2015)
Concurrent session 6C
2.45pm Thursday
19
Implementation at the OU
© Transport for London
p.20
The complexity challenge
What is project complexity?
● “Complicated”: e.g. a Swiss watch● “Complex”: from the Latin ‘complexus’ (braided together). Nonlinear and
unpredictable.●Like quality – it is hard to quantify and is something that is experienced
● Language: an analogy – not based in complexity science / complex adaptive systems theory
● Subjective not objective● Complexity is art not science
Maylor et al (2013)
p.21
Complexities
● Structural complexity●Number, size, financial scale, interdependencies, variety, pace, technology, breadth
of scope, number of specialties, multiple locations/time zones● Socio-political complexity●People, politics, stakeholder / sponsor commitment, resistance, shared
understanding, fit, hidden agendas, conflicting priorities, transparency● Emergent complexity●Technology and commercial maturity and novelty, clarity of vision / goals, clear
success criteria / benefits, previous experience, availability of information, unidentified stakeholders
● Assessed through the ‘Complexity Assessment Tool’
Maylor et al (2013)
p.22
How complex is the OU Analytics project?
Structural
Socio-politicalEmergent
OU Analytics Project Complexity
H
M
L
p.23
Responding to complexities
Complexity Response
Structural Socio-political Emergent
Plan and control
Plan comms (inc. clear visualisation); isolate key
tasks; create project board of stakeholders
Co-location; use PMO as point of control; scenario planning; change control
RelationalPrioritise communication with stakeholders; reach
out to others Socialise changes; revisit
assumptions; increase formal communication
Flexibility (Risk and change)
Anticipate refinement and testing; change
control; parallel developments
Manage expectations of change; revisit benefits regularly; ‘look-ahead’
with client
Maylor et al (2013)
p.24
Complexities faced at the OU
Structural Socio-political Emergent
Benefits - clarity
Unfamiliar technology
Supply chain not in place
Skills shortage
Integration of technical disciplines
Dependencies
Pace
Experience of staff
Culture change needed
Impact of organisational change
External stakeholder alignment and understanding
Benefits and success measures will become
clear
Technology will become familiar and change
Scope, schedule and resource availability likely
to change
Stakeholder engagement will improve
p.25
What have we done, what have we learned?
Structural Socio-political Emergent
Effective projectmanagement controls in
place
Agile method – early delivery and iterate
You can never do enough communicating
Revisited benefits regularly
Project board – wide representation – including
the doubters
High profile amongst senior leadership
Spend time on key (loud) stakeholders
Direct control of resources – small dedicated team
leading the way
Get small pilots going and people come on board
Change control – use it!
p.26
You cannot control the complexity…
Thank you…
Are there any questions?
For further details please contact:● Kevin Mayles – [email protected]● @kevinmayles
References:BARTON, D. and COURT, D., 2012. Making Advanced Analytics Work For You. Harvard business review, 90(10), pp. 78-83. CALVERT, C.E., 2014. Developing a model and applications for probabilities of student success: a case study of predictive analytics. Open Learning: The Journal of Open, Distance and e-Learning.MAYLOR, H.R., TURNER, N.W. and MURRAY-WEBSTER, R., 2013. How Hard Can It Be? Research Technology Management, 56(4), pp. 45-51. RIENTIES, B., TOETENEL, L. and BRYAN, A., 2015. “Scaling up” learning design: impact of learning design activities on LMS behaviour and performance. Proceedings of the 5th Learning Analytics and Knowledge Conference 2015.SCHÖN, D.A., 1987. Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco, CA, US: Jossey-Bass.