Ascilite 2012

Analytics and Complexity

Learning and leading for the future

Colin Beer (CQUni)David Jones (USQ)

Damien Clark (CQUni)

http://farm1.staticflickr.com/175/416049380_a6ec6d985f_z.jpg?zz=1

Some definitions

http://farm4.staticflickr.com/3296/2981195093_a1c3b329ff_z.jpg

Managerialism

“The teleological approach to the management of universities is known as managerialism and its influence has extended to how universities manage their learning and teaching”

Beer, Jones & Clark (2012)

Educational Data Mining“Educational Data Mining is an emerging discipline, concerned with developing methods for exploring the unique types of data that come from educational settings, and using those methods to better understand students, and the settings which they learn in.”

George Siemens, 2011 (http://www.learninganalytics.net/?paged=2)

http://farm6.staticflickr.com/5102/5675315304_b933fb0141_z.jpg

Academic Analytics…“marries statistical techniques and predictive modeling with the large data sets collected by HEI, including those collected by the LMS. Academic analytics has been described as business intelligence for HEI and is focused on the needs of the institution, such as recruitment, retention and pass rates”

Open University, 2012

Learning Analytics…“the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs”

George Siemens, 2011 (http://www.learninganalytics.net/?paged=2)

Educ

atio

nal D

ata

Min

ing

Academic

Analytics

Learning Analytic

s Course Level

Program Level

Faculty Level

School Level

Institutional Level

National Level

Adapted from Siemens (2011)

• Multiple CQUniversity internal L&T grants• DEHUB research grant (2011)• Numerous publications• Numerous conference presentations• Established 2008

The Indicators story so far

http://farm8.staticflickr.com/7013/6681523431_2163173930_z.jpg

Blackboard

300M+ records

Moodle

80M+records

PeopleSoft+80,000 Student results

PeopleSoft+80,000Student records

IndicatorsPlatform

independent data

Moodle 2

30M+records

SRQ

+5000records

Some simple patterns

WF F P C D HD0

50100150200250300350400450

Hits (n=39087)

Student Grades

Stud

ent

clic

ks

Some simple patterns

F P C D HD

-4-3-2-1012345

First Day of Access (n=35623) Distance Students

Student Grades

Firs

t da

y of

acc

ess

Some simple patterns

F P C D HD0

0.20.40.60.8

11.21.41.61.8

2Number of question marks

Student Grades

Aver

age

num

ber

of q

uest

ion

mar

ks

http://farm1.staticflickr.com/55/127555697_993c92dcde_z.jpg?zz=1

The next big thing

“BIG data sets showing what students do online may prove as vital to education as genome databases have been to genetics or Europe's Large Hadron Collider to physics”

The Australian (15th October 2012)

The next big thing

“EDUCAUSE and the Bill and Melinda Gates Foundation have targeted learning analytics as one of 5 categories for funding initiatives”

Educause (2012)

The next big thingLearning analytics promises to harness the power of advances in data mining, interpretation, and modeling to improve understandings of teaching and learning, and to tailor education to individual students more effectively.

Horizon report (2011)

http://farm1.staticflickr.com/150/354943641_c0157ed7ab_z.jpg?zz=1

Potential Problems

Abstraction losing detailOrganisational structuresConfusion between correlation and causationAssumptions of causality

“…the nature of learning analytics and its reliance on abstracting patterns or relationships from data has a tendency to hide the complexity of reality”

Gardner Campbell (2012)

Abstraction losing detail

Some simple patterns

F P C D HD0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

Forum Posts Forum Replies

Student Grades

Aver

age

num

ber

of p

osts

and

re

plie

s

The mythical mean

0

2

4

6

8

10

12

14

16

18

20

Moodle courses across a single year

Aver

age

num

ber

of c

ontr

ibut

ions

pe

r st

uden

t

Single HD student

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

5

10

15

20

25

30

35

Individual courses

Num

ber

of fo

rum

con

tri-

buti

ons

Organisational Structures

http://farm4.staticflickr.com/3468/3860278043_87001cbb7f_z.jpg

Design by division

http://farm6.staticflickr.com/5051/5429942502_c443eee156.jpg

Inter-departmental rivalry

http://farm3.staticflickr.com/2276/1951521730_589af2f4b2_z.jpg

Confusing correlation with causation

http://farm6.staticflickr.com/5254/5415215092_acd78b7b18_z.jpg

Some simple patterns

F P C D HD0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

Single HD student

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

5

10

15

20

25

30

35

Assumptions of causality

http://farm4.staticflickr.com/3420/3949835761_7df5f8ddf3_z.jpg?zz=1

Minimum Standards

61%39%

Complex adaptive systems

http://farm1.staticflickr.com/24/55070909_c728cc77ff_z.jpg?zz=1

Complex adaptive systems

“A CAS is a dynamic network of semiautonomous, competing and collaborating individuals who interact and coevolve in nonlinear ways with their surrounding environment. These interactions lead to various webs of relationships that influence the system’s performance”Boustani (2012)

Macro level

Micro level

http://farm4.staticflickr.com/3538/3784932357_f634e5e440_z.jpg?zz=1

Educ

atio

nal D

ata

Min

ing

Academic

Analytics

Learning Analytic

s Course Level

Program Level

Faculty Level

School Level

Institutional Level

National Level

Adapted from Siemens (2011)

http://farm1.staticflickr.com/137/343753843_4c34982317_z.jpg

http://farm4.staticflickr.com/3487/3191664147_689377427c_z.jpg