Learning Analytics – Ethical questions and dilemmas

Ethical questions and

dilemmas of Learning

Analytics Tore Hoel, Henri Pirkkalainen, Kati Clements, Thomas Richter, Thomas

Kretschmer

and Christian M. StrackeeConference, Belgrade, September 2014

Co-organised withlaceproject.eu

Outline

• What is Learning Analytics

• Scenarios of the quantified learner

• Workshop to gather (ethically reflected) solutions

• Introduction to Potter Box – a method used for the developing the solutions

• Ethical questions, dilemmas and solutions

What is Learning Analytics?

“actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data”

Cooper, A. 2012 – Cetis Analytics Series What-is-Analytics-Vol1-No-5

Data

From Data to Insights

Data Analytics

From Data to Insights

Data Analytics Insight

From Data to Insights

Data Analytics Insight

Who?InstitutionTutorSelf…EducationalCommercial

From Data to Insights

Data Analytics Insight

Who?InstitutionTutorSelf…EducationalCommercial

How?Social networkDiscourse ContentDispositionContext…Administration

From Data to Insights

Data Analytics Insight

Who?InstitutionTutorSelf…EducationalCommercial

How?Social networkDiscourse ContentDispositionContext…Administration

What?PlatformService…AvailabilityAccess

From Data to Insights

Data Analytics Insight

Who?InstitutionTutorSelf…EducationalCommercial

How?Social networkDiscourse ContentDispositionContext…Administration

What?PlatformService…AvailabilityAccess

From Data to Insights

Why?

From Data to Insights

Handling ethical dilemmas

• Finding the signal in the noise, patterns in the chaos (Silver, 2012)

• “Data and data sets are not objective; they are creations of human design. We give numbers their voice, draw inferences from them, and define their meaning through our interpretations” (Crawford, 2013)

Data Flows…

… watch ContextsIntegrity Norms

... and now, – the workshop

Using a ethical approach, following the 4 steps of the Potter Box

The taskWe want some advice! You should give me some ethical & valid solutions:

We have concerns about Privacy in LA. What is the most serious concern? What is your recommendation / solution (that stand an ethical test)?

E.g., Concern: Control of data. Solution: Learner should control all use of their own data.

Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

The Potter Box Model of Reasoning

Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

The “Potter Box”

• Dr. Ralph Benajah Potter, Jr.

• Professor of Social Ethics Emeritus BA,  Occidental College BD,  McCormick Theological Seminary ThD,  Harvard University

Ralph Benajah Potter, Jr., who retired in July 2003, began teaching at HDS in 1965.  He is an ordained Presbyterian minister and the author of the book War and Moral Discourse and assorted scholarly articles. He is a founding fellow of the Hastings Center for Bioethics and is a member of the American Academy of Religion, the Society for Christian Ethics, Societe Europeene de Culture, the Society for Values in Higher Education, and, at Harvard, the Senior Common Room of Lowell House. His 1997 HDS Convocation Address was titled "Moralists, Maxims and Formation for Ministry." 

Source:http://www.hds.harvard.edu/faculty/em/potter.html

Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

Definition -Establishing facts

Values -Justification

Loyalties

Principles→

↓ ↑

Four Dimensions of Moral Analysis

Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

Use of Ethical Principles

Definition Problem

Values Principles

Loyalties

No conclusion can be morally justified without a clear demonstration that an ethical principle shaped the final decision.

What Actually Happens What Ought to Happen

Descriptive Normative

SITUATION

Teacher introduces an app that leaks data to 3rd party

VALUES / JUSTIFICATION

Students are motivated and learn better

Teacher trust 3rd party company

PRINCIPLE

Respect for Individual Integrity

Accountability of industry

LOYALTIES

To the learners. To app provider

To the teacher and the results

JUDGEMENT

The school has to inform better about digital learning practices and support

transparency

Potter Box applied to a school case I

Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

Potter Box' 4 steps

Empirical Definition

Identifying Values Appeal to Ethical Principles

Choosing Loyalties

Particular Judgement or Policy

Sociological

Immediate

External

Philosophical

Reflective

Internal both positive & negative

Feedback

virtue, duty, utility, rights, love

Facts

Ethical questions & dilemmas

• Does the administration let students know their academic behaviors are being tracked?

• What and how much information should be provided to the student?

• How much information does the institution give the teachers?

• Does the institution provide a calculated probability of success or just a classification of success (e.g., above average, average, below average)?

Ethical questions & dilemmas

• How should teachers react to the data? Should the teacher contact the student? Will the data influence perceptions of the student and the grading of assignments?

• What amount of resources should the institution invest in students who are unlikely to succeed in a course?

• What obligation does the student have to seek assistance?

From: Willis, J. E., III, Campbell, J., & Pistilli, M. (2013). Ethics, big data, and analytics: A model for application.

More questions

• What are the dangers in learning analytics?

• Is “raw data” an oxymoron?

• Should students be allowed to opt-out of having their personal digital footprints harvested and analysed?

• To what extent should students have access to the content of their digital dossiers, who have access to these dossiers, and what it is used for?

• How complete and permanent a picture do our data provide about students?

More questions• To what extent do we provide students the

option to update their digital dossiers and provide extra (possibly qualitative) data?

• Do students have the right to request that their digital dossiers be deleted on graduation?

• If we outsource the collection (and analysis) of student digital data to companies, do students need to give consent? [Who owns a student’s data?]

• Is bigger data sets always better or provide more complete pictures?

• What responsibility comes with ‘knowing’?

“Ethical questions and dilemmas of Learning Analytics ” workshop facilitated by Tore Hoel, Oslo and Akershus University College of Applied Sciences, was held at eConferece, Belgrade, 23 September 2014.

Presentation in co-operation with Henri Pirkkalainen & Kati Clements (University of Jyväskylä), and Thomas Richter, Thomas Kretschmer & Christian M. Stracke (University of Duisburg-Essen).

For further information: tore.hoel@hioa.no @tore

This work was undertaken as part of the LACE Project and Open Discovery Space project, both projects supported by the European Commission

These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.

www.laceproject.eu@laceproject

opendiscoveryspace.eu