#Opened16 Conference Presentation

Open. But not for criticism?

Dr Vivien Rolfe BSc PhD PFHEANational Teaching Fellow

@vivienrolfe

University of the West of England, Bristol, UK

I am calling for a (radical?) pedagogy caucus, a core, self-identified group committed to placing pedagogy at the center of the OpenEd movement.(Robin De Rosa)

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

This is important, because a community of practice is a shared history of that practice.http://followersoftheapocalyp.se/keep-the-fire-notes-on-my-opened15-presentation/

Two interesting aspects of OpenEd 2015…

The trust and willingness to be open to criticism within the space of the community…

…but a perceived lack of criticality within it.

Critical, criticism, criticality…..

At the heart of innovation is the reuse of knowledge and ideas and ability to critically reflect and reject old solutions (Kuhn 1970).

Criticality can be viewed as a pedagogical outcome with three interrelated elements: critical knowledge, critical thinking skills and critical spirit (James 2001).

Are we being critical? Where arewe being critical? Is this enough?

Meta-research approaches

Citation network analysisAnalysis of inter-disciplinarityHistory and legacy

What are we publishing and where, to theorise about openness?

Are we critical in our writing and thinking?

Sociospatial and historical aspects?

Retrieve a non-biased sample of papers through quasi-systematic approach: #search for ‘open*’ ‘student’ ‘learning outcomes’

Analysis of article outcomesCitation analysisOther biases?

(References – see end)

Retrieval(Pubmed MeSH + ERIC Thesaurus+open keyword searching)

Results186 (Pubmed) + 627 (ERIC) retrieved

REVIEW OF TITLES AND ABSTRACTS TO EXCLUDE

● Reports and other article types● Interventions that weren’t “open”● Those not an evaluation of learning

(but satisfaction)

REVIEW OF FULL PAPERS● Interventions that weren’t “open”● Not an evaluation of learning

53 articles

5 articles did evaluate the impact of open education on learning and learner outcomes

1. There are few evaluations of the impact of open on learning.

2. A good proportion of abstracts didn’t contain the detail to judge the quality and content of the paper.

Search results

1. Publication bias evident (within small sample).

2. Some evidence of critical reflection and writing.

3. Citation bias - introductions contained affirmatory articles.

4. Ball (2015) noted in his science study, 2.4% citations in papers were negative.

All 5 presented positive findings as primary and secondary outcome e.g. learning gain, test results

All stated limitations of their methodology and approaches.

Of 62 citations within the introduction sections, 6 were of a negative critical context (9%).

5 articles did evaluate the impact of open education on learning and learner outcomes

Open s

ource

Learn

ing VLE

Open a

cces

s

Techn

ology

evalu

ation

MOOC

OER/OCW

Open t

extbo

oks

Open u

nivers

ity

Event

evalu

ation

Open c

ourse

Open r

espo

nse q

uesti

ons

Openn

ess

0

1

2

3

4

5

6

7

8

9

10

Not evaluation of learning / open (i.e.

satisfaction)

Not open Case study (not evaluation)

0

5

10

15

20

25

30

35

Categories of articles retrieved / exclusion criteriaN

umbe

rs o

f art

icle

s re

trie

ved

Weller 2016 – OER Knowledge Cloud analysis = Policy, Practitioner, OER in developing nations, Pedagogy, Open data/practice/access

From the 3 Rs to the 5 Rs

1970-74 1975-79 1980-84 1985-89 1990-94 1995-99 2000-04 2005-09 2010-15 20160

20

40

60

80

100

120

140

160

180

Number of articles retrieved over time

Num

bers

of a

rticl

es re

triev

ed

Open universities = open as in ‘entry’

Bulk of papers open schools = open pedagogy and classrooms

Open = licensing / technology / re

sources / textbooks

“Open education is used here to designate a general approach to teaching and learning which presumes the child's right and competence to make important decisions; views the teacher more as a facilitator of learning than a transmitter of knowledge, and abundant alternatives and choice for students”.Barth 1971Well cited

“Open education movement (Katz 1972)…commitment to humanistic values including self-determination, freedom of children and aesthetic appreciation”.Cited once in 2013 and not since 1984

“While the open education movements and educational technology are often seen as mutually hostile, the challenge in education for the future is to find ways to develop the full range of each individual’s capacities.”Resnick 1972Cited once in 1996 and not since 1970’s

Laura Czerniewicz talks about global knowledge inequalities – financial, social, xxxhttp://www.slideshare.net/laura_Cz/laura-czerniewicz-open-repositories-conference-2016-dublin

= Paper

= Citation

= Paper

= Citation

Does the discipline lack hetergeneity, (it showed confirmatory bias of citing many papers from that journal) and does this represent egotism of some scholarly work? Lawani 1982

Discussion

1. Publishing and citation creates a footprint and is our legacy.

2. There was more theorising in the 1970’s than today. This history goes largely unrecognised despite parallels with our present humanistic approaches and shared values.

3. The open education community is critical within itself but not of itself. There are few robust evaluations within my chosen topic, and biases not dissimilar to research in general.

4. This small sample may suggest an element of geographical inequality, and subject differences in citation hetergeneity.

5. We need evidence and legacy for creating persuasive arguments.

BHAGs?

Shared spirit and ethos of openAltruistic communityLowering costsA successful movementTheoretizing openness

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

BHAGs?

Shared spirit and ethos of openAltruistic communityLowering costsA successful movementTheoretizing openness

Evaluations?

CriticalUnbiasedPublishedCited

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

BHAGs?

Shared spirit and ethos of openAltruistic communityLowering costsA successful movementTheoretizing openness

Evaluations?

CriticalUnbiasedPublishedCited

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

Criticality

Shared history

Acknowledged and built-upon

BHAGs?

Shared spirit and ethos of openAltruistic communityLowering costsA successful movementTheoretizing openness

Evaluations?

CriticalUnbiasedPublishedCited

Criticality

Shared history

Acknowledged and built-upon

Community

Critical spirit

Multidisciplinary bridging communities

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

BHAGs?

Shared spirit and ethos of openAltruistic communityLowering costsA successful movementTheoretizing openness

Evaluations?

CriticalUnbiasedPublishedCited

Critical knowledgeCritical skillsCritical spirit

My sense of #OpenEd2015 is that there was widespread interest in ambitions beyond open textbooks but, lacking a clearly articulated ladder of ambition, there wasn’t a lot of focus on it. http://mfeldstein.com/is-open-education-a-movement/

Criticality

Shared history

Acknowledged and built-upon

Community

Critical spirit

Multidisciplinary bridging communities

So get thinking!!

Felix by @mdvfunes  https://www.flickr.com/photos/97994829@N03/29650137712 CC BY-NC-SA

Background

OpenEd2015 Archive. http://openedconference.org/2015/index.html%3Fp=368.html

Including:

Croon A (2015). Is that what we meant? http://adamcroom.com/2015/11/is-that-what-we-meant/De Rosa R (2015). Open textbooks. Ugh.http://robinderosa.net/uncategorized/open-textbooks-ugh/

Barth RS (1972). Open Education and the American School.Czerniewicz L (2016). Open Repositories Conference Dublin. Available: http://www.slideshare.net/laura_Cz/laura-czerniewicz-open-repositories-conference-2016-dublinLawani, Stephen M. "On the heterogeneity and classification of author self-citations." Journal of the American society for Information Science 33.5 (1982): 281.James N (2001). Criticality, Critical Pedagogy and a Critical Legal Education. WG Hart 2001 Legal Workshop.Resnick LB (1972). Open Education: Some Tasks for Technology." Educational Technology 12(1), 70-76.Katz L G (1972). Research on Open Education: Problems and Issues.Kuhn T (1970). Scientific Revolutions (2nd. ed., Enlarged), Chicago: The University of Chicago Press.Weller M (2016). Different aspects of the emerging open education discipline. https://altc.alt.ac.uk/2016/sessions/different-aspects-of-the-emerging-open-education-discipline-1283/

Methods

Higgins JPT and Green S eds (2008). Cochrane handbook for systematic reviews of interventions. Vol. 5. Chichester: Wiley-Blackwell. (Guide to systematic searching)

Gasevic D et al (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning 15(5). (Details of content and network analysis)

Rodrigues V (2013). Publication and reporting biases and how they impact publication of research. Editage.com. Available: http://www.editage.com/insights/publication-and-reporting-biases-and-how-they-impact-publication-of-research (Introduction to biases in publishing)

Ioannidis JPA, et al (2015). Meta-research: evaluation and improvement of research methods and practices. PLoS Biol 13(10). (Meta-research techniques for evaluating research practice).

EPPI Centre (2016). Reviewing tools: keyword strategy. Available: https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=184. (Toolkits for systematic reviews in education – keywords, study quality evaluation).

Ball P (2015). Science papers rarely cited in negative ways. Nature News.

Knoth P and Herrmannova D (2016). Semantometrics http://semantometrics.org/

Neylon C (2016) What constitutes research data? What is citation? https://rdmetrics.jiscinvolve.org/wp/2016/04/12/what-constitutes-research-data-what-is-citation/