Students’ use of Web 2.0 tools for assessment in higher

education: What constitutes good practice?

An ALTC Priority Project 2009-2011

HERDSA Conference Workshop6 July 2010

Kathleen Gray & Celia Thompson

Workshop outline

1. The story behind this ALTC project

2. What’s YOUR interest in participating today?

3. What have we found out so far?

4. What things about Web 2.0 do we need to consider, to be sure of “good practice” when we use it to assess students?

[over to YOU – over a cuppa]

5. Our draft framework & pilot testing

6. YOUR feedback & following up your interests

Project team

Jenny Waycott (project manager), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne.

Celia Thompson, School of Languages and Linguistics, University of Melbourne.

Margaret Hamilton, School of Computer Science and IT, RMIT University.

Joan Richardson, School of Business Information Technology, RMIT University.

Kathleen Gray (project leader), Faculty of Medicine / Department of Information Systems, University of Melbourne.

Rosemary Clerehan, Faculty of Medicine, Nursing and Health Sciences, Monash University.

Judithe Sheard, Faculty of Information Technology, Monash University.

Project background

This slide showed a spoof graph by dante88, retrieved from http://graphjam.files.wordpress.com/2009/05/song-chart-

memes-research-paper.jpgIt was a pie chart entitled “Composition of my 25-page research

paper” showing that 50% of the content of the paper was “Wikipedia information disguised as something else”

The future of scientific & scholarly communication

Chodorow

(2000, p.91)

Chodorow, S. (2000). Scholarship & scholarly communication in the electronic age. Educause Review, 35(1), 86-92. http://www.educause.edu/ir/library/pdf/ERM001B.pdf

• “the form and substance of scholarly communications will change over time, so that it will be difficult to trace the historical flow of the work”

• “a free-flowing stream of scholarly discourse will reduce the role of scholarly authority in the progress of research”

• “the roles of individual authors will be obscured in the electronic environment”

Why and how might we want our students to use the social web to demonstrate their learning?

• This slide showed the image “Social media landscape” published by Fred Cavazza, 9 June 2008, at FredCavazza.net. Retrieved 20 June 2010 from http://www.fredcavazza.net/2008/06/09/social-media-landscape/

Implications for university learning, teaching & assessment?

O’Reilly & Battelle

(2009, p. 2)

O’Reilly, T., & Battelle, J. (2009). Web Squared: Web 2.0 Five Years On. Special Report for the Web 2.0 Summit, 20-22 October , San Francisco CA. http://assets.en.oreilly.com/1/event/28/web2009_websquared-whitepaper.pdf

“One of the fundamental ideas underlying Web 2.0 [is] that successful network applications are systems for harnessing collective intelligence ... a large group of people can create a collective work whose value far exceeds that provided by any of the individual participants”

Implications for university learning, teaching & assessment?

Kakutani

(2010,

paras 13-14)

Kakutani, M. (2010, 17 March). Texts without context. [Book review]. New York Times. http://www.nytimes.com/2010/03/21/books/21mash.html?ref=books

“jump to the summary, the video clip, the sound bite — never mind if context and nuance are lost in the process; never mind if it’s our emotions, more than our sense of reason, that are engaged; never mind if statements haven’t been properly vetted and sourced”

“tweet and text one another during plays and movies, forming judgments before seeing the arc of the entire work”

“power-search for nuggets of information that might support their theses, saving them the time of wading through stacks of material that might prove marginal but that might have also prompted them to reconsider or refine their original thinking”

Use of Web 2.0 in university learning and teaching ...lots to talk about

‘Web 2.0 and emerging technologies in online learning’

‘Analysis of 10 popular Web 2.0 tools used in higher education’

‘Facilitating new forms of discourse for learning and teaching: harnessing the power of Web 2.0 practices ’

‘The changing space of research: Web 2.0 and the integration of research and writing environments’

‘Can Web 2.0 and social software help transform how we measure quality in teaching, learning, and research?’

What’s YOUR interest in participating today?Please tell us

your name, organisational affiliation, roles / responsibilities, etc.

What are your thoughts at this stage about using Web 2.0 to assess student learning in higher education?

e.g.“The assessment of student web 2.0 activities is

............. for university learning and teaching”.

Project aims

Assist universities when assessment uses Web 2.0 by:

1. Investigating current practitioners> Survey and interview teaching academics (September 2009)

2. Identifying principles of good practice > Advisory group and national roundtable (November 2009)

3. Iterating practicable guidelines > Pilot testing in learning and teaching settings (February to June 2010)

4. Producing and sharing resources (July 2010 ff)

Current Web 2.0 assessment practices

Some preliminary findings

about how, where and why

60 Australian academics

are using Web 2.0

for assessment of student learning

that is more than just formative

(i.e. that earns marks in a subject)

Current Web 2.0 assessment practices: How

Type of Web 2.0 activity Number of responses

Wiki writing 32

Blogging/microblogging 31

Social networking 17

Audio/video podcasting 16

Virtual world activities 12

Social bookmarking 11

Current Web 2.0 assessment practices: How

Where students complete assignment Number of

responses

Off campus elsewhere (e.g., at home during

independent study time)52

On campus but out of class 25

On campus in class 16

Off campus while undertaking fieldwork or

workplace learning7

Current Web 2.0 assessment practices: How

How much the assignment is

worth

Number of responses

01-10% 7

11-20% 11

21-30% 9

31-40% 6

41-50% 9

51-60% 2

61-70% 0

71-80% 3

81-90% 2

91-100% 4

Current Web 2.0 assessment practices: Where

Field of Study Number of respondents

Humanities / Society & Culture16

Education15

Information Technology11

Medicine & Health9

Management & Commerce6

Other 3

Current Web 2.0 assessment practices: Where

Number of students

enrolled in unit

Number of responses

Less than 50 21

50-100 10

101-200 9

More than 200 7

Current Web 2.0 assessment practices: Where

Level of study Number of responses

Bachelor or honours degree 35

Postgraduate coursework degree 16

Current Web 2.0 assessment practices: Why

Intended learning outcomes Number of

responses

Generic or graduate skills or attributes 35

Specialised knowledge or skills required in a

discipline or profession29

Foundation knowledge or skills preparatory to

a discipline or profession28

Current Web 2.0 assessment practices

Would you like to comment on

any of the other survey data

that are in your handout?

What would “good practice” look like ... ?

… when university students are asked to demonstrate their learning using Web 2.0 activities / authoring tools / attitudes to content production and consumption?

Some things to think about: What Web 2.0 allows / enables The assignment, from go to woeAcademic policies that pertain

Small groups + refreshments as required + report back

What would good practice look like? Affordances

Affordances checklist ...

What is an appropriate fit between what assessment is trying to achieve and what Web 2.0 can do?

• Open publishing

• Communication styles and texts

• Personal identity and experience

• Co-creation, collaboration, crowdsourcing

• Content management

What would good practice look like? Affordances

Open publishing:

• Student work can be made easily accessible to an audience of peers for mutual benefit including reviewing and rating.

• Review and assessment of student work from outside the university can be invited or anticipated.

What would good practice look like? Affordances

Communication styles & texts

• Web 2.0 assignments can involve frequent short pieces of work employing conversational language and combining audio, video, images & text.

• Feedback can be exchanged rapidly, using rating or ranking systems, informal rejoinders, audio, video, images, icons.

What would good practice look like? Affordances

Personal identity & experience: • Students’ online identity can be

different from the student who is recognisable in class.

• Students’ social or cultural experiences of web authoring can influence the work they produce for assessment.

• Reflection and self-reflection about the idea of identity are prompted by the need to create and express an online identity.

What would good practice look like? Affordances

Co-creation, collaboration, crowdsourcing:

• Group work can scale between a small closed group and a large free-to-join learning community

• Individual contributions to group work can (sometimes) be distinguished.

• Groups can work on large, complex tasks.

What would good practice look like? Affordances

Content management

• Students’ assessable work may consist of remixing web content from diverse sources.

• Students’ assessable work may be posted on several host sites. Work posted on one site may be syndicated by others and tracked back.

• Students can control the content they produce for assessment in accordance with terms of service, end user agreements or other governance policies of host sites.

What would good practice look like? Processes

Processes checklist ...

How do teachers use Web 2.0 to support student, self- and organisational learning throughout the cycle of activities involved in the assignment?

Design

Implement

MarkFeedback

Review

What staff have said about ...

Pick out a quote that interests you

from the checklist on “stages in the assessment cycle”:

Designing the assignment

Implementing the assignment

Marking the assignment

Giving results and feedback to students

Reviewing how well the assignment works

What would good practice look like? Policies

Policies checklist ...

How can assessment using Web 2.0 be made safe and fair for students and staff?

• disability

• access to IT services or equipment

• appropriate conduct

• identity and privacy

• academic honesty and integrity

• special consideration

• moral rights and copyright

Surveyed staff were not always sure whether they were clearly observing assessment policies: some examples

Policy area % Not sure

Copies of students’ marked work are available if there is a need to deal with appeals/complaints

20

This assignment encourages academic honesty and integrity 20

Students’ identity and privacy in online environments are safeguarded

20

Students are provided with timely feedback on marked work for this assignment

20

This assignment provides for equitable assessment for students with a disability

23

Students’ moral right and copyright in work they produce are protected

27

Students whose work shows evidence of cheating or misconduct are formally disciplined

28

What can (and can’t) be done in practice in subject L&T settings?

Draft guidelines pilot-tested

for practicability

in 17 subjects

at 5 universities

in Victoria

during Sem 1, 2010

BloggingCinema Studies / Criminal Law

Cultural Studies / Media Studies

Social bookmarking Education

Social networking Languages

Video sharing Business / Economics

Photo sharing Communication Design

Virtual worlds Languages

Wiki writing

Accounting / Education

Information Technology

Languages / Science

Combined

Web 2.0 tools

Information Management

Information Technology

Sharing project progress

Moodle: www.groups.edna.edu.au/course/view.php?id=2146

Blog: http://web2assessment.blogspot.com

Bookmarks: www.citeulike.org/tag/assessment20

Webinar: www.transformingassessment.com/events_26_may_2010.php

Workshops 2010-11 @ HERDSA, ATN Assessment, ASCILITE, ACE

Papers: • Gray, K., Thompson, C., Clerehan, R., Sheard, J., & Hamilton, M. (2008). Web 2.0

authorship: Issues of referencing and citation for academic integrity. The Internet and Higher Education, 11(2), 112-118.

• Gray, K., Thompson, C., Sheard, J., Clerehan, R., & Hamilton, M. (2010). Students as web 2.0 authors: Implications for assessment design and conduct. Australasian Journal of Educational Technology, 26(1), 105-122.

Acknowledgements

Project Advisory Group• Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Buckworth,

Denise Chalmers, Geoffrey Crisp, Leitha Delves, Bobby Elliott, Jacqui Ewart, Glenn Finger, Tom Franklin, Merrilyn Goos, Scott Grant, Ashley Holmes, Christopher Hughes, David Jones, Marj Kibby, Adrian Kirkwood, Mark Lee, Catherine McLoughlin, Beverley Oliver, Kaz Ross, Alison Ruth, Royce Sadler, Mary Simpson, Arthur Winzenried, Katina Zammit, Lynette Zeeng.

Project Reference Group• Michael Abulencia, Robyn Benson, John Benwell, Marsha Berry, Marilys Guillemin, Laura

Harris, Deborah Jones, Gregor Kennedy, Shaun Khoo, George Kotsanas, Lauren O’Dwyer, Jason Patten, Emma Read, Julianne Reid, Gordon Sanson, Cristina Varsavsky.

Project Pilot-testing Group• Matthew Absolom, Anne Davies, Cathy Farrell, Scott Grant, Terry Hallahan, Michael

Henderson, John Hurst, Ramon Lobato, Warren McKeown, Michael Nott, Kerry Pantzopoulos, Michele Ruyters, Michael Smith, Sandra Smith, Robyn Spence-Brown, Elizabeth Stewart, John Terrell, Jenny Weight, Lynette Zeeng

ALTC Support for this project has been provided by the Australian Learning and Teaching Council Ltd. (www.altc.edu.au), an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this presentation do not necessarily reflect the views of the Australian Learning and Teaching Council, or the views of individual contributors apart from the project team.

Your comments, questions, feedback, follow-up?

• This slide showed a web page “Learning Web2.0 tools and applications” published by Go2web20.net and retrieved 2 July 2010 from http://www.go2web20.net/#learning