Assessment Matters for Diversity

Ginny Saich

DAICE

Overview

• The H.E. Context for Assessment

• The Stirling Context for Assessment

• What is assessment? - What are its characteristics?

• Why assess? - What is its purpose?

• What is the effect of assessment?

• How do we assess?

• What are common problems with assessment?

• What trends are apparent in assessment?

• How do we grade?

• What are the characteristics of good feedback?

• Good practice in assessment

H.E. Context of Assessment

Recent legislation including: the disability discrimination act (DDA Part IV) the race relations amendment act places an onus on university’s to embrace inclusive practices (including for learning and teaching)

Assessment is one of the Quality Assurance Agency’s (QAA) enhancement themes for 2003/4.

Implications?

Concerns about plagiarism with increasing prevalence of online resources

Stirling Context of AssessmentThe University of Stirling’s Learning, Teaching and Quality Enhancement Strategy: 2003-2005 has the following action items relating to assessment for implementation during 2003/4:

Instigate a critical review of assessment practices with reference to the relevant section of the QAA Code of Practice and in light of the Quality Enhancement engagement of 2003-4

Investigate and report on the implications for Learning and Teaching of Disability, Special Educational Needs, and Racial Equality legislation, with recommendations for action

Monitor progress on embedding the accessible curriculum

Report on use of diagnostic testing to identify the needs of students

Report on effectiveness of the Special Academic Arrangements Panel after its first year in operation

Report on use of JISC Plagiarism Detection Service pilot

Preventing and Detecting Plagiarism

Preventing and Detecting Plagiarism

Prevention is more effective than detection!

Good Practice Guide from Oxford Brookes contains recommendations on preventing plagiarism

(see http://www.northumbria.ac.uk/jiscpas - web site of the JISC Plagiarism Advisory Service)

JISC Plagiarism Detection Service - detects plagiarism of online resources (and collusion) - to be piloted at Stirling in Spring 2004

Why Assess?Why

Assess?

TheStudent• to pass or fail

• to grade or mark

• to allow to proceed

• to licence • to predict success

• to select for further training

• to detect strengths & weaknesses

• to motivate • to give feedback

• for public relations

• Licence to proceed

• Ranking order

• Improving learning

• Licence to proceed

• Ranking order

• Improving learning

• to meet expectations

• to indicate performance • to help learning

Purpose of Assessment

For Whom? Why?

The student feedback

Other students relative position

Tutors progression

Employers comparative skills

Universityadministration

student outcomes

Government bodies quality indicator

If we wish to discover the truth about an educational system, we must look into its assessment procedure.

What student qualities and achievements are actively valued

and rewarded by the system?

Rowntree (1987) Assessing StudentsLondon: Kogan Page, 2nd edition, p.1

What is Valued?

Assessment influences student learning eg.

Surface learning is induced by:-Heavy workloads, assessment by unseen exam alone,

multiple choice questions that test only recall.

Deep learning is induced by:-Reasonable workloads, some choice, a variety of

assessment tasks, project work, multiple choice questions that test understanding

(Marton and Säljö, 1976: surface and deep learning)

The Effects of Assessment on Learning

The kinds of assessments set are important!The kinds of assessments set are important!

If you want to change the mode of learning then change the

assessment procedure.

Brown, G., Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge

Assessment is …..

a “systematic basis for making inferences about the learning and development of students …

the process of defining, selecting, designing, collecting, analysing, interpreting and using information to increase student’s learning and development”(Erwin, 1991: 15)

What is Assessment?What is Assessment?What is Assessment?What is Assessment?

SampleSample

Draw inferenceDraw inference

Estimate worthEstimate worth

Formative assessmentFormative assessment Summative assessmentSummative assessment

FeedbackFeedback

??

Student Experiences of Assessment

“Continuous assessment meant that I knew how I was getting on, rather then waiting until the end of the session, then failing because I didn’t know what was required.”

“I failed the first one but I don’t really know why. All I got were a lot of crosses and a few ticks and some exclamation marks.”

Functions of AssessmentFunction Description Examples

Formativeassessment - helpsstudents learn

offered as feedback oradvice to students - may ormay not contribute markstowards final results. Mayfocus on particular skills orknowledge within themodule.

Summativeassessment -measures howmuch a studenthas learned

always graded andcontributes towards creditfor a course. Coversrequired skills andknowledge for the module.

In practice.......In practice.......In practice.......In practice.......

So-called formative assessment is often So-called formative assessment is often

multiple summative assessmentmultiple summative assessment

- and little feedback is given.- and little feedback is given.

Characteristics of AssessmentCharacteristics of Assessment

Efficiency - saves you time

Validity - effectively tests what you intended ie. fitting your learning outcomes etc.

Reliability - provides consistency

Enablement - develops understanding and expertise

The difficulty lies in achieving all these characteristics together.

What is effective assessment?

What is effective assessment?

• It assesses what is taught and learnt

• It emphasises deep learning

• It measures skills and transfer

• It is efficient for lecturers

• It is a reasonable workload for students

Do you have an assessment strategy?Does your department?

Reliability of Assessment

Reliable assessment strategy same results if repeated or if marked by a different marker

Even apparently objective assessment may have a highly subjective component.

Objectivity 1

A candidate was required, for 10 marks, to multiply 4567 by 23. Below is the candidate’s answer compared with the correct answer.

Candidate 4567 Correct: 4567

x 23 x 23

13801 13701

91330 91340

105131 105041

What mark would you award?

Objectivity 2Four markers comments:

1. This candidate cannot do simple long multiplication. Mark = 0

2. Layout and general procedure correct. Accuracy missing. Mark = 4

3. Errors in multiplication, but most correct. Addition correct. Mark = 5

4. 12 out of 14 arithmetical operations carried out correctly. Mark = 8

Validity of Assessment

A valid assessment is trustworthy

tests what it is supposed to test.

A test may be technically highly reliable but not valid eg. a test for critical thinking that asks questions requiring only memorisation

Inclusive Assessment

Planning for inclusiveness in assessment means: Ensuring that one group of students are not unduly advantaged or

disadvantaged by the selected mode of assessment. Ensuring that all students have the means to acquire the assumed

knowledge and skill to successfully complete the course. Being clear about the essential learning outcomes to be attained

and willing to consider alternative means for student to demonstrate the attainment of them

Stirling has a developing set of policies and guidelines for supporting students with disabilities. (Further information is available from departmental disability officers or the University’s disability officer, Catriona Mowatt in SISS.)

Levels of Objectives (Bloom’s Taxonomy)

1. Knowledge

2. Comprehension

3. Application

4. Analysis

5. Synthesis

6. Evaluation

Assessment TermsAssessment TermsMethodsMethodsEssays Problem sheets Unseen prose Lab reports Presentations Projects Group projects Posters Work-based learningWritten exams VivasMCQ'sMRI's

InstrumentsInstruments• Implicit criteria• Global• Explicit criteria• Criteria reference grading• Specific criteria• Marking scheme• Dimensions• Rating schedules• Checklists

SourcesSourcesLecturerOther tutorsPostgraduate tutorsDemonstratorsStudent selfStudent peersEmployersMentorsExternal examiner

Criterion referenced Assessment

Purpose Description Examples

- finds out if acertain criterionon a skill, ability,or behaviour hasbeen reached

- finds out what ischaracteristic ofthe individual

Establishes standards(criteria) for passing orfailing. The goal is foreveryone to meet anestablished standard.Once the criterion isreached the test ispassed.

Theoretically allstudents could get firstsor all could fail.

Norm referenced AssessmentPurpose Description Examples

- comparesstudents inrelation to theirpeers

- finds out astudent's abilityin relation toothers

Allocates marks accordingto how well individuals doin relation to others takingthe test. Grading on thenormal curve is an example.

Results may indicate howgood a student is in relationto peers in that cohort, butnot at what the student iscompetent.

Assessment methods: what are they good for?Assessment methods:

what are they good for?EssaysEssays - for analysis, synthesis and creative perspectives- for analysis, synthesis and creative perspectives

MCQsMCQs - to sample knowledge and understanding of principles- to sample knowledge and understanding of principles

ProblemsProblems - to sample problem solving skills!- to sample problem solving skills!

PracticalsPracticals - to sample practical skills!- to sample practical skills!

Lab sheetsLab sheets - to sample report writing, understanding of procedures,- to sample report writing, understanding of procedures, data analysis and interpretationdata analysis and interpretation

Projects Projects - as for lab sheets plus deeper understanding,- as for lab sheets plus deeper understanding, literature review (?), reflective learning (?), literature review (?), reflective learning (?), time managementtime management

LearningLearning - reflective and experiential learning - reflective and experiential learningLogs Logs

Common WeaknessesCommon Weaknesses

• Overload of staff and studentsOverload of staff and students• Too many assignments set with the same deadlineToo many assignments set with the same deadline• Lack of alignment between assessment and teachingLack of alignment between assessment and teaching methods and/or learning outcomes and course aimsmethods and/or learning outcomes and course aims• Poor assessment design Poor assessment design plagiarism/surface learning plagiarism/surface learning

Wide variations Wide variations • in unit demands in unit demands

• in marking across unitsin marking across units• in marking criteria/guidancein marking criteria/guidance

Inadequate feedback (quality/quantity/timeliness) to studentsInadequate feedback (quality/quantity/timeliness) to students

AligningAssessment

AligningAssessment

FeedbackFeedback

CriteriaCriteria

MarkingMarking

AimsAims

Intended Learning OutcomesIntended Learning Outcomes

Methods of LearningMethods of Learning

Assessment methods and tasksAssessment methods and tasks

An emerging modelAn emerging modelAn emerging modelAn emerging model

FeedbackFeedback

Quality Management:• Standards• Benchmarking• Code of Practice• Prog. specification.

University:• Policy• Strategy

Professional Bodies

AimsAims

OutcomesOutcomes

Learning MethodsLearning Methods

MarkingMarking

Assessment Criteria Assessment Criteria

External Influences on Assessment

Trends in assessmentTrends in assessment

Tutor led Tests, Examinations

Student led

Peer marking,Peer assessmentPresentationsGroup workProjects, Group projects

Choice of assignments, dissertations

Student Centred

Marking

• Who does it?

• What points of reference are used?

• Types of assessment criteria

• Marking schemes

• Marking strategies

• Factors influencing marks awarded

Feedback

• Problems with feedback

• Characteristics of good feedback

• The feedback ‘sandwich’

• Providing feedback efficiently

• Care required…….

Good Practice in Assessment

Good Practice in Assessment

Consider:

1. What kinds of things do you want them to learn?

2. What learning opportunities will be provided?

3. What assessment tasks will be set?

4.What criteria will be used?