Learning Outcomes – from Theory to Practice

8/13/2019 Learning Outcomes from Theory to Practice

1/44


2/44

Learning Outcomes fromtheory to practice

Workshop within the frame of the projectINARM

Veronika Nitsche (WUS Austria)

[email protected]
mailto:[email protected]://www.wus-austria.org/mailto:[email protected]://www.wus-austria.org/http://www.wus-austria.org/http://www.wus-austria.org/http://www.wus-austria.org/mailto:[email protected]:[email protected]:[email protected]


3/44

WP 4Learning outcomes for MA and BA

Duration: 9 months (month 14-22)

Lead partner: KNU

Role of EU partners:(according to the proposal)

The EU partners will participate in the development of

the learning outcomes for BA and MA, organizepeer-review of the learning outcomes, feedback onthe methodology for LO development and feedbackon the national versions of the LOs. The EU partnerswill travel to the PCU to jointly develop the LOs.


4/44

Description of WP 4 (I)

4.1 Intended learning outcomes for MA and BA: Thenext step of the project would be to design and developintended LOs drawing on the SQFs for informatics and

management. The academic staff involved in the SQFDevelopment Committee will embark on the developmentof LOs drawing on the Tuning methodology. []

4.2 Peer-review by EU partners: To ensure the LOs are

comparable with the European ones peer-review by EUpartners and external experts will be undertaken. Drawingon the peer-review results, the learning outcomes for BAand MA in informatics and management will berevisedand refined. []


5/44

Description of WP 4 (II)

4.3 Methodology of developing LOs: To ensure widerapplication of the methodology, guidelines on the LOdevelopmentwill be developed. The guidelines will be

translated into native languages and will be produced inArmenian, Russian, Ukrainian and English.

4.4 Adapted versions of the LOs for MA and BA:Drawing on the general versions of the learning outcomes

for BA and MA in informatics and management the PCuniversities will embark on the adaptation to the localneeds. The adapted versions will be circulated among thestakeholders for feedback, the LOs will be refinedandprepared for publications.


6/44

Objectives for this Workshop

It is the overall objective for this session to

- generate a common understanding on how touse LO,

- share experiences, and

- (if applicable) to decide on how to proceed inWP 4.


7/44

Session Outline

The bigger picture theBologna ProcessRecalling the key

features of learningoutcomesWhere and how to uselearning outcomes

Learning outcomes andassessmentHow to proceed in WP 4


8/44

Bologna process - shift in

paradigm:

from input-oriented to output-oriented

What do you associate with the Bologna Process?


9/44

The bigger picture (I)

The Bologna Process, launched with theBologna Declaration of 1999, is one of themain voluntary processes at Europeanlevel; it is today implemented in 47 states,which define theEuropean Higher

Education Area (EHEA).


10/44

The bigger picture (II)

The three overarching objectives of the

Bologna process : introduction of thethree cycle system(bachelor/master/doctorate), qualityassurance and recognitionof

qualifications and periods of study.


11/44

The bigger picture (III)

Detailed priorities for 2010-2020:

1. Ensuring a qualityhigher education system

2. Adopting a two- or three-cycle system of study (BA, MA, PhD)

3. Promoting the mobilityof students and academic and administrativestaff

4. Introducing a credit system(ECTS) for the assessment of studyperformance

5. The Recognition of levels: adopting a system of easily identifiable andcomparable levels

6. The Active involvementof higher education institutions, teachers andstudents in the Bologna Process and student participation in themanagement of higher education (> teaching and learning experience)

7. Promoting a European dimensionin higher education

8. Promoting the attractiveness of the European higher education area

9. Lifelong learning

10. A European higher education area and a European research area twopillars of a society based on knowledge


12/44

The bigger picture (IV)

Status quo:

HE across the EHEA countries looks

substantially different from 10 years ago Large difference in the speed of

implementation and varying levels ofcommittment

Lack of resources and expertise to guide thedomestic policy process in some countries

Short-term credit mobility > degree mobility(east>west)


13/44

The bigger picture (V)

Challenges/outlook:

LO is a controversial topic

Linking allocation of credits to studentworkload

No common understanding ofmodularisation

Integrating the social dimension

Recognition of prior learning


14/44

The bigger picture (VI)

LearningOutcomes/Competences

Teachingmethods/learning

experience

ECTS/curriculumdevelopment


15/44

Shift in paradigm:from input-oriented to output-

oriented

The princial question asked of the

student/graduate will no longer be

what did you do to obtain the degree?but

what can you do now that you have obtainedyour degree?

( Purser, Council of Europe, 2003)


16/44

FROM THEORY


17/44

Learning Outcomes

... are a statement of what a learner isexpected to know, understand, or able to doat the end of a learning process.


18/44

From the definition of LearningOutcomes we see

... an emphasis on the learner and an emphasis onthe learners ability to do something.

Focus on teaching aims and objectives anduse of terms like know,understand, be familiar

with.

Focus learning -outcomes: what wewant the student to beable to do - use of terms

like define, list, name,recall, analyse,calculate, design, etc.


19/44

How do I write Learning

Outcomes?

Writing Learning Outcomes is aProcess not an Event (Bloom)


20/44

Benjamin Bloom(1913 1999)

He looked on learning as a processwe buildupon our former learning to develop more

complex levels of understanding Carried out research in the development of

classification of levels of thinking behaviours inthe process of learning;

Worked on drawing up levels of these thinkingbehaviours from the simple recall of facts at thelowest level up to evaluation at the highest level.


21/44

Blooms Taxonomy ofEducational Objectives

Blooms taxonomy (1956) is a very useful aid towriting learning outcomes.

The taxonomy consists of a hierarchyof

increasingly complex processes which we wantstudents to acquire.

It provides a structurefor writing learningoutcomes

Blooms Taxonomy is frequently used by teachersin writing learning outcomes as it provides a readymade structure and list of verbs.

The taxonomy has been edited a few times sinceits establishing.


22/44

Bloom (1956) proposed that knowingis composed of six successive levels

arranged in a hierarchy.

6.Evaluation

5. Synthesis

4. Analysis

3. Application2. Comprehension

1. Knowledge


23/44

Blooms Classification ofCognitive Skills

Category Definition Related Behaviors

Knowledge recalling or remembering something withoutnecessarily understanding, using, or changing it

define, describe, identify, label, list, match, memorize,

point to, recall, select, state

Comprehension understanding something that has been

communicated without necessarily

alter, account for, annotate, calculate, change, convert,

group, explain, generalize, give examples, infer, interpret,

paraphrase, predict, review, summarize,

Application using a general concept to solve problems in a

particular situation; using learned material in new

and concrete situations

apply, adopt, collect, construct, demonstrate, discover,

illustrate, interview, make use of, manipulate, relate,

show, solve, use

Analysis breaking something down into its parts; may focus

on identification of parts or analysis of relationships

between parts, or recognition of organizational

principles

analyze, compare, contrast, diagram, differentiate, dissect,

distinguish, identify, illustrate, infer, outline, point out,

select, separate, sort, subdivide

Synthesis creating something new by putting parts of different

ideas together to make a whole.

blend, build, change, combine, compile, compose,

conceive, create, design, formulate, generate,

hypothesize, plan, predict, produce, reorder, revise, tell,

write

Evaluation

judging the value of material or methods as they

might be applied in a particular situation; judging

with the use of definite criteria

accept, appraise, assess, arbitrate, award, choose,

conclude, criticize, defend, evaluate, grade, judge,

prioritize, recommend, referee, reject, select, support


24/44

Some examples ...

At the end of the course students will be able to:- Identifyand consider ethical implications of scientific investigations.

- Differentiatebetween civil and criminal law

- Selectand employ sophisticated techniques for analysing theefficiencies of energy usage in complex industrial processes.

- Compareand contrast the different electronic business models- Proposesolutions to complex energy management problems both

verbally and in writing

- Predictthe effect of change of temperature on the position ofequilibrium.

What do you think of this categorisation anddescription by Bloom?


25/44

Benefits and potential problems

of Learning Outcomes?


26/44

The benefits of LearningOutcomes

Help to explain more clearly to students what isexpected of them and thus help to guide them in theirstudies.

Help teachers to focus more clearly on what exactly

they want students to achieve in terms of knowledgeand skills.

Help teachers to define the assessment criteria moreeffectively.

Help to provide guidance to employers about theknowledge and understanding possessed by graduatesof programmes.

Help to start discussion on teaching and learning inthird level institutions.


27/44

Potential problems withLearning Outcomes

Could limit learning if learning outcomes writtenwithin a very narrow framework lack ofintellectual challenge to learners.

Learning outcomes should not be reductionist butrather expansive and intended to promote thehigher order thinking skills.

Danger of assessment-driven curriculum if

learning outcomes too confined. Could give rise to confusion among students and

staff if guidelines not adhered to when drawingup learning outcomes, etc.


28/44

Learning Outcomes(summary)

Discussions at European and national level

highlight the need for common ground withrespect to LO this does not imply thattheir need to be a common approach todefining and using LO.


29/44

Where LO can be found

LearningOutcomes

Occupational

standards

Assessmentcriteria

QualificationsQualificationsframework

Curricula


30/44

TO PRACTICE


31/44

What is your experience/theexperience of your university

with learning outcomes?

Have learning outcomes being

introduced at your university?


32/44

Practical advice for writing

learning outcomes (I) Begin each learning outcome with an action verb.

Use only one verbper learning outcome.

Avoid vague terms like know, understand, learn, be familiar with,be exposed to, be acquainted with, and be aware of. These terms

are associated with teaching objectives rather than learningoutcomes.

Avoid complicated sentences. If necessary use more one thanone sentence to ensure clarity.

Ensure that the learning outcomes of the module relateto the

overall outcomes of the program. General recommendation: 5 8 learning outcomes per module.

The learning outcomes must be observableand measurable.

Ensure that the learning outcomes are capable of beingassessed.


33/44

Practical advice for writing

learning outcomes (II) Timescale: ask yourself if it is realistic to achieve the LO within

the time and resources available.

Assessment: How will you know if the student has achieved theselearning outcomes? Very broad LO may be difficult to assess

effectively. With very narrow LO, the list of learning outcomesmay be too long and detailed.

Before finalizing the learning outcomes, ask your colleagues andpossibly (former) students if the LO make sense to them.

When writing LO for students at levels beyond first year, try toavoid overloading the list with learning outcomes which are drawnfrom the bottom of Blooms taxonomy.

It is vital that learning outcomes are clearly written so that theyare understood by students, colleagues and external examiners.


34/44

Checklist for writing learning

outcomes for modules Have I begun each outcome with an active verb?

Are my outcomes SMART: specific, measurable,acceptable, realistic, time-limited?

Do all the outcomes fit within the aims andcontent of the module/program?

Bloom recommends to avoid terms like know,understand, learn, be familiar with, be exposed

to, be acquainted with, be aware of andappreciate


35/44

GROUP WORK

Writing Learning Outcomes


36/44

ASSESSMENT

-> STEPS involved in linking LearningOutcomes, Teaching and Learning Activities

and Assessment

Clearly define the learning outcomes.

Select teaching and learning methods that arelikely to ensure that the learning outcomes areachieved.

Choose a technique or techniques to assess the

achievement of the learning outcomes.

Assess the learning outcomes and check to seehow well they match with what was intended


37/44

Linking Learning Outcomesand Assessment Criteria

Learning outcomes imply the assessmentcriteria!

Assessment criteria may be developed from thelearning outcome or from the assessment task

but in either case they should relate to thelearning outcome

Assessment criteria should test, assess orrelate to the learning that is mentioned in

the learning outcome. e.g if you say studentsmust write something, you should not testthem orally.


38/44

What forms of assessment

(tests/exams)do you use?

When the student has passed eachcourse can you be sure that he/she

also achieved the Learning Outcome(s)of the programme?

(QANU)


39/44

Testing your assessment

- Statistics (test the tests)

- Four eyes- applied to construction of the test as well as theassessment

- External reviewers, not the teachers themselves butindependent reviewers

- Answer models that describe what kind of answer youexpect on an essay question, what elements should be partof it, which concepts should be named and how you weighthese aspects in the final mark

- Assessment forms to be used for theses, reports etc.

(QANU)


40/44

Sum up

- Back to the question: How to assess if the studentshave reached the LOs of the whole programme?

- What if all the courses are assessed by multiple choice

exams?

- What if all the courses ask the students to give an oralpresentation and never a paper or another kind ofwritten report?

- How to assess group work?

(QANU)


41/44

Sum up


42/44

Further reading

Jenny Moon, Exeter University: Linking Levels, LearningOutcomes and Assessment Criteria.

http://www.aic.lv/bolona/Bologna/Bol_semin/Edinburgh/J_Moon_backgrP.pdf

Kennedy, D. et.al.: Writing and Using LearningOutcomes: a Practical Guide.http://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdf

John Biggs: Aligning teaching and assessing to courseobjectives. Teaching and Learning in HigherEducation: New Trends and Innovations. University ofAveiro, 2003
http://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdfhttp://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdfhttp://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdfhttp://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdfhttp://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdfhttp://www.tcd.ie/vpcao/academicdevelopment/assets/pdf/Kennedy_Writing_and_Using_Learning_Outcomes.pdf


43/44

Next steps in WP 4?

4.1 Intended learning outcomes for MA and BA: The nextstep of the project would be to design and developintended LOs drawing on the SQFs for informatics andmanagement.

4.2 Peer-review by EU partners

4.3 Methodology of developing LOs: To ensure widerapplication of the methodology, guidelines on the LOdevelopmentwill be developed.

4.4 Adapted versions of the LOs for MA and BA: PCuniversities will adapt the LO to the local needs. The adaptedversions will be circulated among the stakeholders forfeedback, the LOs will be refinedand prepared forpublications.


44/44

Thank you for your attention!

Documents

Learning Outcomes – from Theory to Practice