Methods and results of the GEMS user survey - External evaluation

Apr 10, 2023 Jón Torfi Jónasson, Anne-Christin Tannhäuser, Andrea G. Dofradóttir

1

External Evaluation- Method and Results

of the GEMS user survey

Guidance for Educators, Mentors and StudentsFinal Conference

November 6th 2008

Jón Torfi Jónasson ([email protected]) and Anne-Christin Tannhäuser ([email protected])

Andrea G. Dofradóttir ([email protected])

Social Science Research InstituteUniversity of Iceland


2

Short note on external evaluations Timeline of evaluation process GEMS material user survey

Outline of the Presentation

Method Results Conclusions


3

Framework for Leonardo projects (Leonardo UK National Agency)

What should be evaluated? the output – what was done; and the process – how the output was achieved,

including management and transnational partnership 2 steps

Interim evaluation: formative Final evaluation: summative

Short note: external evaluations


4

Dec 2007 Partner survey

Timeline of GEMS evaluation activities

Jan 2008 Formative evaluation report I

Feb 2008 Partner interviews

Apr 2008 GEMS material user survey

Sep 2008 Working paper on results


5

GEMS material user survey

Receiving countryMentors

Students

Coordinators

Sending/Coordinating institution/organization

Sweden 3 4 1

Ester Mosesson Gymnasium in Sweden

Poland 4 2 1

Nowy Sacz School of Business in Poland

Belgium 8 10 1

Stella Maris Hotel and Catering School in Belgium

Iceland 4 4 1

Kópavogur Grammar School in Iceland

/ / 2 Iðan Fræðslusetur

Ireland 0 0 0

Southwest College in Ireland

TOTAL 19 20 6

Users work

sector

Hospitality/Tourism 6 4

Kitchen 11 14

Restaurant service 2 2

TOTAL 19 20

Participants


6

Pedagogical aspects

Organisational aspects

Economic aspects

Technological aspects i

Cultural aspects

GEMS material user survey Focus of the questionnaire

The POETiC framework


7

Pedagogical aspects

How well or how poorly did the students logbooks/GEMS material help you to communicate with...

5

15

63

25

26

25 30

55

5

0% 20% 40% 60% 80% 100%

... your students about their WBL

... your mentor about your WBL

me

nto

rsst

ud

en

ts

It helped a great deal

It helped somewhat

It helped a little

It did not help at all

I do not know

Results


8

ResultsPedagogical aspects

Do you agree or disagree with the following statement? The students logbooks/GEMS material helped me...

30

5

40

63

68

84

25

21

16

11

11

16

5

5

0% 20% 40% 60% 80% 100%

... to organise my learning at my workplacement.

... to organise my students' work placementperiod.

... to keep track with the learning of mystudents.

... to make the WBL more effective.

stu

den

tsm

en

tors

I strongly agree

I agree

I disagree

I strongly disagree


9

Pedagogical aspects

How effective or ineffective is the GEMS material in making learning outcomes more transparent?

17 67 17

0% 20% 40% 60% 80% 100%

coordinator

It is very effective

It is rather effective

It is slightly effective

Results


10

Organisational aspectsResults

How easy or how difficult was it for you to...

53

25

37

55

11

10

0

0,0 20,0 40,0 60,0 80,0 100,0

... fit the GEMS material into your workarrangements?

... integrate the student logbooks into yourwork placement?

men

tors

stud

ents

It was rather easy

It was rather difficult

It was very difficult


11

Economic aspectsResults

Do you agree with the following statement? Working with the GEMS material is too time-consuming.

5

10

26

55

53

30 5

511

0% 20% 40% 60% 80% 100%

mentors

students

I strongly agree

I agree

I disagree

I strongly disagree

I do not know


12

Economic aspectsResults

In your opinion, of how much or how little use is the GEMS material for making the students work-based learning more

efficient in your training sector?

5 84 5 5

0% 20% 40% 60% 80% 100%

mentors

It is very efficient

It is somewhat efficient

It is somewhat unefficient

I do not know


13

Cultural aspectsResults

How well or how poorly did the students logbook/ the GEMS material match with ...

5

5

74

50

16

35 10

5

0% 20% 40% 60% 80% 100%

... your working culture?

... what you were learning at your workplacement

men

tors

stud

ents

Very well

Rather well

Rather poorly

Very poorly


14

Future use of the GEMS materialResults

Are you planning to use the GEMS material in educating your future students?

63 5 5 26

0% 20% 40% 60% 80% 100%

mentors

I will use it to some extent

I am willing to use the guide ifchanges will be made

I will not use it

I do not know


15

Pedagogical aspects – The material ... assures awareness of rights and obligations during

the WBL, creates a dialogue between mentors, students and

coordinators offers a new approach for mentors and students to

WBL in this sector enhances transparency of learning outcomes, and

can be enhanced further

Conclusions: The GEMS logbooks

Challenge Keep students motivated Clear indicators on choosing mentors and

companies


16

Organisational aspects Mixed feedback

Challenge Adaptability into specific workplaces Flexibility concerning logbook contents, avoid

prescriptiveness, suitable to company profiles

“In my opinion GEMS materials can and will not define range of knowledge needed in my trade. But still the GEMS materials help to define how to work with the students.” [mentor, tourism and hospitality]



17

Economic aspects – A Challenge Recommendation: Reduce time expenditure for

usage

Technical and cultural aspects Translation important to users, comprehensible Preference for digital version



18

General Conclusions

Suggested improvements mainly concern communication channels and operational details

Successful procedure to develope material Output (logbooks) and dissemination rather positive attitude of mentors concerning

future use of logbooks and collaboration for development

transnational partnership


19

Thank you

Jón Torfi Jónasson ([email protected]) and Anne-Christin Tannhäuser ([email protected])

Andrea G. Dofradóttir ([email protected])

Social Science Research InstituteUniversity of Iceland

Technology

Methods and results of the GEMS user survey - External evaluation