D9.2 Interim Evaluation - · PDF fileVISCED Interim Evaluation 1 Evaluator: Dr Maggie McPherson Interim Evaluation Report Deliverable: D9.1 Project Agreement Number: 511578–LLP–1–2010–1–GR-KA3

VISCED Interim Evaluation 1 Evaluator: Dr Maggie McPherson

Interim Evaluation Report

Deliverable: D9.1

Project Agreement Number:

511578–LLP–1–2010–1–GR-KA3-KA3MP

Project funded by the European Commission

Document Title VISCED – Quality Evaluation Interim Report

Deliverable no. D9.2

Date of issue 15 January 2012

Author Maggie McPherson

Contact name Maggie McPherson

Organisation University of Leeds

Address School of Education

Hillary Place

Leeds LS2 9JT

United Kingdom

Telephone +44

Email [email protected]

Contributors to document Project partners

Quality Reviewers Paul Bacsich

Contractual date of delivery 31st

December 2011

Actual date of delivery 15th

January 2012

Approval status Reviewed internally

Abstract

This report summarises the evaluation of each Work Package,

with evidence from partner self-evaluation, ethnographic

observations and documentary analysis.

Keyword list Work Package; quality and evaluation

Distribution list EU Commission [DG3]; VISCED partners

Method of distribution Email

Electronic copy filed “VISCED files” in VISCED Dropbox

Confidentiality status PU

History

Version number Date Revised by Revision date

1.0 15th

January 2012


Table of Contents

1 Introduction: Validity of Project Design ........................................................................ 4

2 WP1 Project Planning and Management [Led by Sero] ................................................... 5

Pre-project meeting in London, January 2011 ....................................................................... 5

Boot Camp Meeting, Brussels, February 9th - 11th ............................................................. 5

Additional Ad-hoc Project Meeting (April 2011) ............................................................... 7

1st Flash Project Partner Meeting (8 June 2011) .............................................................. 7

2nd Flash Project Partner Meeting (6 July 2011) .............................................................. 7

Project Partner Meeting 2 Oeiras (13 September 2011) .................................................. 7

Project Partner Meeting 2 Oeiras (13 September 2011) – WP1 Project Management

[Sero] ..................................................................................................................................... 8

3rd

Flash Project Partner Meeting (12 October 2011) ............................................................ 8

4th

Flash Project Partner Meeting (10 November 2011) ........................................................ 8

3 WP2 Field Research [led by Sero] ................................................................................... 10

Additional Ad-hoc Project Meeting (April 2011) ............................................................. 10

1st Flash Project Partner Meeting (8 June 2011) ............................................................ 10


Project Partner Meeting 2 Oeiras (13 September 2011) ................................................ 10

3rd Flash Project Partner Meeting 12 October 2011 ........................................................ 10

4th Flash Project Partner Meeting (10 November 2011) ................................................. 10

WP2 Summary .................................................................................................................... 11

4 WP3 Analysis & Recommendations [MENON] ............................................................... 12

1st Flash Project Partner Meeting (8 June 2011).............................................................. 12

Project Partner f2f Meeting 2 Oeiras (13 September 2011) ........................................... 12

3rd Flash Project Partner Meeting (12 October 2011) ..................................................... 12


WP3 Summary .................................................................................................................... 13

5 WP4 Critical Success Factors ........................................................................................... 13

6 WP5 Consultation of Experts in this Field [led by Lambrakis] ..................................... 15

1st Flash Project Partner Meeting (8 June 2011).............................................................. 15


1st IAC f2f Meeting Oeiras (13 September 2011) ............................................................. 15



2nd IAC f2f Meeting Berlin (30 November) ....................................................................... 16

WP5 Summary .................................................................................................................... 18

7 WP6 Piloting [led by Tensta] ........................................................................................... 19


8 WP7 Dissemination [led by AtiT] .................................................................................... 20

Project flyer ...................................................................................................................... 20

Project website ................................................................................................................. 20

Project wiki ....................................................................................................................... 20

Project Presentations in 2011 .......................................................................................... 20

Project Presentations considered for 2012...................................................................... 20

Project newsletters. ......................................................................................................... 20

Other Dissemination Avenues .......................................................................................... 20

9 WP8 Exploitation [led by Lambrakis] ............................................................................. 20


10 WP9 Quality and Evaluation ........................................................................................... 21

Project Progress .................................................................................................................. 21

Evaluation of Effectiveness ......................................................................................... 21

Project Challenges ........................................................................................................ 21

Analysis of project activities to-date ........................................................................ 22

Appendix 1: VISCED Meeting 1 (Kickoff) Evaluation Responses ........................................ 23


1 Introduction: Validity of Project Design

This aspect of the interim project evaluation will take into account the baseline condition at

the outset of project. Given that the proposal built on an earlier project that investigated

Virtual Campuses (Re.ViCa), this project had a logical and coherent plan based on previous

experience. It was while the Re.ViCa project was being carried out that it became apparent

that there was a deficit of knowledge in terms of what has been happening with regard to

younger learners taking part in online learning during their formal school years.

Consequently the underlying problem was to discover to what extent there has been an

uptake of online education for virtual schools and further education colleges.

The main strategy for the first year of the project was to properly establish the status quo

for this particular sector. The main strategic components of the project for evaluation

during 2011 are as follows:

• WP1 Project Planning & Management

• WP2 Field Research

• WP3 Analysis & Recommendations

• WP5 Consultation of Experts in this Field

• WP6 Piloting

• WP7 Dissemination

• WP8 Exploitation

• WP9 Quality and Evaluation

This report will chart the progress and report any discrepancies with planned activities and

outputs during the first year of the project.


2 WP1 Project Planning and Management [Led by Sero]

The project management was predominantly carried out by SERO in the UK. The acceptance

of this project for funding was made somewhat later than hoped. As a consequence, this

delay meant that the initial project start up project meeting could not be held in January as

planned, and this had to be postponed until February. Despite the fact that these delays

made management overall more difficult, in order to avoid losing momentum, a few key

partners decided to turn this to advantage by going ahead with holding an early ad hoc

meeting.

Pre-project meeting in London, January 2011

Prior to the first official project launch meeting, a preliminary meeting was held in London

during the BETT Show. The attendees were partners had already arranged to attend the

conference or were in a reasonable position to get to London. This provided the opportunity

for representatives from SERO, Lambrakis, ATiT and Leeds to have an early discussion, which

in turn helped with initial thinking about adjustments needed for the project planning.

Boot Camp Meeting, Brussels, February 9th - 11th

As stated before, the late start to the project led to the launch meeting being delayed until

February. The agenda for this meeting began with clarification of Commission arrangements

and new financial rules. The initial provisional VISCED timeline was presented for to

attendees.

During this first kick-off meeting, it was deemed necessary to have some discussion around

competing stakeholder interests and how these could be aligned. It was noted that although

earlier alliances to the project (e.g. EU School Net) were no longer in a position to become

involved, working relationships with national ministries would continue to play a role. It was

recorded that a communications strategy was needed to ensure partners received

information in a timely manner. This was followed by colleagues’ presentations of their own

institution, philosophy and anticipated contribution to the project. This was not only

intended as an information-sharing exercise, but also contributed an essential element to

team-building and collaborative spirit.

An essential explanation regarding the VISCED wiki ensued. Issued covered were how this

should be edited, that the emphasis would be on exemplars, how experiences from Re.ViCa

might contribute to research results. It was clarified that the emphasis would be on

secondary level education (i.e. pre-tertiary levels) and that final reports would be subject to

peer-review. While discussing WP3 (Influence Maps), consideration was given to countries

might affect others’ ICT policy and implementation, the need to understand of scope issues,

and other such issues. It was explained that the altered start date meant that WP4 could

not begin till year 2. With regard to WP5, it was suggested that the International Advisory

Committee needed to be set up promptly in order to enable attendance at 4 meetings and

to get value from their input and advice. The dissemination plan was outlined, a by line for

the VISCED leaflet was decided upon, project website discussed, and dissemination

strategies presented. The mid-agenda items set out exploitation and evaluation strategies.

The meeting was concluded with final project management issues and future meeting dates

were proposed:


1. Meeting 1 - Brussels Feb’11

2. IAC 1 - Dublin June ‘11

3. Meeting 2 – Athens Oct ’11

4. IAC 2 - Berlin ‘11

5. Meeting 3 – Stockholm May ‘12

6. IAC 3 - Stockholm May ‘12

7. Meeting 4 – Berlin ’12

8. IAC 4 – Berlin ’12


Following the Boot Camp, there were then further delays caused by uncertainty about the

status of P1 Lambrakis as an acceptable applicant, creating some more unavoidable

challenges. Partners were understandably reluctant to commit time, resources and

intellectual energy whilst there was uncertainty about the overall financial status of the

project. To mitigate this hold-up, the team deemed that it was necessary to add in a

supplementary unbudgeted continuation meeting in Brussels at the end of April to discuss

matters related to Work Package 2 and 3, which were key deliverables for 2011.

Additional Ad-hoc Project Meeting (April 2011)

The minutes for this meeting clearly indicate that there was a necessity for another face-to-

face meeting in order to clarify a number of management issues to get the project on track.

Clarification regarding payment was made and an agreement for disbursement of the same

was made. Issues related to communication (Wiki pages, and Dropbox to reduce number of

project emails) were ironed out. To supplement the less frequent face to face (f2f)

meetings, it was decided that regular online Flash meetings were required. SERO agreed to

circulate budget reviews, update Gantt chart of deliverables, and to set up a Risk Register.

ATiT agreed to create a template for reporting.

1st Flash Project Partner Meeting (8 June 2011)

The project managers highlighted a number of issues that had to be attended to. The

budget check revealed a couple of returns from April were still outstanding and partners

were reminded that the next one would be at the start of July. The GANNT chart has been

updated and re-circulated so that timeline for deliverables could be discussed, and it was

noted that WP3 dates needed amendments. The Risk Register was not discussed but the

notes stated that a draft was attached for partners’ comments. The draft Report Template

was still to be completed and would follow. It was noted that with delays and changes to

meetings over the past few months, there was a need to review the project document and

incorporate decisions that have been made in partner meetings and some minor budget

alterations that had resulted from these.

2nd Flash Project Partner Meeting (6 July 2011)

Partners were prompted regarding the imminent six-monthly budget check, and were told

that updated spreadsheets would be circulated. Partners were also reminded about

timescales, i.e. that deliverables to the Commission had to be included with the interim

report at the end of the year, and therefore it was necessary to review internal deadlines

and to strictly adhere to these. As few partners were to be available in August, the next

project gathering would be the 2nd

f2f partner meeting (be held alongside the EFQUAL

meeting, to maximise opportunities to engage experts for consultation).

Project Partner Meeting 2 Oeiras (13 September 2011)

It was hoped that this would allow members identified as being suitable to join the

International Advisory Committee (IAC) to attend. During the preparatory stages for this, it

transpired that this was not the best option for the meeting after all, and the date was

brought forward. In the final event, this meeting was held in Oeiras, Portugal. This proved

fortuitous because it was then also possible to enlist new members to attend a full day IAC

meeting and these experts were able to offer insights into the project achievements so far

and allowed the VISCED team to make timely adjustments to activities still outstanding.


Adjustments were made to the planned meetings, with the first full f2f of 2012 meeting

proposed for Sheffield in May, and the final f2f meeting proposed for Brussels, probably in

October 2012. It was decided that partner meetings should be decoupled from IAC

meetings.

Project Partner Meeting 2 Oeiras (13 September 2011) – WP1 Project

Management [Sero]

Partner Name Attendance Partner Name Attendance

Aarhus - - MENON Claudio Dondi PM only

ATiT Nikki Cortoos All day MENON/Scienter Daniela Proli All day

ATiT Sally Reynolds All day Sero Paul Bacsich All day

EITF Jüri Lõssenko All day Sero Barry Phillips All day

EFQUEL - - Tensta - -

Lambrakis Nikos Zygouritsas All day TIEKE - -

Leeds University Maggie McPherson All day

Work Package 1

Item Notes Action

Next project

partner

meetings

Next Flash meeting not discussed June Sero suggest this be held on Wednesday 12th

October, 1400 – 1600 BST; 1500-1700 CET;

1600-1800 eastern Europe time. Can all partners

confirm that at least one representative can

‘attend’? Sally to organise and book time &

circulate URL to partners

To be confirmed at next Flash meetings

Maggie suggested it would be useful if

partners unable to attend meetings

submitted a vidcast in advance

To be considered at next Flash meeting

Interim

budget

check

[Post-meeting note] Partners are reminded

that the next review of expenditure [Jan-

Sept 2011] will be required shortly

Giles to circulate budget review spreadsheet for

completion and return by partners before 20th

October.

Project

Timeline

GANNT chart to be updated and re-

circulated for comments

Giles & Paul to do this – will be circulated early

in October

this meeting in Berlin prior to the Online Educa Conference during early November 2011.

3rd Flash Project Partner Meeting (12 October 2011)

Project management issues raised during this meeting were the budget check, and it was

noted that some partners were having travel budget problems due to expensive airfares.

The project managers were to whether the EC would strike out expenditure perceived as

excessive. Sero asked all partners to communicate if they were likely to have any problems

meeting deadlines up to end of 2011 at which point it would be possible to review timescale

and amend as appropriate and reissue before next Flash meeting.

4th Flash Project Partner Meeting (10 November 2011)

All partners were asked to complete and return the end-of-year budget check to ensure the

Commission would accept progress report. In addition, partners were asked to inform Sero

know a.s.a.p. if they were likely have any problems meeting deadlines up to end of 2011 and

were asked to respond to progress report requests in a timely fashion.



3 WP2 Field Research [led by Sero]

The project was built on a logical assumption that partners would be able to bring local

knowledge to the field research and that this would speed up the data gathering process for

WP2. The research team found that this work package suffered the knock-on effects of a

delayed start as explained in WP1. However, while for the most part the planned activities

seemed to be realistic and coherent, in reality and in a true spirit of research, the

investigation of virtual schools was delving into an unfamiliar environment.

Additional Ad-hoc Project Meeting (April 2011)

The second part of the meeting was concerned with the WP2 Field Research deliverables.

Colleagues considered levels, context/dimensions, drivers, parental attitudes, quality and

accreditation etc. and how these would relate to exemplars. It was also agreed that

additional guidance for wiki use would be provided.


Although some partners were able to report making progress with data gathering, country

reports had not yet been uploaded, giving the appearance of being seriously behind with

completing and loading. All were urged to contribute to the wiki as soon as possible, and

where partners had not yet really started work, their ‘home’ country was recommended as a

good point to start. Partners were also asked for brief CVs.


Whilst it was noted that there was a great increase in populating the wiki, partners were

urged to continue uploading material on to the wiki as soon as it is drafted, even if it needed

further editing. Additionally, it was apparent that progress was being made on finding

potential exemplars. Partners were reminded to upload brief CVs for themselves as soon as

possible.

Project Partner Meeting 2 Oeiras (13 September 2011)

This was a useful meeting to update project management issues, but more importantly, an

opportunity to clarify field research issues. The importance of time entries of wiki entries for

country reports was emphasised and additional possibilities were welcomed. All partners

were reminded to help with newsletter dissemination.

3rd Flash Project Partner Meeting 12 October 2011

The discussion revealed that country research had progressed and that a considerable

number of virtual schools had been identified, albeit in varying states of evolution. Partners

were asked to ensure that all exemplars found in were included as separate wiki entries.

Nevertheless, it seemed that country reports were sufficiently solid state as to provide a

reasonable basis for starting on WP3.


Despite the fact that many virtual schools had been identified, not all exemplars had been

given separate wiki entries and partners were reminded to do this. This left continental

reports to be addressed and Sero made a commitment to do this early in 2012.


WP2 Summary

Despite time constraints, since finding a shared intellectual understanding was crucial for the

success of a project of this nature, the partners involved in this work felt that it was

necessary to hold to lengthy debates about taxonomy and typology, i.e. to identify what

would be in scope and what would not fit. This inevitably led in turn to some delays in data

collection in terms of country and regional reports, for both within Europe and beyond. As a

consequence, the quality and depth of these reports is variable. Although it the project

managers hope to remedy this in the second year of the project, it has meant that the

quality and quantity of reliable information available for Interim List of Exemplars [D2.3] is

currently not as complete as it was originally envisaged it would be at this stage of the

project.

From an outsider insider point of view, it is thought that in this project, as is the case in most

large partnerships, there has been varying levels of commitment. The effect of this is that

some colleagues have been less involved in implementation than project managers hoped

for. Yet it is sometimes difficult to determine how the strategic aims of this particular

project fits into other workload demands of project partners. Furthermore, on occasion, in

large organisations, it may be that colleagues simply do not have the capacity or the level

influence to change local and urgent demands on their time. Despite this, the project

managers appear to have overcome a number of difficulties and are hopeful of achieving

some success in getting close to the planned objectives, albeit a little later than hoped for,

with some of the exemplars being submitted very recently.

With this in mind, as an evaluator, it has been useful to receive the following detailed

reflection on this activity as highlighted below:

“My main contribution to research and report writing has been in compiling country reports for

Scotland and Latin America, writing the continent report for Latin America [all for D2.2] and editing

and producing D2.3, the interim list of exemplars. This has been a fascinating task and has

highlighted a number of issues for me:

• the extent to which many European virtual schools operate 'under the radar' as far as governmental

awareness and regulatory frameworks are concerned;

• the numbers of virtual schools in Europe is greater than we originally anticipated, but in several

western European and Latin American countries expansion seems to have stalled in the second half

of the first decade of the C21;

• although there are relatively few examples of failed virtual schools, few outside the USA, Canada and

some countries of Australasia have grown to any significant size;

It is likely that our lack of research capacity to cover Asia [especially China] fully has resulted in us

missing a number of virtual schools and colleges in this region - there are certainly plenty of

commercial organisations marketing a range of virtual and distance learning products in the

region.”

As a final word on this work package, the project managers are aware of the knock-on effect

of delayed on WP3 and WP4, and have made a note to update the project Risk Register to

take account of this and detail the remedial measures that need to be taken during Year 2.


4 WP3 Analysis & Recommendations [MENON]

This deliverable was built on the assumption that partners would typify instances of Virtual

Schools & Colleges, create EU Influence Maps, identify potential Critical Success Factors

(CSFs), identify innovative ‘Good Practice’ for piloting, create Interim Policy

recommendations and make interim recommendations for teacher training.


Since the analysis depended on work completed in the previous work package, it was not

surprising that not much had been done on WP3 at the point of this meeting – therefore it

was noted that slight adjustments to timings might be necessary. Partners were informed

that the colleges typology would be relatively straightforward as this built directly on work

completed in Re.ViCa, but that developing the typology for virtual schools would be more

difficult and that it was necessary to create a much larger list of potential exemplars. It was

noted that this WP was central, and that USA experiences of virtual schools would add

considerably to this. It was of interest that whilst in some countries, such as Italy, there

were several virtual universities but that this had not translated into similar initiatives in the

high school sector. Partners were encouraged to look beyond Europe as exemplars beyond

this zone would be very informative.

Project Partner f2f Meeting 2 Oeiras (13 September 2011)

Partners reviewed the current state of progress country and regional reports of virtual

schools discovered thus far and it seemed that most zones were covered to some extent, but

it appeared that there was still more work to be done. Drawing from WP5, and with respect

to WP3, it transpired from IAC advice that there was a move towards blended rather than

pure online. In response, a possible taxonomy of virtual schools as a continuum was

suggested, and a 6-level hierarchy with 5 intermediate points for groups of entities

proposed. Partners were warned that the deadline for Influence Maps was approaching.

With regard to Case Studies, different levels were possible and Susan Patrick offered to

share a template being used by iNACOL for this. The minutes of the meeting mentioned that

it was premature to discuss CSFs at this point but that partners were encouraged to identify

innovative good practice from that point on.


It appeared that examples of virtual schools being found outside Canada and USA were

somewhat smaller than expected and it was identified as a point for further investigation.

There was conjecture that this may be related to the way they were being run, but was also

suggested that there were inconsistencies between the start-up incentives and the needs

and targeting of the venture (programmes, etc.) in countries where schools have more

autonomy. Partners looked to MENON to provide Influence Maps.

It was noted that the first draft of the typology was not yet ready, but project team was

warned to learn from Re.ViCa and to not over-analyse this. It was decided a pragmatic

approach was more appropriate and the USA taxonomy would provide a way forward for

producing case studies. A list of thirteen potential case studies was proposed – ultimately

this would be refined and brought down to around nine – so partners were called upon to

offer more so that these could be chosen from a long list of twenty five. A good discussion

on this issue ensued and it’s clear partners have given due consideration to this task.


Concern was expressed about deadlines and project managers agreed to review timescales.

Attention was drawn to the teacher training circulated before the Oeiras meeting. In

response, a comment was made that small European virtual schools may not concern

themselves with this issue and partners do not yet have the knowledge of teacher training –

this required analysis of the US/Canada experience.


A preparatory document was provided, and this was discussed in the introduction. It was

noted that this had turned out to be a much larger work package than initially envisaged,

and in an ideal world, it would have been divided into 4 separate WPs at the outset. It was

suggested that in this project schools should be regarded as more important than colleges

and that partners should concentrate on these. Follow-up on the typology indicated that

virtual schools are not simply about use of technology, but it was necessary to take a cross-

dimensional approach, e.g. innovation, curriculum development and institutional autonomy,

with others to be added in due course. With regard to Influence Maps, it was decided not to

embark on a metric analysis, but to concentrate on Estonia, Finland and UK as key countries

to illustrate significant differences. Given the progress on the other work packages, it was

noted that Sero was now in a position to begin work on potential success factors. The task

on identifying innovative good practice for teachers was discussed, and partners were

reminded that this was essentially a literature search, and that both existing practice and

innovative practice that could potentially be used in virtual schooling could be included.

National level documents are limited to England, Finland, and Estonia at this point, but for

the final report, documentation for Sweden, Flanders, Scotland is still needed to make

ministry-level policy recommendation. With regard to the Case Studies, work is ongoing,

and partners were asked to do a case study outside own country if nothing was found at

home, and to consider various dimensions, e.g.: geography, culture, GNP, target audience

etc.

WP3 Summary

The project activities did appear to be logical, coherent and realistic but the goals were also

very ambitious, and it is not surprising that this has proved challenging to the team. The

early WPs are crucial for the success of the project as they provide a foundation for the work

in the second year, and the late start date have had an impact in terms of creating

unavoidable delays. Despite this, and partners not always being totally focused, the project

managers have redoubled their efforts to keep everyone to task and to time, whilst being

realistic about what is achievable. A Risk Register has been used to monitor obstacles and to

work out what needs to be adjusted. Overall, through regular meetings the partners in this

WP have been encouraged to think strategically, and now appear to be working more

effectively towards achieving the planned objectives.

5 WP4 Critical Success Factors

This WP has not discussed at the first year meetings as it will only be properly embarked on

in the second year of the project.



6 WP5 Consultation of Experts in this Field [led by Lambrakis]

Although discussed at the Brussels meeting in April, little had been done by the time of the

1st

Flash Meeting (discussed next).


The urgency of identifying suitable experts to join the IAC was noted, and partners was

entreated to provide names from ministries and associated agencies. An introductory letter

was drafted for distribution. Given that two IAC meetings had been proposed, it was

suggested that a smaller IAC meeting should be planned for the EFQUEL Innovation Forum in

Portugal in September, with possibilities for other mini-IAC meetings – e.g. ALT-C [Leeds –

September], EMINENT [Italy – 3rd week in November], MEDEA [end of November] to be

considered. Since the location, schedule and associated costs of IAC meetings are

substantially different to what was originally envisaged, there is a need to revisit the

proposal budget. It was debated whether Informed Consent for IAC members was required,

but this has not been considered necessary. It would however be important to put proper

ethical procedures in place for the piloting, especially when working with younger people.


By this time the desired 110 names were received for the long list. This was reduced to less

than 50 invitations, many of who had already replied in the affirmative. As few were from

industry, it was muted that this may need to be addressed. It was confirmed that the first

IAC meeting would take place on Wednesday 14th September in Oeiras. It was noted that

there would be still a necessity to revisit the IAC budget (as previously mentioned).

1st IAC f2f Meeting Oeiras (13 September 2011)

Greater light was cast on country and regional reports by comments from International

Advisory Committee (IAC) when the meeting was held in Oeiras. Significant contributions

came from Professor Morten Paulsen and Susan Patrick [iNACOL, who works with more than

50 national Ministries worldwide]. At the start of the meeting Susan Patrick gave an

introduction and overview of the work of iNACOL (Policy, QA, working with Foundations)

and parallels with VISCED. It was noted that the wiki provided a much needed resource and

the partners were complimented on this contribution. The other main comments related to

WP3, and is reported in more detail there.


Concerns were voiced about low IAC membership numbers from industry. It was noted that

it was important to know what influence international vendors and suppliers (e.g. Pearson,

Cisco and VLE suppliers in Latin America) were having on virtual developments. It was

agreed that the second IAC meeting would focus on policy and would take place in the

Palace Hotel, Berlin, at the end of November to take advantage of input from experts

already attending Online Educa. Future meetings are as follows: 3rd

meeting (to be organised

by Sero) to be held in Sheffield in May 2012, to coincide with a conference which is currently

being planned and will have a focus on practitioners. The 4th

and final meeting (to be

organised by MENON) is to be held in Brussels in October 2012, and will focus on policy.


Organising partners were reminded to refer to the project budget for details of allowances

for organising accommodation, meeting room, catering and attendees’ expenses.


There was an initial concern about the number of definite acceptances - to make the

meeting useful, 12 attendees were desirable. It was agreed that some additional work to

boost attendance was required.

2nd IAC f2f Meeting Berlin (30 November)

In the event, nine experts were able to attend and provided some valuable feedback. It was

noted that some countries/institutions were more active than others (e.g. Turkey, Open

Institute of Catalonia), and some institutions might be active but not defining their activities

as virtual schooling. Other comments were divided into two parts; general comments and

specific comments about VISCED activities.

General comments can be broken down again into positive factors (inclusion strategies;

‘necessity’ as a driving factor/motivation, i.e. for learners where there is no other way of

accessing education; power of IT as a good driver for learning) and barriers (safety;

infrastructure or security/firewall issues; exams / accreditation; tradition; investment into

classroom equipment). Furthermore, in the USA there is a differentiation between

‘awarding institutions’ (schools) and ‘course providers’ (e.g. commercial sector online

content providers) – this is a taxonomic feature which needs addressing, and pricing was a

driving factor in the growth of the Virtual Universities in the US (because prices are so high).

The question arose as to whether cultural/traditional issues of childminding, socialisation,

etc. do not exist in the US with the growth of Virtual Schools in the same way they do in

Europe. It was also mentioned that Pearsons are delivering 2 virtual degree courses.

IAC feedback on Country Reports on the wiki included suggestions for additional schools and

colleges (in Spain, France, Canada, and Scotland) and suggested others for follow up (US,

Finland, Denmark, Australia, e.g. Connected Classrooms, Sri Lanka, Nepal and UAE). Some

corrections were offered (e.g. Devoirs.fr is not a Virtual School) and additions (in France

there are numerous virtual vocational colleges and courses; the ‘new’ Finland Virtual School

is a Swedish-speaking one). An additional US policy paper was recommended to follow up.

It was noted that there are pitfalls too (the Finland Ministry of Education recently sued a

Virtual School provider for return of funding after it had enrolled students from outside of its

geographical mandate – a judgement is due later this year). The landscape is also fluctuating

(e.g. there has been a change of administration in Denmark and a restructuring of Ministries

– there is now no Ministry of Education – to a Ministry of Child and Teaching and a Ministry

of Education and Innovation) and there may be some additional categories of learner (e.g. in

the Netherlands, there are children of diplomats; Roma; children of Rhine river-shipping

communities; and in the US there are schools for Jewish groups – which apparently could

even go global; Gay and Lesbian groups).

When discussion turned to Success Factors, the IAC group considered two points of

particular importance: the degree of innovation (regular reviewing and updating of both

technologies and pedagogies), and the relevance of tools and activities to the specific

learning environments of the target audience.


The IAC also went on to make a number of general comments about Success Factors:

• “Success depends on Motivation/Access/Knowledge and Skills – if any of these is

missing then success is impossible;

• Consider sub-factors dependent upon the group of learners targeted by the Virtual

School, i.e. success factors for a Bednet type of target group will be different than

those for an iScoil type of target group;

• Virtual Schools have been successful where ‘there is something to be solved and

these schools have solved it’ and can demonstrate this to society;

• We need to agree what we mean by success – and also agree on the extent to which

sustainability is important to this discussion;

• Sustainability is connected to the quality of the Virtual School but notions of scale

also have an impact: a school can be successful but may not be sustainable;

• The importance of a glossary where readers and researchers can check for common

understanding of terms;

• We should consider constructing models to take in account (aspects to do with the

school and how it operates; and aspects to do with state and national policy).

• Could we develop a diagrammatic representation - ‘A systemic model of a school that

operates successfully’ which would show interdependencies?

• This list of Success Factors could be used to educate people as to the range (and

potential) of Virtual Schools.”

With regard to the VISCED taxonomy, there was a general agreement from the IAC that

“tagging everything which may be of use to everyone was a thankless task, and one which

may ultimately be an impediment to those using the wiki”. The experts did not dispute the

main dimensions identified. As such, the following is the full list of potential, additional

dimensions for which there was also general agreement that the information would be of

value. Some (such as Technology used) were simply an attempt for the group to clarify what

the ‘tags’ might look like.

There was not enough have time to discuss the suggested list in depth but the list could

useful as a cross-check with Case Study questions. The ‘tags’ are areas identified by the IAC

members as being of key interest (hence their suggestion that these be explicitly identified).

Several IAC members expressed a desire for some identification of the ‘pedagogical

paradigms’ e.g. the blend, the typology, learner-centric, etc. There was also a question as to

whether it would be useful to add ‘Open’ to Full-time and Supplementary in cases where it

was not known when students are studying because it is ‘open’, although some members

felt this was included by implication under Supplementary. With regard to the Case Study

list , it was suggested that VISCED should look at the updated list of Success Factors and

cross-reference this to see which success factors are not represented – this could then be

used to canvass the IAC via e-mail for suggestions which might ‘fill the gaps’. It was thought

that once a Case Study has been completed and categorised, VISCED should create and

publish a list of institutions that have similar characteristics or may be of interest to anyone

looking at this initial Case Study. Finally, IAC members noted there was no Case Study with a

specific focus on ‘accelerated learning’ but one expert suggested she could supply one from

the USA.


WP5 Summary

The late start of the project meant that the first IAC was held later than originally envisaged,

with the consequence that the experts’ first input came a little bit more behind schedule

than would have been desirable in ideal conditions. Given that this task contributed and is

logically linked to the planned objectives of Work Packages 3 and 4, the project managers

indicated at that point that this would be recorded in the Risk Register in order that

mitigating actions can be taken to avoid adverse affects on upcoming tasks. Nevertheless,

the second meeting was very fruitful in terms of feedback and suggestions for improvement.


7 WP6 Piloting [led by Tensta]

This work package was delayed for reasons already mentioned, but planning for this began

in earnest after the 2nd

f2f meeting in Oeiras.


The Greek pilot is planned for launch with a Study & Jobs Consultancy programme for

teachers and students in some 10 Upper Secondary Schools and the Institutes of pre-service

VET. It will be run through the Network for School Innovation and Lambrakis undertook to

submit a report with the "terms of reference", description, details of the pilot by November.

The first English pilot will be with Notre Dame Catholic High School, Sheffield and has

already started. A teacher has been appointed to lead on extending their use of Moodle for

transition from GCSE to A level Maths, but also seeking to develop the use of the virtual

school element into ‘normal’ classes, thereby encouraging a more blended learning delivery

by all teachers. Maths results improved dramatically last year and this has been attributed

to Moodle, which allowed a great deal of home-based revision leading up to the summer

exams. The shortlist for the second pilot is Sheffield College GCSE or A level English, The

Rowan Centre, Rotherham, or ‘Historiana’ based in the Netherlands. The final choice is still

to be made but Sheffield College seems to be the most viable option at present.

In Sweden, Tensta have started three activities: (i) a series of interactive PowerPoint

presentations for use either in the classroom or for distance learning; (ii) a wiki for

Geography has been set up – also for classroom or distance learning; and (iii) work with

websites and teacher use is being observed.

In all cases, partners in this task were reminded to pay attention to good ethics procedures

before involving young and possibly vulnerable people in sensitive data collection.


8 WP7 Dissemination [led by AtiT]

Throughout the first year, dissemination has been high on the agenda and tasks for this WP

included creating the following:

Project flyer

The flyer was produced promptly after the project started.

Project website

The website too has been available from early on.

Project wiki

The wiki has been the vehicle for sharing ongoing work (WP2 and WP3 in particular)

although there was a question as to whether this should have RSS feeds.

Project Presentations in 2011

ALT-C [Leeds; September]

CONCEDE [Portugal; September]

MEDEA [November]

Project Presentations considered for 2012

EDEN Open Classroom (Oporto June, 2012)

EDUCA (consider workshop

EDEN Research Conference (possible but unlikely), and

e-Learning Expo (Athens – October 2012).

Project newsletters.

The target was 12 articles per newsletter and this was met.

The circulation list is 700 and IAC members will be on copy list.

Other Dissemination Avenues

A VISCED LinkedIn Group has been set up.

A project blog was created.

A full dissemination report will be one of this WP’s deliverables.

9 WP8 Exploitation [led by Lambrakis]

An exploitation plan has been devised and made available to partners - this encourages

identifying innovative practices, piloting these in new environments and developing them in

different contexts, gradually incorporating them into formal and informal systems of

training, into the methods used by business and associations, and into the learning

experience of every individual, all the while making sure that the project outputs are

effectively disseminated as widely as possible.


10 WP9 Quality and Evaluation

Project Progress

The indicators for various tasks described in the original project document were appropriate

and useful for assessing the project’s progress. Project documentation, including the

minutes for meetings, were kept in an accessible form for all partners. Thus it has been

possible for all to keep a check on whether objectives were being achieved. Where outputs

have not been forthcoming in a timely manner, the project managers have been actively

prompting action to try and keep tasks on track.

Evaluation of Effectiveness

Overall, despite some setbacks, the project appears to be making sufficient progress towards

its planned objectives and is likely to achieve its planned objectives upon completion. The

quantity and quality of the outputs produced so far have been reasonable although some

contributions have been more satisfactory than others. It was useful to get feedback from

the IAC that the project approach was producing a useful resource that provided information

well beyond that previously available. Thus, the outputs appear to be contributing to the

project’s aims and objectives and it is hoped that outcomes at the end will meet

expectations. The project organisation has, by and large, achieved the goal of driving

partners onwards to meet targets, which is a great accomplishment, and regular

communication should be maintained to keep the momentum going. Due to time

constraints, there were some preliminary shortfalls in the country reports, but this appears

to be on the road to being addressed. Given the circumstances, it is not entirely clear that

alternative strategies would have been any more effective in achieving the project’s

objectives.

Project Challenges

In general, European projects have a reputation for complicated regulations, even if these

have been put in place for good reasons. Even where project teams have previous

experience of other programmes, to understand the EC Project Handbook and forms for the

LLP and to learn the precise rules for this particular programme, a considerable amount of

time is required.

It has also proved quite complicated understanding the different perspectives of partners in

a wide range of EU countries, both about the intellectual territory of the project and in terms

of their timeliness and attitudes towards responding to project management requests.

The project managers have had to spend a great deal of time chasing a few slow-moving

partners who did not appear to appreciate the urgency for financial data and deliverables to

be provided in a timely fashion. A knock-on effect has been a consequent delay in receiving

the pre-financing payment and distributing it amongst all partners. This has come at a

considerable cost to the project management team in terms of the additional effort that has

had to be committed to achieving the outcomes on time.

A result of the delays outlined, effective research work did not really start until after this

meeting. Despite losing almost six months, partners did not want to ask the Commission for

an extension. This meant that a year's work has to be telescoped into eight months in order


to bring the project timescale back on track. Although this has largely been achieved, this

has been at a high cost of expensive project management time and resources.

Analysis of project activities to-date

Most partners have now taken ownership of relevant work packages and it is hoped that

others will follow this lead. Most of the initial tasks envisaged at the inception of the

project, and yet to be undertaken, are still relevant. However, it is possible that the piloting

will reveal as yet unknown needs that the project (or a future project) should address. The

feedback from IAC appears to link well to the project and will prove useful in improve

outputs further.

Overall, the VISCED has been on something of a roller-coaster ride in its first year. Given

good organisation and timely contributions from all partners, the project managers feel it

should be possible to complete an appropriate successor to Re.ViCa by the end of 2012.


Appendix 1: VISCED Meeting 1 (Kickoff) Evaluation Responses

Meeting Preparation – did you receive all preparatory information in a timely manner?

Definitely Mostly Moderately Limited Definitely not

xxxxxxxxx xx x

Meeting Organisation – was the meeting run efficiently and sessions kept to time?

Definitely Mostly Moderately Partially Definitely not

xxxxxxxx xxxx

Meeting Agenda – were all the essential issues and topics covered?


xxxxx xxxxx

Presentations – were the WP leader sessions well delivered, useful and informative?


xxxxxx xxxxxx

Meeting Discussions – was there sufficient time to discuss essential issues and topics?


xxxxxxxxx xxx

Participant Questions – do you feel that any questions you had have been properly

answered?


xxxxxx xxxxxx

If you have additional questions, could you please elaborate below:

Could parallel working sessions on topics be more useful and effective in coming planning meetings?

Maybe too diluted, 1 day and ½ or 2 would possibly have been enough!

Project Planning – do you feel that you are sufficiently now well prepared for future

activities?


xxxxxxxxxxx x x

If you need additional preparation, could you please elaborate below:

Target groups needs to be clarified, interaction between partners, review cycles, etc. might be drafted.

Deciding SMART objectives in the end for each partner.


Documents

D9.2 Interim Evaluation - · PDF fileVISCED Interim Evaluation 1 Evaluator: Dr Maggie McPherson Interim Evaluation Report Deliverable: D9.1 Project Agreement Number: 511578–LLP–1–2010–1–GR-KA3