E-xcellence benchmarks compared

Embed Size (px)

Citation preview

  • 8/22/2019 E-xcellence benchmarks compared

    1/13

    Paul Bacsich, BELA 1 file E-xcellence-analysed-rel1

    E-xcellence benchmarks compared

    The E-xcellence system of benchmarking e-learning was released to public gaze at the

    EADTU conference in Tallinn, Estonia on 23 November 2006. This is a brief analysis of the

    33 E-xcellence threshold benchmarks in terms of relevance to UK HE. OLF members are

    likely soon to have a briefing on the E-xcellence project from EADTU.

    This paper was developed primarily for assistance to MIT90s institutions looking ab initio

    for carefully written criterion statements and to Pick&Mix institutions looking for carefully

    written potential supplementary criteria covering topics not already dealt with in the current

    version of Pick&Mix 2.0. However, the paper may also be assistance to ELTI institutions,

    specifically those who are members of the Open Learning Foundation (OLF). It is out of

    scope for this work but not particularly difficult to extend the analysis to ELTI or indeed

    ACODE.

    1 What is E-xcellence?

    Quoting from the BenchmarkWiki entry:1

    E-xcellence (pronounced E-excellence) is an EU-funded project run byEADTU

    (the European Association of Distance Teaching Universities) with the assistance of

    12 other partners. It has a project web site athttp://www.eadtu.nl/E-xcellence/.

    E-xcellence started in January 2005 and concludes in November 2006, with a major

    launch at the EADTU conference2 in Tallinn, Estonia, 23-24 November 2006.

    Originally E-xcellence was not envisaged as a benchmarking methodology but as a

    quality monitoring tool, but at about a year into the project there was a shift in

    emphasis and benchmarking is now one of the aims envisaged for E-xcellence. In factthere are three orientations of the methodology:

    Assessment tool (at both institutional and programme level) (i.e.

    benchmarking)

    Quality improvement tool (internal quality care system)

    Accreditation tool for accreditation

    Expected outcomes include:

    Inventory of benchmarks forgood e-learning (i.e. these include relatedcriteria and indicators setting standards of excellence and indicators for

    validation) (a web-based guide)

    On-line self-assessment toolQuickScan3 based on the identified benchmarks

    Manual on good practices directly related to the benchmarks

    1 This ishttp://elearning.heacademy.ac.uk/wiki/index.php/Main_Pageof 25 November 2006. By the nature of

    wikis, the text is subject to change.

    2

    Seehttp://www.eadtu.nl/conference-2006/.3

    Seehttp://www.eadtu.nl/e-xcellenceQS/.

    http://www.eadtu.nl/http://www.eadtu.nl/http://www.eadtu.nl/http://www.eadtu.nl/E-xcellence/http://www.eadtu.nl/E-xcellence/http://www.eadtu.nl/E-xcellence/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://elearning.heacademy.ac.uk/wiki/index.php/Main_Pagehttp://elearning.heacademy.ac.uk/wiki/index.php/Main_Pagehttp://elearning.heacademy.ac.uk/wiki/index.php/Main_Pagehttp://www.eadtu.nl/conference-2006/http://www.eadtu.nl/conference-2006/http://www.eadtu.nl/conference-2006/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/conference-2006/http://elearning.heacademy.ac.uk/wiki/index.php/Main_Pagehttp://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/E-xcellence/http://www.eadtu.nl/
  • 8/22/2019 E-xcellence benchmarks compared

    2/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 2 27 November 2006

    Quality assurance system (internal validation based on the standard of

    excellence)

    Reports on pilots which will test the validation approach

    Establishment and training of a visitation team both for quality assurance and

    for accreditation (to be seen as distinct procedures)

    The basis of the E-xcellence benchmarking methodology is as follows:

    1. some 50 excellence benchmarks are formulated and divided over 6 categoriesof strategic management, curriculum design, course design, course delivery,

    staff support and student support.

    2. Exactly 33 of the 50 benchmarks are categorised as threshold. Threshold for e-learning is defined by the E-xcellence team as crucial for interactiveness,

    accessibility and flexibility of the e-learning programme. These are found in

    theQuickScan.

    3. Each criterion is a bundle of indicators.

    4. Criteria are graded currently at four levels but numeric scores are not used anda total score is not computed.

    5. The threshold level can be determined by theQuickScanonline self-audit,but the excellence level requires in addition a visit from an expert team.

    One of the key partners in the EADTU consortium is the Open Learning Foundation (OLF).

    Members of OLF active in Benchmarking Phase 1 are: Brunel (OBHE), De Montfort

    (OBHE), Derby (Pick&Mix), East London (ELTI), Glamorgan (MIT90s), Lincoln (OBHE),Teesside (Pick&Mix) and Thames Valley (MIT90s).

    2 The E-xcellence criteria

    What follows (on the next and succeeding pages) is a table of the E-xcellence criteria:

    Number

    Statement from E-xcellence (this is a best practice statement)

    MIT90s category: Strategy, Structure, Management Processes, Individuals (split intoStaff and Students), Technology, and External Environment

    Nearest Pick&Mix criterion (the reference used for this isPick&Mix release 2.0 beta1 release notes athttp://elearning.heacademy.ac.uk/weblogs/benchmarking/?p=139)

    Commentary with particular relevance to Pick&Mix in particular but the UK HEquality regime in general

    Recommendation (oriented mainly to MIT90s institutions) as to whether the criterionshould be Recommended (for consideration), Optional or Discouraged (marked R, O,

    D respectively)

    Recommendation (oriented mainly to Pick&Mix institutions) as to whether thecriterion should be a new Supplementary criterion (marked NS) in Pick&Mix 2.0.

    http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/http://elearning.heacademy.ac.uk/weblogs/benchmarking/?p=139http://elearning.heacademy.ac.uk/weblogs/benchmarking/?p=139http://elearning.heacademy.ac.uk/weblogs/benchmarking/?p=139http://elearning.heacademy.ac.uk/weblogs/benchmarking/?p=139http://www.eadtu.nl/e-xcellenceQS/http://www.eadtu.nl/e-xcellenceQS/
  • 8/22/2019 E-xcellence benchmarks compared

    3/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 3 27 November 2006

    2.1 Conclusions from the tabulation process

    The main ones are as follows (and all noted in the table):

    1. Many of the E-xcellence criteria should be split into several (usually just two)subcriteria in order to facilitate effective judgements (especially for scoring).

    2. A number of the E-xcellence criteria as applied in UK HE are superfluous in the sensethat the QAA precepts already cover these aspects or they are far too detailed, thus

    belonging to quality tick-boxes not to benchmarking criteria.

    3. A small number of the E-xcellence criteria appear wishful thinking rather thanoperationally relevant criteriathese are Discouraged.

    The criteria indicated to be split are: 4 (2), 6 (2), 7 (2), 8 (2), 9 (2), 10 (2), 13 (into 3), 14 (2),

    15 (2), 18 (into 3), 19 (2), 20 (2), 22 (2), 23 (2), 24 (2), 25 (into 3), 26 (into 5!), and 27 (2).

    This would make a total of 57 atomic criteria in all. This is rather high; however, a

    considerable number (18) of these are Discouraged (due to being covered by QAA or too

    detailed) and this would bring the number of Recommended/Optional atomic criteria downagain to a not unreasonable 39.

    2.2 MIT90s methodological conclusions

    1. It seemed to be hard to assign many of the criteria to specific MIT90s categories. Thisshould be of serious methodological concern.

    2. The lack of a Learning category paralleling Technology may be seen as a weakness.Could the Culture metacategory (surrounding Structure, Management Processes and

    Individuals) take on this role?

    3. In some cases it seemed better to assign a criterion to two categories (think of the

    categories as intersecting). This idea should be followed upit may paradoxicallymake it easier to assign MIT90s categories (see also the Culture idea above).

    2.3 References

    ACODE http://elearning.heacademy.ac.uk/wiki/index.php/ACODE

    CHIRON http://elearning.heacademy.ac.uk/wiki/index.php/CHIRON

    ELTI http://elearning.heacademy.ac.uk/wiki/index.php/ELTI

    E-xcellence http://elearning.heacademy.ac.uk/wiki/index.php/E-xcellence

    MIT90s http://elearning.heacademy.ac.uk/wiki/index.php/MIT90s

    Pick&Mix http://elearning.heacademy.ac.uk/wiki/index.php/Pick&Mix

    http://elearning.heacademy.ac.uk/wiki/index.php/ACODEhttp://elearning.heacademy.ac.uk/wiki/index.php/ACODEhttp://elearning.heacademy.ac.uk/wiki/index.php/CHIRONhttp://elearning.heacademy.ac.uk/wiki/index.php/CHIRONhttp://elearning.heacademy.ac.uk/wiki/index.php/ELTIhttp://elearning.heacademy.ac.uk/wiki/index.php/ELTIhttp://elearning.heacademy.ac.uk/wiki/index.php/E-xcellencehttp://elearning.heacademy.ac.uk/wiki/index.php/E-xcellencehttp://elearning.heacademy.ac.uk/wiki/index.php/MIT90shttp://elearning.heacademy.ac.uk/wiki/index.php/MIT90shttp://elearning.heacademy.ac.uk/wiki/index.php/Pick&Mixhttp://elearning.heacademy.ac.uk/wiki/index.php/Pick&Mixhttp://elearning.heacademy.ac.uk/wiki/index.php/Pick&Mixhttp://elearning.heacademy.ac.uk/wiki/index.php/MIT90shttp://elearning.heacademy.ac.uk/wiki/index.php/E-xcellencehttp://elearning.heacademy.ac.uk/wiki/index.php/ELTIhttp://elearning.heacademy.ac.uk/wiki/index.php/CHIRONhttp://elearning.heacademy.ac.uk/wiki/index.php/ACODE
  • 8/22/2019 E-xcellence benchmarks compared

    4/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 4 27 November 2006

    3. The table

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    Strategic Management

    1 The institution should have e-

    learning policies and a strategy

    for development of e-learning

    that are widely understood and

    integrated into the overall

    strategies for institutional

    development and quality

    improvement. Policies should

    include both infrastructure and

    staff development.

    Strategy 06 (Strategy) Close match to

    Pick&Mix (PnM) 06

    R

    2 The resourcing of developmentsin e-learning curricula should

    take into account any special

    requirements over and above

    the normal requirements for

    (non-e) curricula. These will

    include items such as equipment

    purchase, software

    implementation, recruitment of

    staff, training and research

    needs, and technology

    developments.

    Managementprocesses

    12 (Costs) and19 (Decisions)

    (not too close:

    see

    commentary)

    This is phrased in away perhaps more

    appropriate to those

    universities without a

    modern approach to

    costing and

    resourcing

    R

    3 The institution should have a

    management information

    system which is reliable, secure

    and effective for the operation

    of the e-learning systems

    adopted.

    Technology Nonehints of

    62 (Integration)

    Those who wish this

    should add as New

    Supplementary

    O NS

    4 .1) When e-learning involves

    collaborative provision the roles

    and responsibilities of each

    partner should be clearly

    defined through operational

    agreements

    .2) and these responsibilitiesshould be communicated to all

    participants.

    External

    Environment

    (external to

    HEI)

    None Recommended as

    New Supplementary

    (perhaps two)

    R

    O

    NS

    NS

    Curriculum Design

    5 E-learning curricula should

    conform to qualification

    frameworks, codes of practice,

    subject benchmarks and other

    institutional or national quality

    requirements in the same way as

    non-e curricula.

    Management

    processes

    or

    External

    Environment

    None (see

    commentary)

    In theory this should

    not be needed in the

    UK in view of QAA

    precepts.

    D

  • 8/22/2019 E-xcellence benchmarks compared

    5/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 5 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    6 .1) Curricula should be

    designed in such a way as toallow maximum flexibility for

    the learner with respect to time,

    place and pace of learning,

    consistent with the satisfactory

    achievement of learning

    outcomes and integration with

    other (non-e) learning activities.

    .2) Use of formative and

    summative assessment needs to

    be appropriate to the curriculum

    design.

    Management

    processes(does not

    seem a good

    match)

    Linked to 08

    and vagueaspects of 04

    This is perhaps two

    criteria.The first has some

    aspirational features

    (i.e. is not suited to

    benchmarking)

    For the second,

    Assessment should be

    added as a New

    Supplementary

    D

    R

    7 .1) Curriculum design shouldensure that appropriate

    provision is made for the

    acquisition of general

    educational objectives and the

    integration of knowledge and

    skills across the programme of

    study.

    .2) When blended learning is

    used, the contribution of e-

    learning components to the

    development of educational

    objectives needs to be made

    clear

    Managementprocesses

    (does not

    seem a good

    match)

    Linked to 08 In theory the firstshould not be needed

    in the UK in view of

    QAA precepts.

    The second is an

    interesting one to

    consider as a New

    Supplementary, even

    if in theory one

    should apply this

    criterion to allcomponents.

    D

    O

    8 .1) Curricula should be

    designed in such as way as to

    require broad participation in an

    on-line academic community.

    .2) As well as student-student

    and student-tutor interactions

    this should include, where

    appropriate, interaction with

    external professionals and/or

    involvement in research and

    professional activities.

    Management

    processes

    (does not

    seem a good

    match)

    Linked to 08 In theory the first

    should not be needed

    in the UK in view of

    QAA precepts.

    The second is

    common sense but

    could be

    overzealously

    interpreted and

    become a flight of

    fancy.

    D

    D

  • 8/22/2019 E-xcellence benchmarks compared

    6/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 6 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    Course Design

    9 .1) Each course should include aclear statement of learning

    outcomes in respect of both

    knowledge and skills. Outcomes

    should be of such a nature as to

    be attainable through e-learning,

    augmented as and when

    necessary by face-to- face

    provision.

    .2) In a blended-learning

    context there should be an

    explicit rationale for the use of

    each component in the blend.

    Managementprocesses

    (does not

    seem a good

    match)

    Linked to 08 In theory the firstshould not be needed

    in the UK in view of

    QAA precepts.

    The second seems to

    be a repetition of 7

    above. It is an

    interesting one to

    consider as a New

    Supplementary, even

    if in theory one

    should apply this

    criterion to allcomponents.

    D

    O NS

    10 .1) Learning outcomes, not the

    availability of technology,

    should determine the means

    used to deliver course content

    .2) and there needs to be

    reasoned coherence between

    learning outcomes, the strategy

    for use of e-learning, the scope

    of the learning materials and the

    assessment methods used.

    Strategy/

    Management

    processes

    alignment

    Linked to 08 The first is an

    aspirational statement

    straight from Quality

    on the Line, and the

    usual caveats apply.

    The second appears to

    repeat (sub)criteria

    repeated earlier.

    D

    D

    11 Learning outcomes should becomparable with those of

    courses delivered by other

    means.

    Individualsstudents

    None (yet) butsee 51 (Service

    Uniformity).

    This is a hard one toassess but it should

    not be ducked. It is an

    interesting one to

    consider as a New

    Supplementary.

    O NS

  • 8/22/2019 E-xcellence benchmarks compared

    7/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 7 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    12 Courses should be designed in

    such a way as to : foster active learning

    facilitate individual study

    and the development of

    study skills

    support the development

    and interaction of learning

    communities

    place the learner in control

    of time, place and pace of

    learning wherever possible

    recognise the diversity oflearners and build on their

    strengths and backgrounds

    make appropriate provisionfor persons with disabilities

    be sensitive in their use of

    materials to the cultural

    diversity present amongst

    learners

    require learners to reflecton, evaluate and provide

    feedback on course

    contents and requirements

    Management

    processes(does not

    seem a good

    match)

    08 at a general

    level

    The principles of

    good pedagogy areaccepted, but the

    bullet points seem too

    specific to be

    benchmark criteria

    they are more of

    quality guidelines

    D

    13 .1) Interactions between

    students and with tutors (both

    synchronous and asynchronous)

    should be facilitated by a

    variety of means including e-

    mail, telephone, group forums

    etc to allow both individual and

    group interactions.

    .2) Access to tutors should be

    designed to be on a regular and

    sufficient basis known to both

    tutors and learners.

    .3) At the minimum level of

    engagement tutors should

    provide learners with timely

    expert advice on course issues

    or materials and individual

    feedback on assignments.

    Management

    processes

    (does not

    seem a good

    match)

    Align with

    Individuals

    students ?

    The first is

    covered in 08 at

    a general level.

    The second and

    third are

    covered in 91(Student

    Expectations)

    These again shade

    into quality

    guidelines.

    D

    D

    D

  • 8/22/2019 E-xcellence benchmarks compared

    8/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 8 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    14 .1) Course design, development

    and evaluation should involveindividuals or teams with

    expertise in both academic and

    technical aspects.

    .2) Integral to the course design

    process should be mechanisms

    for trialling materials and

    incorporating feedback into the

    final product.

    Management

    processes(does not

    seem a good

    match)

    The first is a

    micro versionof 15

    (Organisation)

    with some

    aspects of 10

    (Training)

    The second

    relates to 14

    (Evaluation)

    These again shade

    into qualityguidelines.

    D

    15 .1) Learning materials should be

    designed with a sufficient level

    of interactivity to enable active

    student engagement and toenable them to test their

    knowledge, understanding and

    skills at regular intervals.

    .2) Where self-study materials

    are meant to be free-standing,

    they should be designed in such

    a way as to allow learners on-

    going feedback on their

    progress through self-

    assessment tests.

    Management

    processes

    (does not

    seem a goodmatch)

    Covered in 08

    at a general

    level

    These again shade

    into quality

    guidelines.

    D

    D

    16 Course materials should

    conform to explicit guidelinesconcerning layout and

    presentation and be as

    consistent as possible across a

    programme.

    Management

    processes(does not

    seem a good

    match)

    09 (Learning

    Material)

    Specific guidelines

    (e.g. as in CHIRON)would be the province

    of quality guidelines.

    R

    17 Course materials including the

    intended learning outcomes,

    should be regularly reviewed,

    up-dated and improved using

    feedback from stakeholders as

    appropriate.

    Management

    processes

    (does not

    seem a good

    match)

    Align with:

    Individuals

    students ?

    14 (Evaluation) Good wording. O

  • 8/22/2019 E-xcellence benchmarks compared

    9/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 9 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    18 .1) Courses should provide both

    formative and summativeassessment components.

    Summative assessment needs to

    be explicit, fair, valid and

    reliable ().

    .2) Appropriate measures need

    to be in place to prevent (...)

    plagiarism, especially where

    assessments are conducted on-

    line.

    .3) [split out from last

    paragraph] Appropriate

    measures need to be in place toprevent impersonation,

    especially where assessments

    are conducted on-line.

    Management

    processes(does not

    seem a good

    match)

    Align with:

    Individuals

    students ?

    The first is

    subsumed in 08(Pedagogy).

    The second is

    65 (Plagiarism)

    plus a criterion

    on

    impersonation

    Some institutions may

    wish a NewSupplementary on

    non-plagiarism types

    of cheating to cover

    impersonation etc.

    D

    R

    R NS

    Course Delivery

    19 .1) The technical infrastructure

    maintaining the e-learning

    system should be fit for purpose

    and support both academic and

    administrative functions.

    .2) Technical specification

    should be based on a survey of

    stakeholder requirements andinvolve realistic estimates of

    system usage and development.

    Technology The first has

    aspects of 04

    (Usability).

    The second is

    largely

    subsumed in 07

    (Decision-making

    projects)

    O

    R

    20 .1) The reliability (...) of the

    delivery system should have

    been rigorously tested

    beforehand and appropriate

    measures should be in place for

    system recovery in the event of

    failure or breakdown.

    .2) [split out from last

    paragraph] The security of the

    delivery system should havebeen rigorously tested

    beforehand

    Technology 53 (Reliability)

    Security not

    covered (yet)

    For those institutions

    taking a broad view of

    IT, there may be even

    more new IT-related

    Supplementary

    Criteria to be added.See ACODE for

    guidance.

    O

    O NS

    21 Appropriate provision needs to

    be made for system

    maintenance, monitoring and

    review of performance against

    the standards set and against

    improvements as these become

    available.

    Technology 54

    (Performance)

    but this is a

    more output

    measure

    For those institutions

    taking a broad view of

    IT, there may be a

    new IT-related

    Supplementary

    Criterion to be added

    on evaluation of IT,

    sometimes called

    PnM 14ter.

    O NS

  • 8/22/2019 E-xcellence benchmarks compared

    10/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 10 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    22 .1) The Virtual Learning

    Environment should beappropriate for the pedagogical

    model adopted and for the

    requirements of all users.

    .2) It should be integrated with

    the institutions management

    information system as far as

    possible

    Technology For the first see

    06, 07 and 13but also 04.

    For the second

    see 62.

    In Pick&Mix the first

    would be aconsequence of other

    criteria especially

    PnM 06, 07 and 13

    but see also 04.

    O

    O

    23 .1) The VLE should provide

    information and services to all

    users in a logical, consistent and

    reliable way.

    .2) All users should beconfident that the VLEs

    systems for communication and

    provision of information are

    secure, reliable and, where

    appropriate, private.

    Technology 04 (Usability)

    For the second,some aspects

    have been

    covered earlier.

    Close match

    The second is a kindofstaff satisfaction

    on IT to parallel

    student satisfaction

    this could be added as

    a New

    Supplementary.

    R

    O NS

    24 .1) Materials and information

    accessible through the VLE

    should be regularly monitored,

    reviewed and updated.

    .2) The responsibility for this

    should be clearly defined andthose responsible provided with

    appropriate and secure access to

    the system to enable revision

    and updating to occur.

    Technology For the first 14

    (Evaluation).

    The second is

    related to 15

    (Organisation)

    but with aspectsof Security

    also.

    R

    O NS

    Staff support

    25 .1) All staff with academic,

    media development and

    administrative roles need to be

    able to support the development

    and delivery of e-learning

    programmes without themselves

    being technical experts..2) The institution should ensure

    that appropriate training is

    provided for these staff and that

    this is training is enhanced in

    the light of system

    developments.

    .3) [split out from last

    paragraph] The institution

    should ensure that appropriate

    support is provided for these

    staff.

    Individuals

    staff

    The first may

    be challenged

    in some

    institutions.

    The second is

    10 (Training)

    The third is 16

    (Technical

    Support)

    The first seems

    aspirational and a tad

    old-fashioned. (What

    exactly is a technical

    expertthese days?)

    O

    R

    R

  • 8/22/2019 E-xcellence benchmarks compared

    11/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 11 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    26 .1) Pedagogic research and

    development should beregarded as high status activities

    within institutions with a

    commitment to high quality e-

    learning.

    .2) There should be mechanisms

    within these institutions for the

    dissemination of good practice

    in support of e-learning

    .3) (including good practice

    developed elsewhere and/or

    through consortia),

    .4) and for the training ormentoring of new staff in such

    practice.

    .5) Career development

    incentives need to reflect an e-

    learning culture.

    Individuals

    staff

    none

    Related to 08

    (Pedagogy)

    unclear

    Special case of

    10 (Training)

    18

    (Recognition)

    This seems hard to

    operationalise

    This would be a good

    companion to PnM 08

    (leading indicator

    thereof)

    Good idea for New

    Supplementary

    Compulsory in UK

    but might be checkedre e-learning

    Vital

    D

    R

    O

    O

    R

    NS

    NS

    27 .1) The institution should ensure

    that issues of staff workload

    are taken proper account of in

    the management of courses or

    programmes.

    .2) [split out from last

    paragraph] The institutionshould ensure that [and] any

    other implications of staffs

    participation in e-learning

    programmes (such as

    intellectual property rights over

    programme materials) are taken

    proper account of in the

    management of courses or

    programmes.

    Individuals

    staff

    11 (Workload)

    57 (IPR)

    R

    O

    28 Institutions should ensure that

    adequate administrative support

    (including effectivemanagement information

    systems) is available to

    academic staff, particularly

    part-time tutors/mentors.

    Individuals

    staff

    None Could be a New

    Supplementary

    O

  • 8/22/2019 E-xcellence benchmarks compared

    12/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 12 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    Student support

    29 Prospective students should beprovided with a clear picture of

    what will be involved in

    pursuing the e-learning

    programme and the expectations

    that will be placed on them.

    This should include information

    on technical (system and VLE)

    requirements, requirements

    concerning background

    knowledge and skills, the nature

    of the programme, the variety of

    learning methods to be used, the

    nature and extent of supportprovided, assessment

    requirements, fees, etc.

    Individualsstudents

    Plus

    alignment

    aspect?

    91 (Studentunderstanding)

    But a leadingindicator to PnM 91

    could add separately

    as a New

    Supplementary

    R NS

    30 E-learning students should be

    provided with the equivalent of

    a student handbook setting out

    their rights and responsibilities,

    those of their institution, a full

    description of their course or

    programme, and information on

    the ways in which they will be

    assessed.

    Individuals

    students

    Linked to 91

    (Student

    understanding)

    Well within QAA

    territory

    D

  • 8/22/2019 E-xcellence benchmarks compared

    13/13

    E-xcellence criteria analysed

    Paul Bacsich, BELA 13 27 November 2006

    No. E-xcellence criterion MIT90s

    category

    Nearest

    Pick&Mix

    criterion

    Commentary re

    Pick&Mix etc

    Rec.

    for

    MIT90

    s

    Rec.

    PnM

    supp.

    31 E-learning students should have

    access to learning resources andlearner support systems which,

    although they may be provided

    through different means, are the

    equivalent of those available to

    campus-based students. These

    should include:

    access to library resources

    support for the

    development of key skills

    (including support for e-

    learning skills,

    collaborative working on-line and contributing to on-

    line communities which are

    key skills in an e-learning

    context)

    advice and counselling overchoice of courses and

    progression through the

    programme

    an identified academic

    contact, tutor and/or mentor

    who will provide

    constructive feedback onacademic performance and

    progression

    access to help desk,

    administrative support and

    advisory services

    opportunities to provide

    and receive formal

    feedback on their

    experience on the course

    procedures to handle andresolve any difficulties or

    disputes which may arise.

    Individuals

    students

    51 (Uniformity) Close match

    Some of the criteria

    below may require

    separate treatment

    also

    R

    O

    32 Students should be provided

    with clear and up-to-date

    information on the range of

    support services available and

    how these may be accessed.

    Individuals

    students

    None Out of scope for e-

    learning

    benchmarking?

    D

    33 The expectations on students for

    their participation in the on-line

    community of learners needs to

    be made clear both in general

    terms and in relation to specific

    parts of their course or

    programme.

    Individuals

    students

    Linked to 91

    (Student

    understanding)

    Rather similar to 29

    above

    O