BUILDING A VIRTUAL LEARNING SPACE FOR C&IT STAFF DEVELOPMENT

Márcia A. PereiraRachel A. Harris

Duncan DavidsonJennifer Niven

Centre for Open and Distance LearningThe Robert Gordon University.

______________________________________________________________Paper presented at the European Conference on Educational Research,

Edinburgh, 20–23 September 2000

To be published as part of the University of Sheffield 'Selected Papers in Education Series'

Contact details:

Dr. Márcia A. PereiraCentre for Open and Distance Learning

The Robert Gordon UniversityAberdeen AB10 1FR

Tel: 00 44 (0)1224 262235Fax: 00 44 (0)1224 262231

Email: m.pereira@rgu.ac.ukmarcia.pereira@pop.agri.ch

BUILDING A VIRTUAL LEARNING SPACE FOR C&IT STAFF DEVELOPMENT

Dr. Márcia A. Pereira, Dr. Rachel A. Harris, Duncan Davidson & Jennifer NivenCentre for Open and Distance Learning, The Robert Gordon University.

AbstractThis paper draws upon our experience in ongoing research on the development of a Virtual Learning Space (VLS) for communication and information technology (C&IT) related staff development at three educational institutions in the North East of Scotland: The Robert Gordon University, The University of Aberdeen and Aberdeen College. The project aims to construct a new integrated web environment which will facilitate the development of a partnership approach to C&IT staff development, allowing staff from the three institutions to learn via collaboration, exchanging expertise and resources.Using a participative and reflective methodology, the design of this virtual learning space considers users' needs and expectations, educational theories, and spatial issues, within a continuous reflective process.

Keywords: virtual learning environments design; online C&IT staff development; computer-mediated collaboration.

1. Introduction

1.1 Demand for C&IT staff developmentEducational institutions are called to adopt Communication and Information Technology (C&IT) to improve accessibility, quality and efficiency, supporting a 'lifelong society' as the Dearing Report recommends (Dearing, 1997). In introducing C&IT into education, one of the key issues to be taken into consideration is the development of C&IT skills and knowledge of staff in all levels, as recognised by the Dearing Report itself and others (Milligan, 1999, Atkins, 1998).

1.2 The Virtual Learning Space Project as an answer to this demandThe Virtual Learning Space (VLS) is a project funded by the Scottish Higher Education Funding Council (SHEFC) and it aims to develop C&IT skills of staff at three institutions in the North East of Scotland: The Robert Gordon University (RGU), University of Aberdeen and Aberdeen College. Developing a cross-institution learning community, the VLS will be a space where staff can share best practice, exchange ideas and explore new tools and material. Once populated with experiences, the VLS will act as a knowledge pool for a wide range of online C&IT staff development, addressing the needs identified by projects such as TALISMAN (Alexander, 1999) and SCAITS (Anstey, 2000). How can this be accomplished? How should be the process of building this learning environment in the most effective way? The VLS team experiences are discussed in this paper.

1

2. How to build the VLS?2.1 VLS Design: a methodology 2.1.1 The Design of Virtual Learning Environments (VLEs)Design is difficult to define and the activity of designing is seen in different ways by different people. This is true for software design also (Winograd, 1996), and more specifically for the design of educational software, where different approaches flourish. However, there is a tendency to follow the changes taking place in educational research, with a move from behaviourist and cognitive theories towards constructivist approaches. See Table 1.The Instructional System design approach, mainly based on passive transfer and where cognition is seen as a kind of computation, is criticised for not allowing interactive exploration of the learning experience in a critical and reflective way. It does not consider the complexity and uncertainties of practice (Crook, 1996; Merril, Li et al, 1990; Owen, 1999).An approach which is learner orientated, based on constructivist and situated theories, encourages the design of virtual learning environments (VLEs), where different tools can be explored by the learner. Of the design models proposed within this approach, participatory design was considered to be one of the most appropriate to the VLS project, as we wanted to design a community space where users would be willing to participate and interact. It is also argued that the design of a virtual environment could benefit from the experience which architecture and architects have with the design of physical spaces. This was previously suggested by others such as Bridges and Charitos (1996), Cabral-Filho (1996), Kapor (1996), and also by one of the authors elsewhere (Pereira, 2000). Important issues considered when designing real physical environments could also be considered when designing virtual spaces, e.g. Vitruvius’ attributes of good architecture: commodity, firmness and delight (Vitruvius, 1960); Lynch’s concepts of paths, nodes, boundaries, and landmarks (Lynch, 1960), which are responsible for the ‘legibility’ of the space; or Hertzberger’s discussions on territoriality, form and appropriation, accommodating and stimulating spaces (1991). Attempting to develop these qualities in virtual environments could help users to develop certain familiarity with the virtual spaces and feel safe and more confident when using them.2.1.2 The VLS Design MethodologyBased on discussion of the approaches available, it was decided that the design of the VLS should adopt a combination of approaches within the realm of socio-constructivism, combining participative design and activity theory, with an architecturally-informed methodology, based on reflective practice. Architectural design methodology involves reflective processes, where reflection-in-action and reflection-on-action take place. This process was described and discussed by Donald Schön (1985, 1987). A ‘reflective process’ model to explain Schön’s ideas was proposed by Pereira (1999) and is shown in Figure 1. Schön also recommends this same process to other professionals. This process seems ideal for any design process, what suggests its usefulness also when designing virtual learning environments. An attempt to describe a reflective process for the development of the VLS is shown in Figure 5 (Section 2.6), incorporating several evaluative stages.

2

Table 1: Different approaches to the design of learning experiences according to different educational paradigms (adapted from Pereira, 2000):Theory View of learning Learning activities Design ModelsBehaviourism  and Information  Processing  Theories  (Cognitive  Theories)

The learning process involves outer stimuli and inner conditions: ‘Learning is a relatively permanent change of behaviour that arises based on experience.’ (Hilgard and Atkinson, 1967)

Trainer oriented Highly structured Based on drill and practice, passive transfer and summative evaluation.

Instructional System Design is a concept used to describe models based on these approaches, which separate the content of the teaching and the methods.

Constructivist Theories and Situated Learning

This approach considers the situated character of practice and seeks to the construction of knowledge by the learner, emphasising the importance of social interaction.

Learner orientedFlexibly structuredHighly interactiveExploratoryProject basedCollaborativeCritically reflective

Within this approach, other theories, such as socio-cultural theory and critical pedagogy complement the constructivist view by emphasising the social character of learning and the need for critical reflection, embedded in a meaningful context. Some of the design models proposed within this approach are: Cognitive apprenticeship; Pedagogic tools; Participatory design; Problem-based learning

Figure 1: The process of Reflective Practice (from Pereira, 1999)

3

2.2 The VLS Design Process

2.2.1 A Conceptual FrameworkAn initial framework on which to base the development of the project was felt necessary. Based on the work of Pereira (2000), the initial framework comes from the view that successful adoption of C&IT by educational institutions involves the consideration of three interrelated clusters of issues: pedagogical, organisational and technical. See Figure 3.

Figure3: VLS framework - balancing pedagogical, organisational, and technical aspects.

PEDAGOGICAL

ORGANISATIONALTECHNICAL

2.2.2 Needs IdentificationSpecial consideration was given to users' needs and expectations. They were mainly identified via a literature review, analysis of similar projects, a C&IT skills survey via a questionnaire sent to all staff at RGU, and through focus groups with academic and non-academic staff from the three partner institutions. The focus groups were particularly important, as they represented an opportunity to have direct contact with future users of the VLS. During these focus groups, participants were asked to discuss their needs in terms of C&IT staff development and also their expectations towards the VLS in small groups and in plenary. An electronic tool, named Delphi tool, was especially developed to be used as one of the last activities of the meeting. Using the Delphi tool participants could submit their opinions on what would motivate their participation in a learning environment as the VLS and rate their own and others' opinions online and immediately see the results. This was used not just to collect their feedback, but also to show them an example of how C&IT could be used. The needs identification stage is explained in more detail elsewhere (see Harris at al., 2000), but the main findings are highlighted in Table 2.

4

Table 2: Questionnaires and Focus Groups for needs identification.Method C&IT Questionnaires Focus GroupsParticipation Sent to all RGU staff and all Aberdeen

College staffResponse from RGU of around one third (n=446)Response form Ab.Col (more than 50%): it is being analysed.

Academic and non-academic staff from RGU, Aberdeen University and Aberdeen College

Main Findings Respondents have a high level of competence in applications such as word-processing and spreadsheet packages and over 50% consider themselves to be at least intermediate users of web browsers.

Training is not seen as a priority, mainly due to time constraints (as confirmed by focus groups), emphasising the need for awareness development as identified by TALISMAN (Tomes & Higgison, 1998).

There is a need for more intermediate and advanced development in 'standard' C&IT skills, and more introductory support in new skills, such as web page creation.

Most common needs: The ability to exchange experiences and ideas

via a supportive community Recognition of time constraints Provision of advanced training Motivation Motivating factors: user friendliness; online support; a reliable pool of resources that provides

solutions to common problems. Most cited staff expectations towards the VLS in terms of its qualities: A supportive community that allows sharing

of experiences, learning from others and reassurance

Consultant type of advice that is concise and immediately accessible.

The initial VLS framework was used to encourage discussion about users' needs. The way the findings were arranged around the three clusters shows the importance of considering pedagogical, organisational and technical issues in a balanced way and how they are interrelated. See Figure 3. Figure 3: Focus Groups findings.

5

2.2.3 Identification of Activities and Qualities of the SpaceAs previously mentioned we attempted to use an architecturally informed methodology for the design of the VLS, which combines participatory design with reflective practice and spatial issues. Therefore, having identified the users’ needs, it was thought that the following should be defined: the main activities which would take place within the VLS; the actors who will be involved in these activities; the objects of these activities; the actions which should be taken to promote these activities; the qualities of the VLS space which would favour these actions the functionality which should be developed to promote these activities.The VLS team, composed of four people, is quite multidisciplinary with backgrounds ranging from architecture, education, physiology and psychology to information studies and computer science. This is important to provide a rich discussion environment. Brainstorming sessions were organised, where the whole team discussed the findings and went through defining the points cited above. We started discussing the most cited issue, which is the need for a supportive and collaborative community. Through this discussion we identified the need to think about actions which should be taken outside the VLS as well. Therefore, we decided to add this item to our discussion. Tables were used to organise the issues in different ways and to facilitate and register the results of our discussions. They were: General tables for analysis of activities according to the identified needs. See Table 3

for an example. Tables of activities according to each actor. See Table 4 for an example. Tables of 'action' or 'content' which we ideally would have in the VLS, together with

its qualities. These are shown in Table 5. Tables of desirable qualities of the VLS, showing the functionality and main features

of the space, as shown in Table 6.

Table 3: Activity analysis based on the need to generate 'communication via a supportive community'.Users' Needs

Activities Qualities of the Space

Actors Objects of action

Actions Functionality Actions outside the VLS

Communication via a supportive community

DiscussionExchange of materialProviding feedbackModerationExchange of expertise

Easy to useWell organised/structuredHaving appropriate toolsReliable technologyGood admin

Lecturers ResearchersAdminLibrariansSupport staff:Staff developersIT ServicesProject teamModeratorsAdministratorsFacilitatorsLearners

TipsExamples of best practiceCase studiesGuidelinesExperienceManuals/proceduresOpinions

ExplorationEncouragementAccumulation of resourcesAppropriation of the space

IndexingSearching facilitiesNewsletterFocus groups (online)Eventse.g. e-workshopLink between discussion and resources poolBiblioscape? Personalisation tool?

Face-to-face focus groupsMailing listWorkshopsMeetings with staff developers and trainers

6

Table 4: Lecturers' needs activities they would perform in the VLS.Actor Needs ActivitiesLecturer Communication via a

supportive/collaborative community

Exchange of expertise Time constraints Advanced Training Main concern: making

material available for students on the Web

Online assessment Specialized IT skills &

software Access to specialists Motivation Support (institutional,

pedagogical, technical) Combination of innovative

teaching and research

Finding information & learning about: how to put teaching material on the Web; online assessment; specialized IT skills & software; via:

1.Accessing software (how to use, where to find, how to access); 2.Accessing evaluation of software & equipment3.Accessing evaluation of others’ experience/case studies

Exchanging experience and communicating ideas Contacting specialists Publishing experience/research on teaching (e.g.

online publication associated with online discussion) Searching & selecting information/resources

efficiently Exchanging material Exchanging expertise Providing feedback Moderating Using efficient search facilities/index and having

access to well structured content

Table 5: Objects of action or content of the VLS and its qualities.Objects of action/Content: Qualities of the content: Advanced training Accessibility Accreditation Appraisal schemes and rewards Combination of innovative teaching and research Making material available for students on the Web Motivational issues (e.g. how to encourage students

to use IT resources) Online assessment Pedagogical support (via discussions and resources) Plagiarism/cheating Special needs training Specialised IT skills and software List of specialists – pockets of expertise Resources pool Frequently asked questions and answers Useful links to other sites Common sources Suggestions of courses, strategies, material and

locations. Case studies/best practice Useful links to others with similar interests Open discussions Specific discussions Solutions to common problems Institutional support issues (e.g. institutional

readiness) Good examples, to reduce skepticism (e.g. examples

of C&IT use in teaching, which reduces administrative time)

Self-assessment Reflective discussions/conversations Directive answers

comprehensive in small chunks and digested form; concise. reliable relevant/useful reflective critical mass of information and activity directive good quality

7

Table 6: 'Qualities of the space', 'functionality' and 'main features'.Qualities of the space: Functionality/Functional requirements: Main features: easy to use/user-friendly supportive, reassuring facilitates communication:

sharing experiences and learning from others

up to date reliable functional well organised/structured inclusiveness accessibility ‘continuity’ fun and exciting

Interactivity (with content and other individuals)

Easy maintenance: easily updateable, in terms of content and technology

Consultant type of advice, immediately accessible

Help desk type of advice Meaningful routes to find

information Use of appropriate tools Use of reliable technology Well administered Personal involvement and

management of the system (human administrator(s)/moderator(s)

Highly accessible Allowing constant evaluation Efficient access to material/info

(quick access to relevant material)

Index (resources & site) Search (site & WWW) Newsletter (periodically) News Board (recent news) Chat (scheduled and informal

discussions) Discussion groups or For a

(threaded and non-threaded) Whiteboard/NetMeeting (sharing

material/applications/presentations synchronously)

Calendar Scheduling facility Poll facility Registration facility Uploading and/or submitting

facility Bibliography Personal profile Customisation facility (choice of

different metaphors?) Delphi tool

2.3 VLS Initial Implementation

2.3.1 Web tools & VLEs evaluation within the framework of staff needsSeveral web tools and learning environments have been analysed during the development of the project. This analysis has been done by combining our own framework with different evaluation studies, which focus on features, technical aspects and costs (such as the Landonline Review http://www.ctt.bc.ca/landonline/reviews.html) and more theoretical approaches that incorporate pedagogical and also some organisational issues (Britain & Liber, 1999). The material generated by these evaluations will be added to the VLS resources.None of the VLEs examined in the studies previously described had all of the qualities thought to be necessary for an effective VLS for C&IT staff development. It was therefore decided to attempt to combine features from different tools into an environment that would provide a flexible basis for further development. Studies have started on the implementation of a version of RGU’s Virtual Campus. This has the advantage of providing the project team with easy access to the developers of this environment, and could also facilitate the continuity of the VLS beyond the lifetime of the project. The emphasis should, however, be on developing the VLS via a continuous process of evaluation, which involves the participation of users from the three partner institutions.

2.3.2 VLS StructureThe main objective of the VLS is to develop a supportive learning community, where people will learn from each other, sharing ideas and resources. Therefore, based on the previously cited findings, we thought that the use of a metaphor which relates to a physical community would be the most appropriate. This would also help users to recognise the spaces and connect their functionality with the functions of the real spaces.

8

Therefore, users would easily understand and use the learning environment. It was then decided that we should have a main entrance and two main environments where users could interact with each other and with resources; and a third space which would be a reflective space for users to gather and organise information, and keep their own data. The main structure was then developed as it is shown in Figure 4.

Figure 4 - VLS structure.

User

registrationnewsboard/newsletterCalendar/schedulingfacility

LINKS DOCUMENTS

PedagogicalsupportresourcesCase studiesSpecialistsOnlinejournals…

PapersFAQSolutions tocommonproblems…

FORUMS

CAFÉ

CHAT ROOMS

threadedNon-threaded

Informal discussion

(bysubject/themes)

PERSONALPROFILE

PORTFOLIO

CUSTOMISATIONFACILITY

SELF-TESTRESULTS

PollNotepadSelf-testI Know

PORTAL

PERSONALSPACE

RESOURCESCENTRE

COMMUNITY

9

2.3.3 Navigation & Contents OrganisationA good navigation system is essential as it is cited by one of the authors elsewhere:Navigation is a very important feature of any virtual environment, as much of the reaction of users towards a system is on the facility to go around it. This is related to the issues of territoriality and legibility in architecture. A positive feature of an environment is its legibility, as having well defined marks and paths promote orientation and give a feeling of emotional security. Providing identifiable territories gives also the basis for identity, security and stimulation, while the opposite causes frustration, anxiety and stress. (Pereira, 2000)With this in mind, the VLS interface was organised around three frames: The first frame, located at the top of the screen does not change, when the user

navigates around. It contained four icones that compose the main menu and give access to the four main areas of the VLS, together with access to general tools which could be needed at any time. This general tools are: help, notepad, search, sitemap and 'contact us'.

The second frame, located on the left hand side, changes when the user is in different 'spaces' or 'modes' of the VLS. It contained menus which are specific to each main space.

The third frame is the one where the content is presented and the main actions take place. This changes all the time. See Figure 5 or http://itlearningspace-scot.ac.uk.

In order to improve the 'legibility' of the space, the main areas within the VLS had names which referred to real spaces and their icons contain photographs of real spaces as well. The activities taking place in these four main spaces were: Gateway: is the main entrance to the VLS and contains general information,

registration, access to the main spaces and general tools. Community: is where people get together to exchange ideas and experiences. It is also

where events like workshops take place. Discovery Centre: is where resources for developing C&IT skills can be found,

including courses, useful links, references, a bookshop and where users can find the resources for the workshops held on the VLS.

My Place: is the user's private area where s/he can create and edit their profile, choose preferences, keep an address book, personal resources, notes and discussions.

2.4 Continuous evaluation

2.4.1 VLS Development PlanCoherent with the reflective practice basis, the project proposes an ongoing evaluation, where users' needs and use of the space will be constantly evaluated and provide feedback for the development of the VLS. See the development plan (Figure 6). The initial implementation was again evaluated via an international online workshop which took place on the VLS and via focus groups with staff from the three partner institutions.

2.4.2 OTiS international e-workshopThe VLS project team worked together with the Online Tutoring Skills Project (OTiS) team, supporting and hosting the OTiS e-Workshop in May 2000. This was an international online workshop where experiences of online tutoring were exchanged. The

10

OTiS project aims to establish a process, guidelines and knowledge pool of resources for online tutoring, therefore sharing some of the aims of the VLS. The e-workshop was then used as a pilot test for the initial implementation of the VLS, providing also an opportunity to generate resources and disseminate the space. After the international e-workshop an email questionnaire was sent to all participants. Post-workshop evaluative chats were also organised, combined with the generation of online discussion groups for evaluation of the VLS, within its own space.

Figure 5: The VLS interface - initial implementation.

2.4.3 Focus GroupsFocus groups with academic and non-academic staff from the three institutions were organised. An update on developments of the VLS was presented and participants had the opportunity to use the environment and evaluate it via the discussion rooms. An evaluation of the qualities of the VLS also took place via the use of the Delphi tool. Their feedback was also discussed in groups and in plenary.

2.4.4 VLS re-designFindings from the VLS-OTiS workshop evaluation questionnaire, focus-groups feedback, VLS-OTiS workshop chats & discussion rooms were qualitatively and quantitatively analysed, generating useful material. Most of the feedback was positive and participants said they had experienced learning and enjoyed using the environment. There was also useful criticism and suggestions for improvement and new tools to be added. These were organised in the following issues: content, interface & navigation, organisation, resources, security, technical issues, tools & features.

11

These findings are being used as a basis for the redesign of the VLS, which is currently taking place. We plan to have the space ready to be launched at the beginning of November 2000.

Figure 6: VLS development plan

NEEDSIDEN TIFIC ATIO N

VLSIM PLEM ENTATIO N

EVALUATIO N

FURTHER

DEVELO PM ENTS

EVALUATIO N

DEVELO PM ENT

PLA N

C O NTINUITY

LITERATURESIM ILAR

PRO J EC TS

C & IT

SKILLS

SURVEYFO C US

G RO UPSWEB

TO O LS

EVALUATIO N

C O NVERTIN G

FIN DING SC O NC EPTUAL

M O D ELD ESIG N

G EN ERATIO N

O F

RESO URC ESPO PULATIO NTESTD ISSEM INATIO N

VLS DESIG N

DEVELO PM ENT &

PILO T: O TIS e -WO RKSHO P FO C US G RO UPSPO PULATIO NTESTDISSEM IN ATIO N

PO PULATIO NDISSEM IN ATIO N

RE-D ESIG N & IM PLEM EN TATIO NM EETIN G S, WO RKSHO PSIN TEG RATIO N IN TO EXISTIN G STA FF DEVELO PM EN T

&

N EWSLETTER ARTIC LES, C O N FEREN C ESPRO M O TIO NAL M ATERIAL

O NG O IN G

O NG O ING

12

2.4.5 Population & disseminationWithin the integrative framework attention to population and dissemination of the VLS is very important. Population takes place regarding resources and participants. According to the users' feedback from the focus groups specifying which kind of material they would like to have available on the VLS, the team have been compiling resources. These resources have been organised in categories, so they can be easily archived and accessed via searching or browsing. The resources obtained via the e-Workshop, composed by case studies, recorded chats, discussions and suggestions of related bibliography and URLs are being revised and processed at the present and shall constitute an important part of the VLS collection.The strategy to attract participants and disseminate the space has been in effect since the first series of focus groups, being reinforced by the second series. The OTiS e-workshop also helped to populate the VLS with participants, giving it a more international flavour, and collaborating for its dissemination. Other forms of dissemination and population have been arranged via meetings with staff developers and trainers to discuss how the VLS could best help them and how best to integrate with existing developments.Other forms of dissemination are participation in conferences and workshops, publication of articles in institutions' internal newsletters, the VLS mailing list, the VLS web site, the VLS Newsletter, and project reports. We are also planning to produce promotional material for distribution in the three institutions and elsewhere.

2.4.6. ContinuitySeveral possibilities for the continuity of the VLS are under study and will be further developed from December 2000. The present ideas are based on the need for institutional integration and on the need to encourage the community of users to appropriate this Virtual Learning Space.

3. Conclusion

Taking into consideration the need for C&IT staff development in educational institutions, this paper presents the process of designing a virtual learning space where this development can take place. The importance of the design process it advocates resides on its reflective characteristics based on the users' participation in a collaborative and reflective process. The participative and reflective methodology adopted for the design of the VLS is based on a constructivist approach and considers architectural issues of spatial use. It incorporates the analysis of activities within the virtual space and of the qualities this space has to have to allow and encourage its most efficient and enjoyable use. Evaluative stages are incorporated into the process and become part of the environment itself.We argue, that in adopting such a design process, developers and users are more apt to come to a more effective and efficient product, whose ownership is shared.

13

4. ReferencesAlexander, W. (1999) TALISMAN C&IT Review. http://www.talisman.hw.ac.uk/CITreview

Anstey, P. (2000) C&IT Skills: Developing staff C&IT capability in Higher Education. SCAITS JCALT Final Report. http://www.uea.ac.uk/csed/scaits/

Atkins (1998) An Evaluation of the Computers in Teaching Initiative and the Teaching and Learning Technology Support Network. (HEFCE Report 98/47): http://www.niss.ac.uk/education/hefce/pub98/98_47.html

Bridges, A. and D. Charitos (1996). “On architectural design in virtual environments”. Creativity and Cognition' 96: 184-191.

Cabral-Filho, J. S. (1996). Formal games and interactive design: computers as formal devices for informal interaction between clients and architects. Ph.D thesis. Sheffield, University of Sheffield.

Britain, S. and Liber, O. (1999) A Pedagogical Framework for the Evaluation of Virtual Learning Environments. Bangor, University of Wales: http://www.jtap.ac.uk

Crook, C. (1996) Computers and the collaborative experience of learning. London, Routledge.

Dearing, R. and The National Commission for Investigation in Higher Education (1997) Higher Education in the learning society. http://www.ex.ac.uk/dearing.html

Harris, R., Pereira, M.A. & Davidson, D. (2000). 'Identifying the qualities needed for a virtual learning space in communication and information technology skills'. Proceedings of the 2000 International Networked Learning Conference jointly organised by Lancaster University and the University of Sheffield. Lancaster, April 17th-19th 2000.

Hertzberger, H. (1991. Lessons for students in architecture. Rotterdam, Uitgeverij 010.

Hilgard, E.R. and Atkinson, R.C. (1967) Introduction to psychology. New York: arcourt, Brace & World. 4th ed.

Kapor, M. (1996) "A software design manifesto". In Winograd, T., Bennett, J., De Young, L. and Hartfield, B. (eds) Bringing design to software. Reading, MA: ACM Press, pp. 1-9.

Lynch, K. (1960). The image of the city. Cambridge, Mass., M.I.T. Press.

Merril, M.D., Li, Z. et al. (1990) "Second generation instructional design". Education Technology 30(2): 7-14.

Milligan, C. (1998) The role of VLEs in on-line delivery of staff development. JTAP Report 573. http://www.icbl.hw.ac.uk/jtap-573.

Owen, M. (1999) Paradigms for curriculum design: The design of reflective, situated, collaborative professional development supported by virtual learning environments. Bangor, School of Education, University of Wales.

Pereira, M.A. (2000) ArchCAL: a conceptual basis for the application of information technology into learning and teaching technical subjects in architectural education. Ph.D Thesis. Sheffield, University of Sheffield.

Pereira, M.A. (1999) ‘My reflective practice as research’, Teaching in Higher Education, Vol 4, No.3.

Tomes, N. and Higgison, C.A. (1998) TALISMAN Training Needs Analysis. A Report for The Scottish Higher Education Funding Council. http://www.talisman.hw.ac.uk/

Vitruvius (1960). The ten books on architecture. New York, Dover Publications.

Winograd, T., Bennett, J., De Young, L., Hartfield, B. (eds) (1996) Bringing design to software. Reading, MA: ACM Press.

14