Available online at www.sciencedirect.com

1877–0428 © 2011 Published by Elsevier Ltd.doi:10.1016/j.sbspro.2011.03.161

Procedia Social and Behavioral Sciences 15 (2011) 667–671

WCES-2011

APPRAISALWEB: an online platform for the pedagogical evaluation of Web-based Language Learning Resources

Rafael Seiz-Ortiz a *, Ana Gimeno-Sanz a, Jose Macario de Siqueira a

a CAMILLE Research Group, Universidad Politécnica de Valencia, Camino de Vera 14, 46022 Valencia, Spain

Abstract

This paper presents APPRAISALWeb, a 2-year Research & Development project jointly funded by the Universidad Politécnica de Valencia and the regional government of Valencia (Spain), and developed by the CAMILLE Research Group at that university. In general terms, the project aims to suggest a comprehensive methodology for the pedagogical analysis of the Web as an environment for language learning. More specifically, a major objective of the project is to develop an online interactive platform that serves as a tool to analyze, classify and evaluate Web-based resources for language learning from a pedagogical point of view. As CALL (Computer Assisted Language Learning) research literature suggests, it is necessary to conduct a thorough pedagogical analysis of the Web and its CALL resources if we are to take full advantage of its many possibilities. Thus, the project meets two fundamental needs that have been extensively expressed in the area of CALL: the need for theoretical models and methodologies and the necessity of specific tools for analysing and searching for available Web-based CALL resources. First, the context of the project will be presented. Then the methodology behind APPRAISALWeb will be discussed. Finally, conclusions will be drawn from work in progress during the first stage of the project. © 2011 Published by Elsevier Ltd.

Keywords: Computer Assisted Language Learning (CALL); pedagogical analysis; Web-based language learning; e-learning; APPRAISALWeb Project.

1. Introduction

In the current context of technology-assisted learning, it is increasingly important to base the implementation of educational technology, regardless of its nature and format, on sound pedagogical principles and criteria, if we want to take full advantage of its educational potential. This general principle is particularly relevant within the relatively innovative field of Web-based Computer Assisted Language Learning (CALL). It becomes therefore necessary to undertake a profound analysis of the Web as a language learning environment, not only due to the novelty of associated technologies (social networks, virtual worlds, Computer Mediated Communication, etc.), but also and especially because of the existence of highly heterogeneous contents and the usual lack of quality control for such contents, among other drawbacks (Felix 2003). Consequently, this paper presents a research project related to the pedagogical analysis of Web-based CALL named APPRAISALWeb (Análisis Pedagógicos de Recursos de Aprendizaje Interactivo y Sistemas de Aprendizaje de Lenguas a través de la Web, i.e. Pedagogical Analysis of

* Rafael Seiz. Tel.: +34-963877007.Ext.: 75384; fax: +34-963877539. E-mail address: rseiz@idm.upv.es.

668 Rafael Seiz-Ortiz et al. / Procedia Social and Behavioral Sciences 15 (2011) 667–671

Web-based Language Learning Resources and Systems). The project is led by the CAMILLE research group (Computer Assisted Multimedia Interactive Language Learning Environments), at the Universidad Politécnica de Valencia, in Spain.

The aim of the project APPRAISALWeb is the development, implementation and social dissemination, on the one hand, of an efficient methodology to observe, analyse and evaluate Web-based Language Learning resources (for English, French and Catalan as a starting point) by applying pedagogical criteria, and, on the other hand, of a knowledge base called APPRAISALWEB (built on the work of a previous database called WIRESLABi), which will allow users to catalogue, describe and pedagogically analyse CALL resources currently available on the Web. In this way, a key objective would be to study both the educational possibilities of this environment in language learning and teaching, and the real picture of Web-based CALL through the actual pedagogical analysis and evaluation of currently available online resources.

The suggested methodology, based on a theoretical framework of pedagogical evaluation, aims at the development of a solid tool for the retrieval, classification and interpretation of pedagogical information on Web-based educational resources, consisting of a platform located on a website, which collects data to assess pedagogical and didactic issues of such resources. Since the foundations of this tool (the database called WIRESLAB, also developed within the CAMILLE research group by Rafael Seiz in 2006) are already set, one of the purposes of the present project is its thorough development and improvement so as to build up an online global knowledge base (APPRAISALWEB) dealing with pedagogical aspects of Web-based CALL resources, which is, in turn, openly available on a website for teachers, learners, researchers and developers. Both the pedagogical evaluation methodology and the tool will therefore assist the academic community and the society in the tasks of efficient search, classification, retrieval and use of language learning resources through the Web.

In this presentation of the project APPRAISALWeb, first, the context where it was developed will be discussed, and, then, its relevance as well as theoretical and practical basis, including objectives, purpose and expected results will be presented. Later on, some issues about the project methodology will be dealt with, since a brief discussion on such methodology could be of use in similar attempts to analyse the Web in effective language learning. In this section, we would also like to invite the involvement of the academic, teaching and research communities in the project’s objectives and outcomesii. Finally, some proposals are included for further research and follow-up of the work initiated in this project.

2. The APPRAISALWeb project

APPRAISALWeb is a R&D project carried out by the research group CAMILLEiii from the Universidad Politécnica de Valencia, which has been devoted for more than two decades to research on CALL, including a line of research on Web-based CALL. During the winter semester of the academic year 2009-10, the project obtained financial support both from the Universidad Politécnica de Valencia and the regional government of Valencia (Generalidad Valenciana), within the framework of their support to emerging research groups. The duration of the project is 2 years, starting in December 2009, so the tasks stated in its work plan are well under development. The research team consists of seven members: four researchers that belong to the teaching staff at the University’s Applied Linguistics Department, a research technician (with a double professional background as computer expert and academic researcher), a research assistant (computer technician trainee) and a research team coordinator with fully academic background. As a consequence, it is a multidisciplinary team that works in an independent yet coordinated way on the multifaceted aspects related with the twofold nature of the project, involving both computing and academic (mainly pedagogical) issues. This coordinated teamwork is carried out following the principles of the methodology called template approach as suggested for research and development work within the CALL domain by Ana Gimeno Sanz (2002), whereby team members with varying backgrounds and expertise work on different but related issues in an autonomous and coordinated way, thus providing each other with valuable feedback.

The CAMILLE research group has been working since 1993 on the development, implementation and evaluation of CALL resources and tools, both from a pedagogical and technical standpoint. It should be noted that CAMILLE

Rafael Seiz-Ortiz et al. / Procedia Social and Behavioral Sciences 15 (2011) 667–671 669

is interested in dealing with any aspect and phase of the process of CALL teaching and learning, from an interdisciplinary and global point of view: pedagogical and computer design, didactic implementation, materials development, evaluation and pedagogical analysis. Previous projects are focused on some of these aspects. APPRAISALWeb falls within this context, this time dealing with pedagogical analysis and evaluation of Web-based CALL, and will benefit from the work of previous projects, giving valuable feedback on former and future projects as well.

Within the field of language learning, the Web contains a huge amount of educational resources covering a wide range of contexts, formats, levels and pedagogical purposes. Current methods for information search and retrieval (the most common of which is Web search engines, particularly Google) do not allow users to conduct advanced searches of CALL resources based on pedagogical criteria and characteristics. They are of no use for the purpose of pedagogically analyzing and evaluating those resources, once they have been found, either. The APPRAISALWeb project, on the contrary, aims at meeting those demands, and to do so it develops: (1) a theoretical model for pedagogical evaluation; (2) a methodology for pedagogical analysis; and (3) an online computer tool (APPRAISALWeb) which is dynamic, open and interactive in order to implement that analysis of Web-based CALL resources, from a pedagogical perspective.

As can be seen in previous CALL literature, former theoretical models (Zhao 1996, Godwin-Jones 1999, Felix 2001, 2002, 2003, Hampel and Baber 2003, Chapelle 2001) that could be used to lay the foundations of the pedagogical evaluation suggested here, do not incorporate, in our view, the type of global, comprehensive and detailed view of the language learning process that we are willing to take in our approach. In the state-of-the-art of CALL nowadays there is not any resource or tool for retrieving and analysing pedagogical data of Web-based CALL as exhaustive, open and dynamic as the one suggested here.

The existence and availability of a Web-based resource (APPRAISALWeb) that is able to pedagogically catalogue, describe, analyse and evaluate language learning courses and resources on the Web, as well as conduct advanced searches of educational resources based on pedagogical criteria will be very useful to improve the knowledge of the potential of this environment and technology for language learning, to enhance the efficient pedagogical use of the resources, and to foster the reusability of language learning resources and Learning Objects available on the Web. The open nature of such resource and knowledge base (including the possibility of continuous updates and contributions from the academic and scientific community) will also enable the implementation of the concept of distributed knowledge. This will render benefits to the vast community of people interested in language learning: teachers, learners, researchers, developers, company representatives, publishing companies and educational institutions. The tools that will result from the project will also become interesting in the context of the common European framework of Higher Education, since they will enable European citizens (and also outside the continent) to use the Web efficiently for language (self-)learning and teaching. The initial hypotheses for this work are: (1) the Web is an environment where language learning and teaching can take place efficiently; (2) the Web implies some kind of innovation in language learning; and (3) to meet the previous demands, it is necessary to evaluate the resources pedagogically, which will be possible only if solid pedagogical criteria based on previous research are applied.

The general objective of the project, namely the pedagogical analysis of the Web as a feasible environment for efficient language learning, is then broken down in the following aspects, which are approached in a global manner: (1) to suggest a methodological model for the pedagogical analysis of Web-based CALL; (2) to develop a tool for analysing Web-based CALL resources from a pedagogical standpoint, consisting of a resource or knowledge base (APPRAISALWeb), based on a database (WIRESLAB), which is based on pedagogical criteria, easy to use, allows users to enter data in a collaborative way and is available to a large number of people through the Web; (3) to get to know and be able to use efficiently the Web environment and its language learning resources as an integrated part of the language learning process; and (4) to develop an open web-enabled resource, i.e. an interactive platform for the retrieval and processing of pedagogical data on Web-based CALL resources, potentially useful for several languages and a wide range of users, which will also benefit society as a whole and could be used in future R&D projects.

670 Rafael Seiz-Ortiz et al. / Procedia Social and Behavioral Sciences 15 (2011) 667–671

3. Methodology

The project incorporates both a theoretical and a practical side. On the theoretical side, a comprehensive state-of-the-art review of the CALL and CALL-related literature is carried out, in search for concrete pedagogical parameters that can inform the theoretical model and the computer-based evaluation tools. Part of such work has already been done within the CAMILLE research group (Seiz Ortiz 2006). At a more practical level, the project will: (1) develop computer tools (a database and a knowledge base, APPRAISALWeb, available on a website) in order to carry out the pedagogical analysis and evaluation of Web-based CALL resources; (2) build up a significant corpus of language learning resources on the Web (for English, Catalan and French); and (3) evaluate such a corpus by means of the previously developed model and the tools.

Due to the fully interdisciplinary nature of the project, teams will cooperate that belong to fields as diverse as computer science and applied linguistics, including researchers and teachers. Therefore, these groups will work partly independently and partly in a highly coordinated way. Computer experts will implement and develop the computer tools whose conceptual and pedagogical content will be carried out by teachers and researchers, and, therefore, direct and continuous contact between the two groups will be kept at all times.

At the final stages of the project, when the practical implementation, use and evaluation of the tool will take place, participation will be required of external people, such as learners, teachers, other computer experts and researchers. Contributions made by these people will be incorporated, through forms and questionnaires especially designed to that end, into the project’s website, from where access passwords will be assigned to the new users that will be responsible for evaluating both the platform and the language learning resources. For each one of the 4 groups mentioned above, a participation of between 20 and 40 people will be considered. Results from such evaluation of the APPRAISALWeb resource will be incorporated into the final version of the project website, when it becomes openly and freely available to the general public.

The work plan of the project has been established in 2 years and 3 general stages, distributed in the following way. During the first year, stage 1 of the project is undertaken, that is, the proposal of a theoretical model and the development of the tool WIRESLAB (the database), and stage 2 makes a start, consisting of the development of the online pedagogical evaluation platform called APPRAISALWeb. In the course of the second year of the project, stage 2 is completed, and stage 3 is carried out, with the practical implementation and evaluation of the tool APPRAISALWeb itself, as well as the dissemination of the projects outcomes.

4. Results of the project and future work

The project aims at the pursuit of a series of concrete results that will be of great academic and social interest. The most visible outcome is the APPRAISALWeb websiteiv, an open and interactive platform that will consist of a huge database, a knowledge base and evaluation centre for Web-based CALL, which will incorporate repositories of Web-based learning resources for different languages and will include advanced search facilities based on language learning pedagogical criteria. At a later stage in our research work, it will also incorporate a comprehensive and complete knowledge base devoted to the pedagogical study of the web for language learning. But, former stages such as the specialized research literature review and the proposal of an improved version of a theoretical framework for the exhaustive pedagogical analysis of Web-based CALL, can also be of interest for the profession.

Regarding the project’s dissemination, on the one hand, results and conclusions will be presented in several publications in scientific journals, professional magazines, online forums, blogs, discussion lists and similar professional communications facilities available on the Web. Outcomes will also be presented at specialized conferences. There will also be a publication at the end of the project in the form of a book summarizing the most remarkable academic issues raised by the project work, including reflections and insights that could be of help to similar forthcoming projects. Training activities will also be organised, such as workshops or seminars related with the use and implementation of the APPRAISALWeb resource and related tools, addressed at professionals interested in CALL.

Rafael Seiz-Ortiz et al. / Procedia Social and Behavioral Sciences 15 (2011) 667–671 671

We believe that the potential social interest of the project could result in agreements with companies and public or private institutions, in order to disseminate the results, methodologies and other relevant outcomes of the APPRAISALWeb project. Needless to say, researchers are welcome to participate through the tool website, submitting comments, suggestions and evaluations at a later stage of the project, when the website is fully operational.

5. Conclusions

Since this paper presents work which is under development, too definitive conclusions cannot be drawn yet. Nevertheless, we wish to point out the twofold need of suggesting methodologies and models solidly based in previous research and pedagogical criteria, when analyzing what the innovation of the Web can provide in the field of language learning, on the one hand, and, on the other hand, of developing efficient and easy-to-use tools to search for, classify, retrieve and reuse readily language learning resources available on the Web, thereby taking full advantage of the educational possibilities of such environment. Moreover, our proposal is to do so by carrying out effective multidisciplinary teamwork, the only suitable method when dealing with educational technology in general.

The project presented here has a remarkable social significance and interest, since it may assist all those interested in language learning (students, teachers, developers and researchers) in the efficient and pedagogical use of interactive language learning resources available nowadays on the Web, which otherwise would be difficult to be found, classified and used by the society.

Acknowledgements

The Project APPRAISALWeb is jointly funded by the Valencia Regional Government (Generalitat Valenciana. Project NUMBER GV/2010/070) and the Universidad Politécnica de Valencia (Vicerrectorado de Investigación). The authors are thankful to these institutions for their financial support.

References

Chapelle C. (2001a). Computer Applications in Second Language Acquisition. Foundations for Teaching, Testing and Research. Cambridge: Cambridge University Press.

Chapelle C. (2001b). "Innovative Language Learning: Achieving the Vision", ReCALL, 13/1: 3-14. Felix U. (2001), "A multivariate analysis of students’ experience of Web-based learning", Australian Journal of Educational Technology, 17/1:

21-36. <http://www.ascilite.org.au/ajet/ajet17/felix.html>. [Fecha de acceso: 2/3/2008]. Felix U. (2002), "The Web as a Vehicle for Constructivist Approaches in Language Teaching", ReCALL, 14/1: 2-15. Felix U. (ed.) (2003). Language Learning Online. Towards Best Practice. Lisse: Swets & Zeitlinger. Gimeno Sanz A. (2002). CALL Software Design and Implementation: The Template Approach. Valencia: Universidad Politécnica de Valencia. Godwin-Jones R. (1999), "Web Course Design and Creation for Language Learning", CALICO Journal, 17/1: 43-58. Hampel R., Baber E. (2003). "Using Internet-based Audio-graphic and Video Conferencing for Language Teaching and Learning", in Felix, U.

(ed.).Seiz Ortiz R. (2006). Análisis metodológico de cursos y recursos para el aprendizaje de inglés como segunda lengua a través de la World Wide

Web. Valencia: Editorial de la Universidad Politécnica de Valencia. Zhao Y. (1996), "Language Learning on the World Wide Web: Toward a Framework of Network Based CALL", CALICO Journal, 14/1: 37-51.

i A demo version of this database, developed at the Universidad Politécnica de Valencia (Seiz Ortiz 2006), can be accessed at: http://www.euita.upv.es/dla/wireslab/Default.asp ii The e-mail address wireslab@gmail.com may be used to contact Rafael Seiz Ortiz, the Project coordinator. iii For further information, please access the research group’s website at: http://camilleweb.upv.es/camille iv The Project website is currently under construction and available at the following address: http://camillegroup.upv.es/appraisalweb