Ph.D. Thesis - DTIC

UNIVERSIDAD DE VALLADOLID Dpto. de Teoría de la Señal, Comunicaciones e Ingeniería Telemática

Escuela Técnica Superior de Ingenieros de Telecomunicación

Tesis Doctoral

A pattern-based design process for the

creation of CSCL macro-scripts

computationally represented with IMS LD

AUTORA

Davinia Hernández Leo

Ingeniera de Telecomunicación

DIRECTORES

Juan I. Asensio Pérez

Dr. Ingeniero de Telecomunicación

Ioannis Dimitriadis Damoulis

Dr. Ingeniero de Telecomunicación

Abril de 2007

i

European Ph.D. Dissertation

Defense: June 25th, 2007

Advisors:

Dr. Juan I. Asensio Pérez & Dr. Ioannis Dimitriadis, University of Valladolid, Spain

Three-month research visit in another European country, advisors:

Dr. Colin Tattersall & Dr. Daniel Burgos, Open University of the Netherlands, The Netherlands

Reviewers:

Dr. Hans Hummel, Open University of the Netherlands, The Netherlands

Dr. Jan-Willem Strijbos, Leiden University, The Netherlands

Committee:

Prof. Dr. Carlos Delgado Kloos, University Carlos III of Madrid, Spain

Prof. Dr. Rob Koper, Open University of the Netherlands, The Netherlands

Prof. Dr. Josep Blat, Pompeu Fabra University, Spain

Dr. Andreas Harrer, University of Duisburg-Essen, Germany

Dr. Eduardo Gómez Sánchez, University of Valladolid, Spain

A PATTERN-BASED DESIGN PROCESS FOR CSCL SCRIPTS REPRESENTED WITH IMS LD

ii

iii

ACKNOWLEDGMENTS

It is not possible to separate this Ph.D. Thesis from its social context. My work originated as an

external form of activity before definitively turning to what you can read in this manuscript.

Recalling Vygotsky’s words, I like to think that my contributions are the products of a process of

integration in a knowledge community. The community includes numerous researchers who have

inspired my work. I would like to express my gratitude to all the authors mentioned in the reference

list provided at the end of the thesis. It is also now my great pleasure to take this opportunity to

thank the many people that have personally supported this work one way or another.

I wish to gratefully acknowledge the enthusiastic supervision of Yannis Dimitriadis and Juan I.

Asensio. I cannot thank Yannis enough for offering me an opportunity for initiating research work

in the field of CSCL when I was still an undergraduate student. His guidance on the research topic

and research in general have offered me five years of really gratifying work and have lead me to

become an ardent supporter of CSCL as a means of ultimately achieving a more humanizing society.

I am also deeply indebted to Asen for his attention, immeasurable help, endless discussions, and

keen support. Not only has he extremely influenced the results of this thesis but he has also shown

me the value of the educational experience during the three years that I serve as a teacher at the

School of Telecommunications of the University of Valladolid.

I would like to thank the rest of the members of the multidisciplinary GSIC/EMIC group who

have accompanied and supported me during these years. I have been privileged to conduct my

thesis in such an outstanding social context. I am grateful to Bart, Inés and especially Iván for their

important indications while discussing my work. Their contributions to the design of Collage and

their help in the evaluation phase of the thesis have been significantly useful in shaping this thesis

into completion. The work of Miguel on the Gridcole system has also been very important to

complement and evaluate my contributions. I have been also fortunate to supervise a diploma thesis

of a brilliant student. I sincerely thank Eloy for playing a key role in the development of Collage and

I wish him all the best for the future. Four students are currently making real the future work of this

thesis, extending Collage or developing new related tools. Thanks to Julio, David, Susana and

Vanesa for their enthusiasm carrying out this work. Sincere thanks as well to Alejandra, Eduardo,

Rocío, Guillermo, Miguel Ángel, José Antonio, Guillermo Y., Sara, Sara 2, Sergio, Luis, Momo, Fede,

Óscar, Mendi and Luismi. I would also like to thank Rodri for their help with the University

procedures during the last weeks of finishing this thesis.

A Research Fellowship Program of the University of Valladolid facilitated me a visiting Research

Fellow at the Educational Technology Expertise Centre of the Open University of the Netherlands

(OUNL), where I collaborated with Daniel Burgos, Colin Tattersall and Rob Koper. I wish to express

my sincere thanks and appreciation to them. The three months in Heerlen offered me a great

opportunity for finalizing this work and for meeting extraordinary people. I had insightful

discussions with Hans Hummel, Hubert Vogten and Paul Kirschner, and many informal

conversations with a new circle of friends: Gemma, Ellen, Malik, Jose, Christian, Francis, Youngwu,

Hendrik, Tim, Marco, Liesbeth, Tamara, Bas… I do not want to forget the invaluable help of the

secretaries; thanks to Mieke and Audry. I have been especially fortunate to share my months in the

Netherlands with Juanma Dodero. He has helped me immensely by giving me encouragement and

friendship. He has also collaborated in the ideas, reflected in this thesis, around the integration of

learning design solutions formalized with different languages.


iv

There are three special projects which have enormously influenced my work. I am grateful to the

communities of UNFOLD. From the many participants in the UNFOLD events that have supported

me, I would like to specially stress the role of Dai Griffiths, Josep Blat, Rocío García, Ana Dias, Bill

Oliver, Chris Crew, Griff Richards, Daniel Burgos, and Rob Koper. Participating in the VDS

workshops organized by the CoSSICLE ERT of Kaleidoscope has been also extraordinarily useful for

achieving the results of the thesis. Sincere thanks to all the members of the ERT and especially to

Andreas Harrer, who has provided me feedback at different stages of the thesis and kindly accepted

to collaborate with me in shaping the create-by-reuse framework included in this manuscript.

Moreover, the work accomplished under TELL project has offered me valuable insight around

design patterns. Particularly, the constructive comments of Simos Retalis have greatly improved

this work. I would like also to thank the (co-)recognition of the “2006-2007 European CSCL Award

for Excellence in the Field of CSCL Technology” and the “ICALT 2004 best paper award”

committees; both awards have greatly stimulated me.

This research would have not been possible without the participants in the experiences studied

for the evaluation of the thesis. Special words of thanks to the students of TTG and CTM2 as well as

to the teachers of the University of Valladolid and the University of Cádiz who participated in

Collage workshops. Sincere thanks to Gregorio Rodriguez for showing interest and contacting me to

experience the ideas reflected in Collage. The workshop and panel organized by the laboratories

Syscom (Université de Savoie, France) and CLIPS-IMAG (Grenoble, France) within the ICALT 2006

conference also represent an important element of the evaluation accomplished in this work. I

would like to especially thank Laurence Vignollet and Jean-Pierre David for sending me the video

recorded during the panel.

I am grateful to all my friends from University, Cristina, Noemí, Rocío, Susana, Laura, Bea, Crix;

and from Plasencia, Rita Mª, Virginia, Laura, Araceli and Cristina for their understanding and

friendship during all these years. I wish special success with their thesis to Noemí and Rocío. I am

sure they will accomplish an excellent research. I am also indebted to the rest of the members of the

management team of the University Student Residence “Colegio Mayor María de Molina”. I have

been fortunate to collaborate with Mariví, Ana, Ana Mª, Avelina, Fely and Angelines, who have been

my surrogate family during the last two years and have allowed me to enjoy an enriching experience

beyond my duties at the University. I especially thank their care and attention.

I remain indebted to my parents and my bother for their understanding and patience when it

was most required as well as for their enormous effort to ensure that I had an excellent education. I

would also like to thank the rest of my family in which I include David’s family, who has shown a

continuous interest in my evolution throughout the thesis. More than ever, I would like to give my

heartfelt thanks to David, for his endless love and encouragement. He has been the most important

support to complete this work.

Davinia Hernández-Leo April 2007

v

To my parents,

the best collaboration practice that I know of

To my brother

To David


vi

vii

Abstract

Information and Communication Technologies (ICT) in Computer-Supported Collaborative Learning (CSCL) are mainly used for mediating social interactions as key activators of learning. One of the major concerns of CSCL is however that free collaboration does not necessarily produce learning and that in several circumstances collaboration should be scaffolded so that the probability of reaching successful outcomes increases. CSCL scripts embedded in ICT systems aim at shaping the way learners interact with each other in order to elicit fruitful interactions. The specific focus of this Ph.D. Thesis is on CSCL macro-scripts which describe pedagogical methods defining flows of coarse-grained activities. This document identifies and faces up to three challenges around the problem of facilitating teachers the design of those ICT-embedded CSCL macro-scripts.

The first challenge refers to the design of the potentially fruitful scripts. This work proposes the use of patterns to capture good practices in structuring CSCL situations for the purpose of reusing them in the design of new scripts. In this sense, we present a conceptual model for CSCL scripting pattern languages and a specific pattern language that is compliant with the model. The model defines the different types of patterns and relationships among them so that it is possible to specify numerous meaningful sequences of patterns that shape the design of specific scripts.

The second challenge deals with the implementation of the scripts in ICT systems. With the aim that the scripts can be automatically interpreted without the need of developing new systems, we propose the use of IMS Learning Design (LD) specification to computationally represent the macro-scripts. This approach fosters interoperability and enables teachers participate in the design of the behaviour and functionality of the systems by providing a script adapted according to their particular situations. This work analyzes the support of this educational modelling language for expressing CSCL scripts considering the possibilities of the LD notation but also the use of related specifications and tooling.

The combination of the previous proposals enables us to propose a pattern-based design process for the creation of CSCL macro-scripts computationally represented with LD. The specific patterns considered in the approach are the so-called Collaborative Learning Flow Patterns (CLFPs), a particular type of CSCL scripting patterns that suggest generalized structures of macro-scripts. The main goal of the design process is twofold. On the one hand, it aims at enabling the conceptualization of the expected interaction focusing on CSCL critical elements through the refinement of CLFP-based templates. And on the other hand, it intends facilitating the teacher-friendly creation of LD-represented scripts by hiding LD details; thus facing up to the third challenge related to the fact that computational representations are not familiar to the majority of the teachers. The design process is implemented in an authoring tool (named Collage) which proves its feasibility and enables its proper evaluation.

Overall, the applied research methodology is characterized by the multidisciplinary problem domain within which the dissertation is framed. Particularly, the evaluation phase is accomplished by means of a multicase study that comprises three case studies, which aim at assessing the same contributions but from different perspectives. The cases involve workshops with the target audience (teachers interested in applying CSCL) and experiences with students in authentic situations; but they also involve experts in the collaborative learning or LD fields and researchers proposing related approaches. The results of the evaluation not only show that the objectives of the dissertation have been achieved but they also offer relevant clues for future research directions.


viii

ix

Resumen

Las Tecnologías de la Información y las Comunicaciones (TIC) se utilizan principalmente en el campo del Aprendizaje Colaborativo Apoyado por Ordenador (Computer-Supported Collaborative Learning, CSCL) para mediar interacciones sociales como activadores significativos del aprendizaje. Sin embargo, un problema importante en el CSCL es que la colaboración libre no produce necesariamente aprendizajes. En determinadas circunstancias la colaboración debe ser guiada de manera que aumente la probabilidad de alcanzar beneficios educativos. Precisamente, los guiones de CSCL integrados en sistemas TIC tienen como objetivo indicar cómo los alumnos deben interactuar entre ellos para que tengan lugar interacciones fructíferas. El ámbito de investigación específico de esta Tesis Doctoral recae sobre los denominados macro-guiones de CSCL, que describen métodos pedagógicos formulados como flujos de actividades. Este documento identifica y hace frente a tres retos relacionados con el problema de facilitar a los profesores el diseño de estos macro-guiones integrados en las TIC.

El primer reto hace referencia al diseño de los guiones de manera que éstos sean potencialmente productivos. Este trabajo propone utilizar patrones para capturar buenas prácticas en cuanto a la estructuración de situaciones de CSCL. El objetivo es que los patrones puedan ser reutilizados en la creación de nuevos guiones. Para ello, presentamos un modelo conceptual de lenguajes de patrones para guiones de CSCL, así como un lenguaje de patrones concreto que es conforme con el modelo. Dicho modelo define los diferentes tipos de patrones y relaciones entre patrones de manera que es posible definir numerosas posibilidades de secuencias de patrones que dan forma al diseño de guiones específicos.

El segundo reto tiene que ver con la implementación de los guiones en sistemas TIC. Con el propósito de que los guiones puedan ser interpretados automáticamente sin necesidad de desarrollar nuevos sistemas, proponemos representarlos computacionalmente utilizando la especificación IMS Learning Design (LD). Esta aproximación fomenta la interoperabilidad a la vez que hace posible la participación de los profesores en el diseño del comportamiento y la funcionalidad de los sistemas. Para ello, basta con que los profesores creen los guiones de acuerdo con las condiciones particulares de su situación de enseñanza-aprendizaje. Este trabajo analiza las posibilidades del lenguaje de modelado educativo LD para expresar los guiones considerando la propia notación pero también el uso de otras especificaciones y herramientas relacionas.

La combinación de las propuestas anteriores nos permite proponer un proceso de diseño basado en patrones para la creación de macro-guiones de CSCL representados computacionalmente con LD. Los patrones concretos que se han considerado son los llamados Patrones de Flujo de Aprendizaje Colaborativo (Collaborative Learning Flow Patterns, CLFPs). El objetivo principal del proceso de diseño es doble. Por una parte, pretende posibilitar la conceptualización de las interacciones esperadas de manera que los elementos críticos del CSCL son considerados al refinar plantillas LD basadas en los patrones. Por otra parte, persigue facilitar la creación amigable de guiones LD escondiendo los detalles de la especificación. De este modo, proponemos una solución para el tercer reto que se refiere al hecho de que las representaciones computacionales no les son familiares para la mayoría de los profesores. El proceso de diseño ha sido implementado en una herramienta de autoría (denominada Collage) que demuestra la viabilidad de la propuesta y permite su conveniente evaluación.

En conjunto, la metodología de investigación aplicada se caracteriza por el ámbito multidisciplinar en el que se enmarca la tesis. Particularmente, la fase de evaluación se ha llevado a cabo mediante un estudio múltiple de (tres) casos. Los tres pretenden evaluar las mismas contribuciones pero desde diferentes perspectivas. Incluyen talleres destinados a la audiencia potencial de nuestra propuesta principal (profesores con interés en aplicar CSCL) y experiencias con alumnos en situaciones reales. Los casos también involucran a expertos tanto en aprendizaje colaborativo como en LD e investigadores que proponen aproximaciones relacionadas. Los resultados de la evaluación no sólo muestran que los objetivos de la tesis se han conseguido sino que también ofrecen indicaciones relevantes de trabajo futuro.


x

xi

CONTENTS

CHAPTER ONE: INTRODUCTION

1.1 Introduction ........................................................................................................................... 1 1.2 Objectives of the dissertation................................................................................................. 4 1.3 Research methodology .......................................................................................................... 9 1.4 Structure of the dissertation................................................................................................... 12

CHAPTER TWO: DESIGN OF CSCL SITUATIONS

2.1 Introduction ........................................................................................................................... 15 2.2 From TEL to CSCL situations: design challenges................................................................. 17

2.2.1 CSCL as a research field within Technology-Enhanced Learning....................... 17 2.2.2 Instructional Design and CSCL............................................................................ 19 2.2.3 Scripting CSCL .................................................................................................... 20 2.2.4 Participatory Design and CSCL ........................................................................... 22 2.2.5 Discussion: research focus and derived challenges .............................................. 23

2.3 Design Patterns...................................................................................................................... 25 2.3.1 Patterns in Architecture ........................................................................................ 25 2.3.2 Patterns in Software Engineering ......................................................................... 27 2.3.3 Patterns in TEL and CSCL................................................................................... 30 2.3.4 Discussion: the generative problem and the generation of potentially effective scripts ............................................................................................................................ 32

2.4 Educational Modelling Languages ........................................................................................ 34 2.4.1 IMS Learning Design ........................................................................................... 34 2.4.2 Other approaches.................................................................................................. 37 2.4.3 Discussion: IMS LD as a candidate to computationally represent CSCL scripts ............................................................................................................................ 38

2.5 Conclusion............................................................................................................................. 39

CHAPTER THREE: CONCEPTUAL MODEL FOR CSCL SCRIPTING PATTERN LANGUAGES

3.1 Introduction ........................................................................................................................... 41 3.2 Methodology applied for proposing the model and the pattern language.............................. 43

3.2.1 Capturing the experience reported in the literature .............................................. 44 3.2.2 Using case studies as a starting point ................................................................... 45

3.3 CSCL scripting pattern languages model .............................................................................. 46 3.3.1 Aggregation model and types of connecting rules ............................................... 47 3.3.2 An illustrating CSCL scripting pattern language: hierarchical structure.............. 50 3.3.3 General guidelines for applying the PL described with the conceptual model..... 53 3.3.4 Discussion: other categorizations of patterns ....................................................... 54 3.3.5 Discussion: other sample patterns that fit in with the proposed conceptual model............................................................................................................................. 56

3.4 Examples of applying the CSCL scripting patterns............................................................... 57 3.4.1 Computer Architecture (CA) course .................................................................... 57 3.4.2 The use of ICT resources in Education (NNTT) course....................................... 59 3.4.3 Computer Networks Protocols (TTG) course....................................................... 61 3.4.4 Discussion: moral preoccupation, coherence, generativeness and creativity ....... 70

3.5 Potential computer-supported applicability ........................................................................... 72 3.6 Conclusion............................................................................................................................. 73

CHAPTER FOUR: IMS LD SUPPORT FOR COMPUTATIONALLY REPRESENTING CSCL MACRO-SCRIPTS

4.1 Introduction ........................................................................................................................... 77 4.2 Methodology applied in the analysis ..................................................................................... 79 4.3 Requirements of CSCL macro-scripts ................................................................................... 80

4.3.1 Group composition............................................................................................... 80 4.3.2 Role/resource distribution .................................................................................... 82 4.3.3 Coordination......................................................................................................... 83


xii

4.3.4 Flexibility ............................................................................................................. 85 4.4 Expressing the requirements using IMS LD notation ............................................................ 87

4.4.1 Group composition ............................................................................................... 88 4.4.2 Role/resource distribution..................................................................................... 90 4.4.3 Coordination ......................................................................................................... 92 4.4.4 Flexibility ............................................................................................................. 96 4.4.5 Discussion and revision of the limitations regarding LD notation ....................... 96

4.5 Supporting the requirements using related tools or specifications......................................... 98 4.5.1 Addressing the limitations using complementary specifications and tooling ....... 99 4.5.2 Discussion: general specification of group-services............................................. 101 4.5.3 Discussion: embedded in tools vs. computer-interpretable micro-scripts............. 102

4.6 Conclusion ............................................................................................................................. 103

CHAPTER FIVE: DESIGN PROCESS FOR THE GENERATION OF IMS LD SCRIPTS REUSING CLFPS

5.1 Introduction............................................................................................................................ 107 5.2 The “create-by-reuse” conceptual framework ....................................................................... 109

5.2.1 Reuse of learning design solutions ....................................................................... 110 5.2.2 Design processes for creating units of learning .................................................... 112 5.2.3 Discussion: other potential dimensions ................................................................ 114

5.3 The role of CSCL scripting patterns in design processes....................................................... 115 5.3.1 Assistant vs. templates.......................................................................................... 116 5.3.2 Implementing CLFPs as refinable LD templates.................................................. 118

5.4 A CLFP-based design process for the generation of LD scripts ............................................ 122 5.4.1 Scope: support the creation of LDs ...................................................................... 123 5.4.2 Combinations and concatenations of CLFP-based templates ............................... 124 5.4.3 The design process in the create-by-reuse framework.......................................... 125 5.4.4 Focus on CSCL critical elements: selecting the templates and authoring the scripts ............................................................................................................................ 126 5.4.5 Collage authoring tool: implementing the proposed design process .................... 131

5.5 Creating LD scripts using Collage......................................................................................... 140 5.5.1 CTM2 (Network Management) ............................................................................ 141 5.5.2 NNTT ................................................................................................................... 144 5.5.3 Pyramid-based paper discussion........................................................................... 145 5.5.4 Job interview simulation....................................................................................... 146 5.5.5 STA (Advanced Telematics Systems) .................................................................. 146

5.6 Discussion: towards more general approaches ...................................................................... 147 5.6.1 Using more general visual representations ........................................................... 147 5.6.2 Assembling learning design solutions formalized with different languages......... 147

5.7 Conclusion ............................................................................................................................. 149

CHAPTER SIX: EVALUATING THE DESIGN PROCESS WITH A MULTICASE STUDY

6.1 Introduction............................................................................................................................ 153 6.2 Formulation of the multicase study........................................................................................ 155

6.2.1 The quintain.......................................................................................................... 155 6.2.2 Overview of the cases forming the multicase study ............................................. 156 6.2.3 Mixed method....................................................................................................... 161

6.3 Case study A: Collage workshops ......................................................................................... 162 6.3.1 Description of the case study................................................................................ 162 6.3.2 Conceptual structure ............................................................................................. 167 6.3.3 Case findings ........................................................................................................ 168

6.4 Case study B: Planet game..................................................................................................... 179 6.4.1 Description of the case study................................................................................ 179 6.4.2 Conceptual structure ............................................................................................. 181 6.4.3 Case findings ........................................................................................................ 182

6.5 Case study C: Network Management..................................................................................... 195 6.5.1 Description of the case study................................................................................ 196 6.5.2 Conceptual structure ............................................................................................. 202 6.5.3 Case findings ........................................................................................................ 203

6.6 Cross-case analysis ................................................................................................................ 209

xiii

6.6.1 Assertion I ............................................................................................................ 212 6.6.2 Assertion II........................................................................................................... 214 6.6.3 Assertion III ......................................................................................................... 214

6.7 Discussion ............................................................................................................................. 216

CHAPTER SEVEN: CONCLUSIONS AND FUTURE WORK

7.1 Conclusions and main contributions...................................................................................... 219 7.2 Future research directions...................................................................................................... 224

APPENDIX A: A CSCL SCRIPTING PATTERN LANGUAGE

A.1 Collaborative learning flow level ......................................................................................... 230 A.2 Activity level ........................................................................................................................ 237 A.3 Resource level....................................................................................................................... 242 A.4 Roles and common collaborative mechanisms level ............................................................ 245

APPENDIX B: WELL-KNOWN CSCL SCRIPTS: RUNNING UNIVESANTÉ AND ARGUEGRAPH WITH AN LD ENGINE

B.1 Well-known CSCL scripts .................................................................................................... 250 B.2 Universanté Unit of Learning ............................................................................................... 253 B.3 ArgueGraph Unit of Learning............................................................................................... 260

APPENDIX C: SUPPORT EVALUATION DATA OF THE COLLAGE WORKSHOPS CASE STUDY

C.1 Pattern-based design process ................................................................................................ 266 C.2 Focus on CSCL critical elements.......................................................................................... 271 C.3 Use of Collage ...................................................................................................................... 274 C.4 Potential audience characteristics ......................................................................................... 279

APPENDIX D: SUPPORT EVALUATION DATA OF THE NETWORK MANAGEMENT CASE STUDY

D.1 Meaningfulness of the CSCL script created with Collage.................................................... 286 D.2 Enactment of the CSCL script using Gridcole...................................................................... 291 D.3 Educational innovation with respect to previous students’ experiences ............................... 293


xiv

xv

LIST OF FIGURES

Figure 1.1 General schema of the dissertation including its context, the aimed objectives, the original contributions as well as the accomplished evaluation ............................................................................... 6

Figure 2.1 Scheme of ideas developed in this chapter ...................................................................................... 17 Figure 2.2 Taxonomy of PD practices .............................................................................................................. 23 Figure 2.3 Relations between some proposals regarding patterns in TEL........................................................ 31 Figure 2.4 The conceptual model of IMS LD................................................................................................... 35 Figure 3.1 CSCL scripting pattern language conceptual model package and relationships with other models 47 Figure 3.2 Aggregation model of CSCL scripting Pattern Languages.............................................................. 48 Figure 3.3 Hierarchical structure of the CSCL scripting PL showing how the PL fits in with the conceptual

model. The numbers on each node reference the patterns as listed in Appendix A. Because of representational limitations not all the possible relationships are drawn ................................................ 51

Figure 3.4 “Life cycle” of CSCL scripts and related tools................................................................................ 72 Figure 4.1 LD representation of the groups involved in the PYRAMID structure (a) and an example of the

assignment of roles to users in a particular PYRAMID -based UoL (b)................................................... 88 Figure 4.2 Excerpt of a TAPPS-based LD (arrows point referenced element definitions) ............................... 91 Figure 4.3 Expressing the JIGSAW learning flow with the LD method............................................................ 93 Figure 5.1 Dimensions of the create-by-reuse framework: reusable learning design solutions at different level

of granularity and completeness............................................................................................................ 111 Figure 5.2 Design processes for creating UoLs by assembling and refining learning design solutions ......... 113 Figure 5.3 From a CLFP to a UoL: scheme of the process for obtaining CLFP-based UoLs......................... 119 Figure 5.4 General modules of a system implementing LD ........................................................................... 123 Figure 5.5 CLFPs hierarchies ......................................................................................................................... 125 Figure 5.6 CLFP-based design process for the creation of LD scripts within the create-by-reuse framework:

refinement process................................................................................................................................. 125 Figure 5.7 CLFP-based design process for the creation of LD scripts within the create-by-reuse framework126 Figure 5.8 CLFP-based design process for the creation of LD scripts: selecting the templates and authoring

the scripts .............................................................................................................................................. 128 Figure 5.9 Two dimensions of LD tool design ............................................................................................... 132 Figure 5.10 Plug-in framework for an LD editor ............................................................................................ 133 Figure 5.11 Collage selection interface .......................................................................................................... 134 Figure 5.12 Help information about the JIGSAW CLFP ................................................................................. 135 Figure 5.13 Collage authoring interface ......................................................................................................... 136 Figure 5.14 Configuring the flow of the PYRAMID-based template in Collage ............................................. 137 Figure 5.15 Configuring the flow of the SIMULATION-based template in Collage ....................................... 138 Figure 5.16 Configuring the flow of the TAPPS-based template in Collage.................................................. 138 Figure 5.17 Planning the learning flow........................................................................................................... 142 Figure 5.18 Graphical refinement of the template resulting of the combination of CLFPs using Collage (I) 143 Figure 5.19 Graphical refinement of the template resulting of the combination of CLFPs (II)...................... 143 Figure 5.20 Authoring the NNTT script with Collage.................................................................................... 145 Figure 5.21 Example design process in which various learning design solutions are integrated into refinement,

assembly and mixed processes, according to the create-by-reuse framework....................................... 148 Figure 5.22 Snapshot of the example design process that integrates assembly, refinement processes and mixed

processes, in accordance with the create-by-reuse framework.............................................................. 149 Figure 6.1 Graphic representation of the multicase study............................................................................... 157 Figure 6.2 Authoring the script with Collage ................................................................................................. 183 Figure 6.3 Refining the JIGSAW-based template with the description of the activities and the collaborative

tool supporting the activities ................................................................................................................. 184 Figure 6.4 Enacting the Planet game script created with Collage using Gridcole integrating a shared

repository: clue distribution................................................................................................................... 186 Figure 6.5 Enacting the Planet game script created with Collage using Gridcole integrating a discussion forum

and a chat: cooperative phase................................................................................................................ 186 Figure 6.6 Enacting the Planet game script created with Collage using Gridcole integrating a questionnaire

tool: proposing the solution................................................................................................................... 187 Figure 6.7 Hierarchy of group types as created by Collage according to the combined template .................. 198 Figure 6.8 Lab distribution according to the different groups ........................................................................ 199 Figure 6.9 Extract of the document provided to the students.......................................................................... 199


xvi

Figure 6.10 Global view of different moments of the experience, in which the students used the different integrated tools ......................................................................................................................................200

Figure 6.11 A student in the Expert group phase ............................................................................................200 Figure 6.12 Applied mixed evaluation method ...............................................................................................202 Figure B.1 UML activity diagram of Universanté script ................................................................................254 Figure B.2 UML activity diagram of ArgueGraph script................................................................................261

xvii

LIST OF TABLES

Table 2.1 Comparison between Alexander’s and Gamma’s patterns ............................................................... 29 Table 3.1 Types of connecting rules (relationships) between patterns at the same and different levels of

aggregation.............................................................................................................................................. 49 Table 3.2 TCP mechanisms to be studied in the course and the related scenarios of traffic interchange to be

analyzed and/or designed ........................................................................................................................ 64 Table 3.3 Sequence of lab sessions and the corresponding individual and collaborative learning activities.... 65 Table 3.4 Main expected competencies and skills ............................................................................................ 67 Table 3.5 Labels used in the text to quote the data sources of the “Network management” experience .......... 68 Table 3.6 Relevant evaluation conclusions and associated evidence................................................................ 69 Table 4.1 Computationally representing the group composition requirements of Universanté script .............. 89 Table 4.2 Computationally representing the group composition requirements of ArgueGraph script.............. 90 Table 4.3 Computationally representing the role/resource distribution requirements of Universanté script .... 91 Table 4.4 Computationally representing the role/resource distribution requirements of ArgueGraph script ... 92 Table 4.5 Computationally representing the coordination requirements of Universanté script ........................ 94 Table 4.6 Computationally representing the coordination requirements of ArgueGraph script ....................... 95 Table 4.7 Computationally representing the flexibility requirements of Universanté script ............................ 96 Table 4.8 Computationally representing the flexibility requirements of ArgueGraph script............................ 96 Table 5.1 Excerpt showing the process for obtaining the JIGSAW-based UoL............................................... 121 Table 5.2 Refining the template resulting of the combination of CLFPs towards the ready-to-run script

(students’ activities) .............................................................................................................................. 141 Table 5.3 Refining the template based on a combination of a JIGSAW and a PYRAMID towards the ready-to-

run script................................................................................................................................................ 145 Table 5.4 Refining the template based on the PYRAMID towards the ready-to-run UoL............................... 146 Table 6.1 Summary of the experiences involved in the multicase study ........................................................ 158 Table 6.2 Labels used in the text to quote the data sources of the “Collage workshops case study” ............. 169 Table 6.3 Narrative of the Planet Game scenario proposed to the participants of the workshop.................... 180 Table 6.4 Participants in the panel and the workshop under analysis in the “Planet game case study”.......... 181 Table 6.5 Labels used in the text to quote the data sources of the “Planet game case study”......................... 182 Table 6.6 Summary of the UoL (based on JIGSAW CLFP) created using Collage ........................................ 184 Table 6.7 Labels used in the text to quote the data sources of the “Network management case study” ......... 204 Table 6.8 Partial matrix for generating theme-based assertions from case findings, case A. ......................... 210 Table 6.9 Partial matrix for generating theme-based assertions from case findings, case B. ......................... 211 Table 6.10 Partial matrix for generating theme-based assertions from case findings, case C ........................ 212 Table B.1 Universanté script........................................................................................................................... 250 Table B.2 ArgueGraph script .......................................................................................................................... 251 Table B.3 ConceptGrid script ......................................................................................................................... 252 Table B.4 Narrative use case description of Universanté ............................................................................... 253 Table B.5 Users and their associations to roles (groups): “country group” .................................................... 254 Table B.6 Users and their associations to roles (groups): “thematic group” and “case group” ...................... 255 Table B.7 Screenshots of CopperCore running the LD-represented Universanté script................................. 255 Table B.8 Narrative use case description of the ArgueGraph script ............................................................... 260 Table B.9 Screenshots of CopperCore running the LD-represented ArgueGraph script ................................ 261 Table C.1 Partial results and support data concerning the information question “is the selection of the CLFP-

based LD templates and their representation useful and satisfactory?” ................................................ 266 Table C.2 Partial results and support data concerning the information question “does it achieve a satisfactory

trade-off between reuse of CLFPs and the creation of scripts contextualized according to the situational needs?” .................................................................................................................................................. 268

Table C.3 Partial results and support data concerning the information question “does the design process implemented in Collage help to determine the learning objectives, task-type and expected interaction that will be develop?”............................................................................................................................ 271

Table C.4 Partial results and support data concerning the information question “does the design process implemented in Collage help to understand and determine the structure regarding the flow of activities and the hierarchy of groups?” ............................................................................................................... 272

Table C.5 Partial results and support data concerning the information question “does the design process support also the definition of group-size, resource distribution, computer support and the structure within activities?”.................................................................................................................................. 273


xviii

Table C.6 Partial results and support data concerning the information question “can the teachers use successfully Collage?” ..........................................................................................................................274

Table C.7 Partial results and support data concerning the information question “how can Collage be improved?” ............................................................................................................................................277

Table C.8 Partial results and support data concerning the information question “which are the characteristics and motivations of the potential audience of Collage?”........................................................................279

Table D.1 Partial results and support data concerning the information question “is the CSCL script contextualized to the actual learning situation?” ...................................................................................286

Table D.2 Partial results and support data concerning the information question “does the CSCL script guide the learning process coordinating the students at the activity level according to the CLFPs on which is based?” ..................................................................................................................................................287

Table D.3 Partial results and support data concerning the information question “does the CSCL script foster the desired objectives related to collaborative learning?” .....................................................................290

Table D.4 Partial results and support data concerning the information question “can the students follow successfully the CSCL script using the Gridcole system?” ...................................................................291

Table D.5 Partial results and support data concerning the information question “how can the enactment of the CSCL script be improved?”...................................................................................................................292

Table D.6 Partial results and support data concerning the information question “does the enactment of the CSCL script enhance students’ previous experience in terms of structuring collaboration and use of supporting technology?”........................................................................................................................293

CHAPTER ONE

INTRODUCTION

This chapter introduces the main problems in the Computer-Supported Collaborative Learning field of research that motivate the dissertation. The chapter formulates the objectives, the expected contributions and the applied research methodology as well as the structure of the dissertation.

1.1 Introduction

Computer-Supported Collaborative Learning (CSCL) represents a rather new multidisciplinary

paradigm within the field of Technology-Enhanced Learning (TEL), in which Information and

Communication Technologies (ICT) are employed in order to improve various educational aspects

(Koschmann, 1996; Stahl, Koschmann, & Suthers, 2006). The main characteristics of CSCL include

highlighting the importance of social interactions as an essential element of learning (Dillenbourg,

1999a), as well as the need of participatory modes of designing new technological environments

(Häkkinen, 2002). CSCL solutions (developed by technologists) should offer the functionality

desired by the set of potential actors that participate in collaborative learning situations (mainly

teachers and students).

One of the main concerns in CSCL is that the expected interactions that would lead to learning

outcomes do not necessarily occur when the students are asked to collaborate freely (Dillenbourg,

2002). Among many different approaches that share the goal of enhancing effective collaboration

we can mention two important ones. The first approach is to monitor the collaboration and intervene

as necessary in order to redirect the group work in a more productive direction (Soller, Martínez-

Monés, Jermann, & Muehlenbrock, 2005). The other solution refers to an increase of the probability

of reaching successful CSCL situations by providing students with a set of instructions that guide

potentially fruitful collaboration (Dillenbourg, 1999b). When the instructions are technology-

mediated, they form what is called a computer-supported collaboration script (CSCL script or

simply script).


2

The goal of the guidance provided by the scripts refers to cognitive or educational objectives.

This difference is characteristically related to the granularity of the scripts (Fischer, Kollar, Haake,

& Mandl, 2007). Micro-scripts are intended for facilitating that students internalize them from a

cognitive psychologist perspective (e.g. learning how to argument by following a script that

scaffolds argumentation). They typically provide support for specific activities by describing the

fine-grained actions (e.g. sentence starters) that each participant should accomplish (Weinberger,

Fischer, & Stegmann, 2005). On the contrary, macro-scripts denote pedagogical methods defining

flows of coarse-grained activities (Dillenbourg & Jermann, 2007). They aim at organizing situations

that elicit desired interactions potentially leading to learning outcomes from the educational point of

view (e.g. understanding the key ideas of a topic by following a script that distributes the

knowledge and promotes mutual explanation).

Unfortunately, up to now the scripts are “hardwired” in specifically devoted learning

environments (Jermann, Soller, & Lesgold, 2004; Häkkinen & Mäkitalo-Siegl, 2007). This fact

limits their reusability in a different situation and imposes significant time and cost efforts when a

new script needs to be implemented. Moreover, developing a new scripting environment is not

trivial. This is mainly due to the inherent characteristic of multidisciplinarity in CSCL, which

implies a need for mutual understanding among the involved stakeholders (mainly experts in

education and in ICT) (Stahl et al., 2006). This need demands active participation of all these

stakeholders during the whole development cycle of CSCL solutions. Participatory Design (PD)

approaches (Muller & Kuhn, 1993) propose a diversity of practices with the goal of working

directly with users and other stakeholders in the design of social software systems.

In CSCL, it has been shown that there is a significant efficiency problem in performing

identification and analysis of requirements for the development of CSCL solutions that support

effective ways of learning (Dimitriadis, Asensio-Pérez, Martínez-Monés, & Osuna-Gómez, 2003).

Collaborative learning practitioners also become active players in the process of customizing

technological solutions to their particular needs in every teaching-learning situation. PD poses a

new requirement that CSCL technologists should tackle: how to obtain technological solutions for

collaborative learning capable of being particularized/customized by teachers that usually do not

have technological skills. The domain problem undertaken in this dissertation is related to

facilitating teachers play the role of designers of those technological solutions. Specifically, the

problem to be solved consists in how PD can be enabled by providing authoring tools for creating

macro-scripts which can be automatically interpreted and executed by learning environments such

as Learning Management Systems (LMSs). This type of systems makes possible the delivery of

learning activities and digital content to students (E-LANE, 2004; Bote-Lorenzo, 2005; Burgos,

Tattersall, Dougiamas, Vogten, & Koper, 2006).

INTRODUCTION

3

One of the problems associated to this research focus refers to the fact that scripts need to be

computationally represented (formalized) in order to enable their automatic interpretation.

Educational Modelling Languages (EMLs), in contrast to metadata specifications (e.g. Learning

Object Metadata, LOM) for describing reusable chunks of learning content (so-called Learning

Objects or LO) (Hodgings, 2000; Duval, 2001), are focused on specifying teaching-learning

processes (Rawlings, van Rosmalen, Koper, Rodríguez-Artacho, & Lefrere, 2002). IMS Learning

Design specification (IMS LD or simply LD) is currently accepted as the de facto standard EML and

the amount of developments around LD is significant (IMS, 2003b; Koper & Tattersall, 2005;

Burgos & Griffiths, 2005).

The aim of LD is to enable the creation of complete, abstract and portable descriptions of any

pedagogical approach taken in a course (or part of a course), which can be realized by a compliant

system. The key idea is that it represents the learning activities (performed by learners) and the

support activities (performed by teachers), including those comprising multi-role teaching-learning

processes and personalized learning routes (Koper & Olivier, 2004). Motivated by the

interoperability prospects and the specification declaration of intent, we consider LD as an

interesting candidate to computationally represent CSCL scripts.

However, the LD support for formalizing collaborative learning processes is not clear (Caeiro-

Rodríguez, Anido-Rifón, & Llamas-Nistal, 2003). This concern is motivated by the fact that the

specification is still very recent and it has not been widely adopted in real practice yet (Burgos &

Griffiths, 2005). Besides, there is a lack of significant examples and efforts showing the

possibilities of LD for CSCL (e.g. description of group hierarchies). Although partial work has been

already accomplished (Gorissen & Tattersall, 2005; Koper & Burgos, 2005), a more complete and

systematic analysis is needed.

On the other hand, the current LD compliant editors require a high level of expertise on LD and

therefore they are not indented for teachers but for expert instructional designers or educational

technologists (Milligan, Beauvoir, & Sharples, 2005; van der Vegt, 2005; Miao, 2005; de la Teja,

Lundgren-Cayro, & Paquette, 2005; Sampson, Karampiperis, & Zervas, 2005). With the aim of

enabling teachers to play the role of script designers, CSCL specific LD-based authoring tools

should incorporate (visual) design techniques (Botturi & Stubbs, in press) that hide the details of

LD (Griffiths & Blat, 2005) as well as the inherent difficulties involved in modelling scripts. These

difficulties are caused by the complex mechanisms and components that comprise the definition of

scripts (Kollar, Fischer, & Hesse, 2003), such as the interrelations of groups or the synchronization

of collaborative activity sequences.

Furthermore, these authoring tools should implement design processes that guide teachers in the

design of potentially effective scripts. This is especially important if the teachers (and students) are


4

novice in collaborative learning, since putting into practice collaborative learning experiences is not

trivial (Johnson & Johnson, 1999) and because of the risks of over-scripting the situations and thus

coercing relevant natural interaction (Dillenbourg, 2002). It is important to consider that traditional

Instructional Design (ID) approaches (devoted to individual instructional sequences) based on

general theories (Reigeluth, 1999) are too rigid and underestimate the complexity of CSCL

(Goodyear et al., 2004; Kenny, Zhang, Scwier, & Campbell, 2005). In this sense, CSCL requires

more pragmatic design processes grounded in practice (Karagiorgi & Symeou, 2005) which make

the critical elements for eliciting productive interaction explicit (Strijbos, Martens, & Jochems,

2004).

We argue that a promising solution to approach this problem is to propose a design process that

facilitates the reuse of generalizations of successful collaboration scripts (best/good practices)

formulated as design patterns. This would also avoid the costly efforts related to re-inventing script

strategies. Despite the fact that the word “pattern” has been used for centuries with slightly different

meanings, its use is more known in the fields of Architecture (Alexander et al., 1977) and Software

Engineering (Gamma, Helm, Johnson, & Vlissides, 1995). A pattern provides a means of

organizing information regarding a contextualized common problem and the essence of its broadly

accepted solution, so that it can be repetitively applied. A collection of interconnected (related)

patterns which enable the generation of a coherent whole (e.g. a town) is called a Pattern Language

(PL). Recently other domain specific patterns have been proposed, including TEL and CSCL

(Goodyear, 2005; Derntl & Botturi, 2006; Retalis, Georgiakakis, & Dimitriadis, 2006). However,

the existing pattern approaches in TEL, which vary in scope and purpose (e.g. patterns for designing

LMSs (Avgeriou, Papasalouros, Retalis, & Skordalakis, 2003) vs. patterns for mathematical games

(Learning Patterns, 2005)), do not include any specific proposal devoted to designing scripts.

Therefore, the general problem undertaken in this dissertation refers to the definition of a design

process that takes advantage of insights offered by best/good scripting practices (formulated as

patterns) for the creation of particularized educationally-sound LD-represented macro-scripts as a

means of providing PD approaches which enables teachers to influence in the behaviour and

functionality of CSCL scripting environments. In this way, section 1.2 introduces the objectives of

this dissertation. The research methodology followed to tackle them is presented in section 1.3.

Finally, section 1.4 concludes this introductory chapter.

1.2 Objectives of the dissertation

According to the research problems described in the previous section, the global aim of this

dissertation is:

INTRODUCTION

5

To propose and evaluate a design process based on patterns for facilitating the creation of

potentially effective CSCL macro-scripts computationally represented with IMS Learning Design so

that they can be interpreted by learning environments such as Learning Management Systems.

This global aim can be divided into the following more specific objectives. These derived

objectives and the original contributions of this dissertation are schematically represented in Figure

1.1 and described as follows:

• To identify the types of patterns and relations between patterns that can be used for

generating CSCL scripts.

In order to tackle this objective, it will be necessary to have a unifying view of several

different representative pattern-based approaches in TEL. That will allow us to situate

and describe the scope and audience of what we will define as CSCL scripting patterns.

An iterative process will be followed. It will include case studies, the experience of the

GSIC/EMIC (Intelligent & Cooperative Systems Group / Education, Media, Informatics

and Culture) research group (GSIC/EMIC, 1994), the results of TELL (Towards

Effective network supported coLLaborative learning activities) project (TELL, 2005b)

and a review of the literature with regard to design and, particularly, scripting in CSCL.

Once the types of patterns and the relations between them are identified, a method for

applying the patterns will be discussed. Furthermore, a CSCL scripting PL (with own

and adopted patterns) as well as three CL situations generated using the PL will be

provided to illustrate how the patterns can be applied.

Regarding this objective, the main contributions of this dissertation are the proposal of a

conceptual model for describing CSCL scripting Pattern Languages and an actual CSCL

scripting PL. Both contributions will be useful within this dissertation to situate the other

contributions. They also provide to the scientific community a starting point towards an

agreed high-level structure for the production of CSCL scripting patterns and PLs.

Part of these contributions have been published in (Hernández-Leo, Villasclaras-

Fernández, Asensio-Pérez, Dimitriadis, & Retalis, 2006d), which introduces the

hierarchical structure for CSCL scripting patterns characterized by the conceptual model,

and (Hernández-Leo, Asensio-Pérez, & Dimitriadis, 2006), which presents a real

experience that applies a script generated with the proposed pattern language


6

CONTEXT

Patterns

- Contextualized common problem and the essence of its reusable broadly accepted solution

- Patterns in Architecture, Software Engineering and also in TEL

Design of Computer-Supported Collaborative Learning situations

OBJECTIVES

To analyze the suitability of IMS LD for computationally representing CSCL

macro-scripts

To propose a design process that facilitates the reuse of Collaborative Learning Flow Patterns in the creation of CSCL macro-scripts (computationally represented with IMS LD) in a way that allows teachers to particularize the patterns according to the

needs of their educational situation, making explicit the CSCL critical

elements

CONTRIBUTIONS

3. Design process for the generation of CSCL scripts reusing CLFPs as LD

templates. Authoring tool implementing the design process (Collage)

1.Conceptual model for CSCL scripting pattern languages. A

CSCL scripting pattern language

EVALUATION

A multicase study

A. Four workshops in which slightly different audiences (mostly target users: teachers) create CSCL scripts based on CLFPs using Collage

B. Participation in a workshop of the ICALT 2006 conference in which we create a scenario proposed by a third-party using Collage. The results are compared to other approaches

C. Putting into practice an authentic blended learning scenario (in engineering education) that uses a CSCL script created with Collage

Educational Modeling Languages

- Computer-interpretable notations to represent units of learning

- Interoperability, IMS Learning Design as the de facto standard

Three CL situations generated using a

CSCL pattern language situated in

the proposed conceptual model

GLOBAL OBJECTIVE: To propose and evaluate a design process based on patterns for facilitating the creation of potentially effective CSCL macro-scripts computationally represented with IMS Learning Design

so that they can be interpreted by learning environments such as Learning Management Systems

To identify the types of patterns and relations between patterns that can be used for

generating CSCL scripts

2. Analysis of the possibilities and

limitations of LD for computationally

representing the CSCL characteristics

comprised in macro-scripts

Participatory Design

- CSCL is a multidisciplinar field - Specific needs of each situation - Most teachers do not have advanced technological skills

Scripting CSCL

- Planning CSCL scenarios so that they elicit expected fruitful interaction among the participants

- Micro-scripts (instructions within activities) vs. macro-scripts (flows of activities)

- It is not trivial to design scripts: risk of coercion, lack of experience practicing collaborative learning

- Scripts hardwired in specific learning environments prevent reusability

Instructional Design

- CSCL requires more pragmatic design processes, grounded in practice, than traditional ID approaches

- Focus on CSCL critical elements

Figure 1.1 General schema of the dissertation including its context, the aimed objectives, the original contributions as well as the accomplished evaluation

INTRODUCTION

7

• To analyze the suitability of IMS LD for computationally representing CSCL

macro-scripts.

The first step so as to fulfil this objective will be to identify common CSCL

characteristics in CSCL macro-scripts. The importance of these characteristics will be

justified according to the literature and related work such as the CoSSICLE project

(CoSSICLE, 2005) as well as the experience based on real context of the GSIC/EMIC

research group (GSIC/EMIC, 1994). After that, LD-represented scripts including the

identified characteristics will be developed. That will allow us to clarify the possibilities

of LD for computationally representing CSCL macro-scripts, differentiating the LD

notation itself from related specifications and supporting tools.

On this topic, the contribution of this dissertation is the analysis of the possibilities and

limitations of LD for computationally representing the CSCL characteristics comprised

in CSCL macro-scripts. The global conclusions have been submitted for publication

(Hernández-Leo, Burgos, Tattersall, & Koper, submitted) and are currently under

review. However, significative partial results have been already published in

(Hernández-Leo, Asensio-Pérez, & Dimitriadis, 2005), extended version of (Hernández-

Leo, Asensio-Pérez, & Dimitriadis, 2004) which received a “Best Paper Award” in the

conference where it was presented, and (Hernández-Leo et al., 2005a), extended version

of (Hernández-Leo et al., 2005b).

• To propose a design process that facilitates the reuse of CLFPs (Collaborative

Learning Flow Patterns, a particular type of CSCL scripting patterns) in the

creation of CSCL macro-scripts (computationally represented with IMS LD) in a

way that allows teachers to particularize the patterns according to the needs of

their educational situation, making explicit CSCL critical elements.

Having the aforementioned proposals as a starting point, we will describe an approach

facilitating the reuse of CLFPs for the generation of CSCL scripts computationally

represented with LD. The design process will meet the following requirements. Firstly, it

will achieve a satisfactory trade-off between particularizations of CLFPs so that the

resulting LDs are contextualized according to particular CL situations and the loss of the

meaningfulness captured in CLFPs. Secondly, it will allow teachers to focus on CSCL

critical features (e.g. learning objectives, task type, level of pre-structuring, group size).

Finally, it will not require high technical knowledge, particularly of LD. An important

element, in this sense, will be the implementation of the design process in an authoring

tool (Collage) in order to prove its feasibility and to enable its proper evaluation.


8

The design process for the generation of CSCL scripts reusing CLFPs is the central

contribution of this dissertation since it links several of the previous specific objectives.

The design process together with its implementation in Collage has been published in

(Hernández-Leo et al., 2005; Hernández-Leo, Villasclaras-Fernández, Asensio-Pérez, &

Dimitriadis, in press; Hernández-Leo et al., 2006e) and has been also co-honoured with

the 2006-2007 European Award for Excellence in the Field of CSCL Technology. The

“create-by-reuse” framework in which the process is situated has also been published in

(Hernández-Leo, Harrer, Dodero, Asensio-Pérez, & Burgos, 2006a).

• To evaluate the proposed pattern-based design process for CSCL macro-scripts

computationally represented with IMS LD.

In order to evaluate the design process, we will carry out three different case studies:

A. The first case study will comprise four workshops in which participants (mainly

potential users: teachers) will create LD-represented CSCL scripts based on CLFPs

following the proposed design process integrated in Collage. These experiences will allow

us to value to which extent the design process implemented in Collage facilitates the reuse

of CLFPs in the creation of particularized LD-represented scripts, in a way that allows

teachers to focus on CSCL critical elements.

B. The second case study will involve the participation in a workshop where several

researchers proposing related approaches design a scenario proposed by the workshop

organizers. Therefore, this experience will provide indications regarding whether our

proposal can be used for creating a script representing a scenario proposed by a third party.

Moreover, the workshop will represent a good opportunity to compare our contributions

with related work. Partial conclusions of the this case study are published in (Hernández-

Leo et al., 2006b)

C. The third case study will deal with a real situation in engineering education where

students will experience a CSCL script created according to the design process. This case

study will allow us to show that the scripts are meaningful and can be applied in authentic

educational situations. The results of this case study have been accepted for publication

(Hernández-Leo et al., in press).

Furthermore, this evaluation will allow us to extract conclusions that will be useful for

future research.

INTRODUCTION

9

1.3 Research methodology

The objectives of this dissertation are framed within a multidisciplinary problem domain. This

fact demands a hybrid methodology that includes elements of diverse research approaches (Adrion,

1993) and highlights the need to consider the social context (Stahl et al., 2006). Multiple domain

knowledge, combining theory and praxis, is the key factor of the methodology, which focuses on

real practitioners’ needs.

Therefore, research and practice evolve together in the applied research approach. It uses four

phases: informational, propositional, analytical and evaluation (Glass, 1995). Several iterations, in

which the findings of each phase (essentially the analytical and evaluation phases) feed earlier

phases, are accomplished until “satisficing” results are achieved. (Satisficing is a concept coined by

Herbert Simon which identifies the decision making process whereby one chooses an option that is,

while perhaps not the best, good enough (Simon, 1982).) That expresses the significance of the

evaluation phase, which is also critical for validating the proposals.

Within each phase of the research approach, the applied research methods are described as

follows (Glass, Vessey, & Ramesh, 2002; Zelkowitz & Wallace, 1998):

• Informational phase

The aim of this phase is to gather information in order to, on the one hand, identify and

clearly formulate the research questions and, on the other hand, have an outline of the

current knowledge involved in the problem domain.

The main methods involved in this phase include tasks belonging to both the scientific

(observing the world) and the engineering (observing existing solutions) approaches:

- The search, review and analysis of literature regarding the topics of the problem domain:

CSCL with emphasis in design and scripting problems, pattern-based approaches in TEL

and EMLs.

- The participation in the GSIC/EMIC multidisciplinary research team (GSIC/EMIC,

1994) whose field of expertise (including research and practice) is collaborative learning

and its computer support. Particularly, the experience and findings of two case studies

investigated by the group constitute an important legacy for this research work

(Martínez-Monés et al., 2005; Ruiz-Requies, Anguita-Martínez, & Jorrín-Abellán,

2006).

- The participation in several conferences and projects whose topics include the keywords

related to this research work. The projects are TELL e-Learning project

EAC/61/03/GR009, Kaleidoscope Network of Excellence FP6-2002-IST-507838,

Spanish Ministry of Education and Science projects TIC-2002-04258-C03-02 and


10

TSI2005-08225-C07-04 and Autonomous Government of Castilla and León, Spain,

projects VA009A05, UV46/04 and UV31/04. Particularly, the outcomes of TELL

project (TELL, 2005a; TELL, 2005a) represent a significant input to this dissertation.

This phase has been also largely benefited by the informal but active involvement in the

UNFOLD (Understanding New Frameworks Of Learning Design) project (UNFOLD,

2004; Burgos & Griffiths, 2005) (related to IMS LD specification) as well as the

participation in two workshops offered by the Kaleidoscope Virtual Doctoral School and

organized by the multidisciplinary European Research Team CoSSICLE (CoSSICLE,

2005).

• Propositional phase

In this phase we propose and formulate the solutions to the identified research questions

using the information aggregated in the previous phase.

- The core of the first and the third contributions of this dissertation (as numbered in

Figure 1.1) are sketched in this phase, namely the conceptual model for CSCL scripting

patterns languages and the design process for creating CSCL scripts based on CLFPs.

Regarding the formulation of the patterns included in the example of the CSCL scripting

PL, two kinds of pattern mining methodologies are employed (Baggetun, Rusman, &

Poggi, 2004; Retalis et al., 2006): deductive or top-down (using best or good practices in

structuring collaborative learning) and inductive or bottom-up (using the conclusions of

case studies).

- The proposal of computationally representing the CSCL macro-scripts and CLFPs using

IMS LD is also a result of this phase.

• Analytical phase

The purpose of this phase is to analyze and explore the proposals which may lead to a

demonstration or formulation of principles.

- A concept implementation (proof of concept) is performed in order to analyze the

proposal related to the first contribution. It consists in providing a feasible CSCL

scripting pattern language, which can be described with the proposed conceptual model,

and in theoretically generating CSCL scripts that illustrate how the patterns might fit

together. The concept implementation is complemented with examples of how the

patterns can be applied. Whilst these tasks are realized, several iterations proceed back to

the propositional phase as far as the first contribution is concerned.

- The analysis of the possibilities and limitations of IMS LD for computationally

representing significant CSCL scripting characteristics that appear in CSCL scripts is

INTRODUCTION

11

also accomplished in this phase. The applied method consists mainly in trying to develop

LD-represented scripts that code these CSCL characteristics. Part of this work is realized

at OTEC (Educational Technology Expertise Centre) in the OUNL (Open University of

the Netherlands) during a three-month research stay (OTEC, 2006).

• Evaluation phase

This phase is devoted to evaluating the proposals and the analytic findings by means of

several case studies (Zelkowitz et al., 1998; Lundgren-Cayrol, Marino, Paquette, Léonard, &

de la Teja, 2006; Jorrín-Abellán, Dimitriadis, Rubia-Avi, Anguita-Martínez, & Ruíz-

Requies, 2006) organized as a multicase study (Stake, 2005). The case studies aim at

assessing the same contributions but from a different perspective:

- A case study comprising four workshops in which different audiences create CSCL

scripts using the proposed design process. The audience is mainly the target users, i.e.

teachers of two different universities (University of Cádiz and University of Valladolid,

both in Spain) with interest in applying CL and ICT in their practice, but also experts in

the field of research: educational technologists (UNFOLD members) and CSCL

practitioners and researches (members of the GSIC/EMIC group). Some of the CSCL

scripts that they create in the workshops are designed beforehand in laboratory

experiments, in which the corresponding UoLs are created and validated using the

reference LD engine, CopperCore (Martens & Vogten, 2005).

- A case study in which we design a scenario proposed by a third-party using our

approach. It involves the participation in an ICALT 2006 (6th IEEE International

Conference on Advanced Learning Technologies) conference workshop (Vignollet,

David, Ferraris, Martel, & Lejeune, 2006). We create a script reflecting the scenario

using Collage and further execute it using Gridcole (Bote-Lorenzo, 2005).

- A case study in which a real-world educational situation uses a CSCL script created

according to the proposals of this dissertation. The experience is part of an eligible

course on Network Management within Telecommunication Engineering studies at the

University of Valladolid.

In the case studies, a mixed method combining quantitative and qualitative data collection

techniques is employed (Goubil-Gambrel, 1992; Jorrín-Abellán, Rubia-Avi, Anguita-Martínez,

Gómez-Sánchez, & Martínez-Monés, in press; Martínez-Monés, Dimitriadis, Rubia-Avi, Gómez-

Sánchez, & de la Fuente-Redondo, 2003). The emphasis is more on qualitative than on quantitative

research, which is only considered useful for showing trends and indicating probabilities. In

contrast, qualitative research is used to identifying salient features or variables in particular

representative settings according to which the results can be interpreted (Denzin & Lincoln, 2005).


12

1.4 Structure of the dissertation

The rest of this dissertation is structured as follows:

- After this introduction, Chapter Two focuses on the domain problem related to the design of

CSCL situations. With this purpose, it reviews CSCL as a multidisciplinary field of research

within TEL and analyzes important design approaches in TEL and CSCL. In this sense, the

chapter describes the role of Instructional Design (ID) in CSCL, focuses on the scripting

CSCL approach and discusses the importance of Participatory Design (PD) in the design of

this type of ICT applications. The analysis leads us to formulate three challenges around the

problem of enabling participatory modes of design in which teachers create their own CSCL

macro-scripts embedded in software environments. Besides, Chapter Two also explores

research directions that envisage solutions to tackle the identified challenges. These

directions include the use of design patterns and Educational Modelling Languages (EMLs)

as well as their combined application through design processes integrated in authoring tools

for the creation of computer-interpretable scripts.

- Chapter Three is devoted to the research direction referred to the use of design patterns. Its

main function is to identify the types of patterns and the relationships between them that can

be jointly applied in the design of CSCL scripts. In this sense, it presents a model for CSCL

scripting pattern languages and discusses a specific pattern language (PL) which is

compliant with the model. To illustrate that the PL enables the generation of many scripts,

the chapter also includes three authentic situations that apply a sequence of interconnected

patterns selected from the PL.

- Chapter Four in turn analyzes the suitability of IMS Learning Design (LD) specification, the

most significant EML at the moment, for computationally representing CSCL macro-scripts.

In this sense, it points out important requirements of the scripts and studies the possibilities

and limitations of the specification to support them. In the analysis, which is illustrated by

means of relevant cases, the scope of the LD notation is confronted to the role of related

tooling facilities and eventually complementary specifications.

- Chapter Five proposes a design process that combines the contributions of the previous

chapters. In particular, the design process facilitates the reuse of CLFPs (Collaborative

Learning Flow Patterns, a particular type of CSCL scripting patterns) in the creation of

CSCL macro-scripts represented with LD. The proposed design process is situated and

compared to related work by means of a framework that conceptualizes different (existing

and yet-to-come) approaches that drive the creation of full-fledged LD Units of Learning

(UoL) by reusing different types of design solutions. Our proposal, targeting teachers

without high (LD) technical knowledge, aims at achieving a satisfactory trade-off between

particularizations of CLFP-based templates according to specific CL situations and the loss

INTRODUCTION

13

of the solutions captured in CLFPs. Besides, the design process allows teachers to focus on

CSCL critical features that are involved in the elicitation of expected interaction processes.

The chapter also presents Collage, an authoring tool that implements the design process

proving its feasibility and enabling its proper evaluation. Moreover, the creation of LD

scripts using this authoring tool is illustrated with several examples.

- In Chapter Six the evaluation of the proposed design process is accomplished by means of a

multicase study. The multicase study comprises three cases which aim at assessing the same

contributions but from different perspectives. The first case is devoted to workshops where

the target audience use the design process implemented in Collage. The second case implies

the design of a scenario proposed by a third-party using our approach and its comparison

with related approaches. The last case analyzes an authentic educational situation where

students follow a script created according to the design process. The chapter finishes with a

cross-case analysis which emphasizes the combined results of the studied cases as the global

assumption of the evaluation.

- Chapter Seven draws together the main conclusions of the dissertation listing its main

contributions and pointing out future research directions.

- Appendix A contains the CSCL scripting pattern language cited in Chapter Three.

- Appendix B includes three well-known CSCL scripts, two of which (Universanté and

ArgueGraph) are also formulated as narrative use case descriptions and UML activity

diagrams since they are used to illustrate the analysis of Chapter Four. The appendix also

shows screenshots of their LD representations (UoLs) running with an LD engine. The

ready-to-run UoLs are available in a CD-ROM attached at the end of the dissertation.

- The remaining appendixes (C and D) collect support data employed in the multicase study

presented in Chapter Six. The raw data is also available in the attached CD-ROM.


14

CHAPTER TWO

DESIGN OF CSCL SITUATIONS

The aim of this chapter is to present the domain problem of the dissertation, putting into focus the specific challenges that it undertakes. It starts by introducing CSCL as a multidisciplinary research field within TEL, which emphasizes the social interactions as activators of learning in the accomplishment of joint activities that are supported by technology. The zoom is applied to the design issues around CSCL, considering the role of the teachers in the design of CSCL situations embedded in learning environments (scripts). Within the scripting CSCL approach, participatory design requirements together with the lessons learned from Instructional Design lead us to identify three challenges as well as the directions that envisage solutions to tackle them: in order to design the scripts capable of eliciting fruitful interactions, we explore the design patterns approach; besides, to computationally represent the scripts so that they can be interpreted by the environments without the need of developing new systems, we seek for the use of an Educational Modelling Language; the combination of both approaches through a design process to be implemented in high level authoring tools for creating scripts would allow teachers to influence in the behaviour and functionality of the learning environments that supports CSCL situations.

2.1 Introduction

How can technology facilitate and improve the teaching and learning experience? The

introduction of TEL is significantly more complex than a mere addition of technology to the

existing educational practices (Rogers, 2002). Reaching the ultimate goal of the TEL domain

requires a fundamental redesign of the educational systems (Jochems, van Merriënboer, & Koper

R., 2004) as well as important research efforts related to the design of meaningful technical

solutions and authentic usage situations (also called scenarios or settings) (Herrington & Oliver,

2000; Häkkinen, 2002; Dimitracopoulou & Petrou, in press). The emphasis is clearly on learning

with technology and not on learning from technology. We use the term TEL instead of e-Learning,

because the latter is often associated only to distance learning (Littlejohn, 2005). However,

technology-mediated learning can also benefit face-to-face classrooms or blended situations, which

combine non-technology and technology mediated face-to-face and/or distance education (Bonk,


16

Graham, Cross, & Moore, 2006). Dillenbourg et al. (2007) refer to these situations as “integrated

learning”, stressing this broad perspective that may also consider different pedagogical approaches.

Though we acknowledge the importance of other interrelated research dimensions such as

“evaluation” (Martínez-Monés, 2003; Soller et al., 2005), this dissertation relies on the crucial role

that “design” plays in the particular field within TEL (concerted with collaborative learning) termed

CSCL (Koschmann, 1996; Dillenbourg, 1999a; Stahl et al., 2006). Specifically, the dissertation

focuses on the ICT challenges that emerge from the role of teachers as the ultimate stakeholders in

charge of designing potentially effective CSCL situations that make use of learning environments

(or systems) (Häkkinen, 2002). The significance of this research focus within CSCL is motivated,

among other aspects detailed along the chapter, by the fact that most teachers are not technologist

and do not have always the support of technical experts when they need to arrange or modify the

functionality of the learning systems, so that it satisfies the requirements of their concrete

educational situations (Williams, Coles, Wilson, Richardson, & Tuson, 2000; Griffiths et al., 2005).

Before elaborating on this focus and its derived challenges, this chapter introduces the specific

characteristics of CSCL that differentiate it from related fields within TEL. Then, as schematized in

Figure 2.1, we analyze two important TEL design approaches, namely “Instructional Design” (ID)

(Reigeluth, 1999; Merril, 2002) and “Participatory Design” (PD) (Muller et al., 1993; Muller,

Wildman, & White, 1993), highlighting their implications for CSCL. Moreover, the distinguishing

facets of CSCL demand the study of a particular research line within CSCL, known as “Scripting

CSCL”, which proposes to structure the CSCL situations by means of scripts that guide

collaboration (O'Donnel & Dansereau, 1992; Dillenbourg, 2002; Kirschner, 2002; Fischer, Kollar,

Mandl, & Haake, 2007). Furthermore, in order to tackle the identified challenges we propose to

combine the approaches of two other parallel design-oriented research lines: Design Patterns (or

simply patterns) (Alexander et al., 1977; Gamma et al., 1995; Derntl et al., 2006) and EMLs

(Rawlings et al., 2002; Koper et al., 2004).

Therefore, the structure of the chapter is as follows. Section 2.2 introduces CSCL as a research

field within TEL. It describes the role of ID in CSCL, focuses on the scripting CSCL approach and

discusses the importance of PD in the design of this type of ICT applications. This analysis leads us

to identify a set of challenges for technically supporting the design of CSCL situations, which we

point out at the end of the section. Section 2.3 and section 2.4 are devoted to patterns and EMLs,

respectively, since we utilize them to propose a solution overcoming the challenges. A conclusion

envisaging how we generally approach this proposal, gradually developed in the following chapters,

is included in section 2.5.


17

Focus: Teachers as the designers of CSCL situations in learning environments

Technology-Enhanced Learning

Computer-Supported Collaborative Learning

Instructional Design Participatory Design

Scripting CSCL

Educational Modelling Languages

Design Patterns

Challenges

Figure 2.1 Scheme of ideas developed in this chapter

2.2 From TEL to CSCL situations: design challenges

In principle, TEL includes different forms of technology: from the classical audio- and video-

tapes (Avery, Avery, & Pace, 1998) to the new interactive furniture for classrooms (such as

electronic boards or interactive tables) (Mäkitalo-Siegl, Kaplan, Zottmann, Dillenbourg, & Fischer,

2007). However, it is the use of computers, and mainly computer networks, what has strongly

influenced the progress of the TEL domain (Koschmann, 1996). Our scope in this dissertation

focuses, accordingly, on the application of ICT to enhance education, particularly when

emphasising the role of collaboration as an important element of learning (Dillenbourg, 1999a).

2.2.1 CSCL as a research field within Technology-Enhanced Learning

More than a decade ago Timothy Koschmann, in (Koschmann, 1996), presents CSCL as a

emerging field of research in TEL (or Instructional Technology, as he refers to the broad domain of

using technology for instructional purposes). The historical evolution of CSCL is reviewed in detail

in a recent essay by Stahl et al. (2006). This review depicts how a global CSCL community has

developed from a series of biennially devoted conferences to the new international journal specific

to CSCL (namely ijCSCL).


18

In his highly referenced book chapter (Koschmann, 1996), Koschmann describes previously

existing TEL fields, namely CAI (Computer-Assisted Instruction), ITSs (Intelligent Tutoring

Systems) and the Logo-as-Latin paradigm, and points out the different assumptions that

characterize CSCL. The key difference relies on its perspective concerning the implications of the

social issues in the teaching and learning processes (Roschelle & Teasley, 1995). That is, CSCL

specifically approaches the support of social interactions among the students themselves, with a

teacher playing a mediator or facilitator role (Stahl, 2006).

In this sense, the theories underlying the “collaborative learning” model of education (Johnson

& Johnson, 1999) are socially oriented: Social Constructivism Approaches, Soviet Sociocultural

Theories, Theories of Situated Cognition, etc (Koschmann, 1996; Martínez-Monés, 2003).

Moreover, the CSCL research field is characterized as multidisciplinary (or inter-disciplinar) by

definition. It involves educational, computer and social sciences. As Koschmann states, CSCL is

“built upon the research traditions of those disciplines –anthropology, sociology, linguistics,

communication science- that are devoted to understanding language, culture and other aspects of

the social setting (Koschmann, 1996, pp. 11).”

The assumption that computers can support the collaboration process is also considered by the

CSCW (Computer-Supported Cooperative Work) field (Ellis, Gibbs, & Rein, 1991; Grundin, 1992;

Ellis & Wainer, 1994), which in this perspective is very close to CSCL. However, in contrast to

CSCL, which aims at promoting effective learning outcomes, CSCW points towards increasing

productivity in industrial and business settings. Undeniably, the different words of “learning” vs.

“work” that appears in both acronyms imply different fundamental objectives and, thus, different

approaches and methods to reach them. This discussion may lead the reader to wonder whether

there is also a difference between “collaboration” and “cooperation”, since they are also

distinguished in the acronyms. Though there are some authors that do not make distinctions in this

respect, Pierre Dillenbourg discriminates thoroughly cooperative and collaborative learning in

(Dillenbourg, 1999b): while in collaboration the students accomplish the activities “together”,

cooperation refers to a mere assemblage of partial results corresponding to a divided task. In other

words, genuine collaboration requires systematic efforts to work and learn together, the interaction

of participants should not be competitive or accidental (Stahl, 2006). In CSCL there are two main

(complementary) ways to facilitate these systematic efforts:

- monitoring the collaboration so that the teacher intervenes when necessary in order to

regulates the group so that it proceeds more fruitfully (Soller et al., 2005), and

- providing (beforehand) a set of instructions that scaffolds collaboration in such a way that

the probability of reaching successful CL situations increases (Dillenbourg, 1999b).

Concluding, CSCL is a multidisciplinary research field that seeks to enhance learning by

providing (networked) computer (or technology) support for collaborative learning, where “the


19

words ‘collaborative learning’ describe a situation in which particular forms of interaction among

people are expected to occur, which would trigger learning mechanisms, but there is not guarantee

that the expected interactions will actually occur (Dillenbourg, 1999b, pp.5).” For the purposes of

this chapter, focused on the design of CSCL situations, the target is, therefore, to explore the

challenges and potential solutions that provide a satisfactory way to design CSCL situations which

promote the elicitation of productive interactions in terms of learning objectives (related to the

acquisition of knowledge, skills and competences or attitudes). Hence, next subsection discusses the

connections between CSCL and the ID approach largely used in other fields of TEL.

2.2.2 Instructional Design and CSCL

During decades ID methods have been applied to the development of instructional sequences

for individual learning. They offer explicit guidance on how to better help students learn and

develop (Reigeluth, 1999). That is to say, instruction is about designing in advance assignments

intended to promote learning (Kirschner, Carr, van Merriënboer, & Sloep, 2002). The traditional

methodology of ID encompasses the analysis, design, development, implementation, and evaluation

of instructional processes. In this way, ID researchers are proposing the use of notation systems to

record precise and minute details of designs in order to ensure correct duplication, execution and

communication, as other mature fields of study such as chemistry or musicology (Waters &

Gibbons, 2004).

Merril (2002) examines representative ID theories and concludes that, although they use

different terms and emphasize different perspectives, they share fundamental principles. These

principles are related to the idea that learning is promoted when knowledge is activated as a

foundation for new knowledge and it is applied by the student to solve real-world problems

(Herrington et al., 2000). Regardless, in practice designers do not make use of ID models, according

to the evidences discussed in (Kenny et al., 2005). Though these models represent helpful

conceptual frameworks, they are too rigid in that they are excessively linear, systematic and

prescriptive. Besides, it is still not clear whether the utility and adaptability of these models are

aligned along with the practitioners needs. This is probably due to that ID adopts general theories

and their models are not actually grounded in practice. This statement is also supported by

(Karagiorgi et al., 2005), who proposes the use of more pragmatic approaches to ID that could

facilitate the design of more situated, experiential, meaningful and cost-effective TEL situations.

On the other hand, several authors critic the value of the methods and tools derived from ID to

be used in CSCL. This is mainly caused by the design tensions that characterized this educational

field (Tatar, in press). Dillenbourg points out the risks of over-prescribing collaboration in

(Dillenbourg, 2002) and the needs of flexibility (Dillenbourg & Tchounikine, 2007). Goodyear et

al. criticize the teacher-centred behavioural approach of ID methods that underestimate the


20

complexity of learning processes (Goodyear et al., 2004), specially when they are collaborative

(implying flexibility, context dependency and social variables). Undoubtedly, CSCL situations

demand more flexibility than traditional rigid instructional sequences. Nevertheless, it is also true,

as the previous subsection hints, that planning successful CSCL situations entails well-targeted

effort at design time (Santoro, Borges, & Santos, 2004; Strijbos et al., 2004; Goodyear, 2005).

Consequently, some of the ideas coming from ID could be adapted to propose design approaches

for CSCL. In this case the focus is on designing for interaction, which should stress the promotion

of leaning experiences that benefit the students beyond the mere satisfaction of pedagogical or

technological needs of the instructional objectives (Kirschner, Strijbos, Kreijns, & Beers, 2004).

All these ideas are considered in (Strijbos et al., 2004). Strijbos et al. discuss that considering

the multitude of individual and group level variables that may affect collaborative processes, a less

rigid view of ID is necessary in CSCL. In this sense, designers should focus on methods that

support the learning (and interaction) processes, and not so much on the attainment of pre-defined

goals (since this attainment cannot be guaranteed for all participants). Thus, they propose a process-

oriented methodology for the design of CSCL situations. The methodology ensures that student

participation is likely to lead to skill acquisition, by focusing on the elicitation of expected

interaction processes. In this sense, they identify five critical elements that affect the interaction:

learning objectives, task type, level of pre-structuring, group size and technology.

The “level of pre-structuring” element refers to shaping the way students interact with each

other. This scaffolding approach to facilitate social and cognitive processes in collaborative learning

environments is currently known as “scripting CSCL”.

2.2.3 Scripting CSCL

Scripting is a research topic within CSCL studied from cognitive (psychological) (Kollar,

Fischer, & Slotta, 2005), educational (Mäkitalo, Weinberger, Häkkinen, Järvelä, & Fischer, 2005)

and technological (Harrer, Hernández-Leo, & Dimitriadis, 2005; Miao, Hoeksema, Hoppe, &

Harrer, 2005) perspectives (Fischer et al., 2007). As the CoSSICLE European Research Team

describes, “computer-supported scripts (CSCL scripts or simply scripts hereafter) aim at facilitating

social and cognitive processes of collaborative learning by shaping the way learners interact with

each other. Being embedded in the learning environment, computer-supported scripts can optimally

structure interaction as well as support the learners with the very activity they are engaged in

(CoSSICLE, 2005).”

The purpose behind scripting can be of two types (Fischer et al., 2007). The first type refers to

external processes that should be internalized by the learners or influence the already internalized

script (Häkkinen et al., 2007). They provide support for specific activities by describing the fine-

grained actions (e.g. question prompts, sentence starters, or descriptions) that each participant


21

should accomplish (within such activities). This is the reason why they are called micro-scripts.

Weinberger et al. (2005) present two examples of micro-scripts that guide argumentation processes.

The goal of micro-script is that students learn the script; i.e., in the previous examples, how to

argument in order to construct knowledge together. In contrast, macro-scripts aim at organizing

situations that encourage targeted interactions potentially leading to learning outcomes, which are

not necessarily related to the internalization of the script (e.g. the script arranges fruitful discussions

by grouping students with different results in previous activities). Though the granularity distinction

is not purely binary, macro-scripts characteristically denote pedagogical methods defining flows of

coarse-grained activities (Dillenbourg et al., 2007). It is noteworthy that there exists a parallelism of

these ideas with some CSCW notions: (collaborative) learning flow or learnflow are terms used in a

similar way to the CSCW workflow or activity-level coordination, while the specific support within

activities implies relations with the action-level coordination as defined by Ellis et al. (1994).

This dissertation addresses macro-scripts (to facilitate the readability of the dissertation we refer

to macro-scripts as simply scripts hereafter), which mainly describe how groups (and individual

roles) should perform a set of interrelated activities. A framework for the description of scripts is

proposed in (Kobbe et al., submitted). It states that scripts can be specified with a small number of

components: the participants, the activities that they engage in, the roles they assume, the resources

used in each activity and the groups they form. In addition, mechanisms describe the distributed

nature of scripts: how participants are distributed over groups (group formation), how components

are distributed over participants (component distribution) and how both components and groups are

distributed over time (sequencing).

According to (Jermann et al., 2004), there are three different types of environments that can be

used to script collaboration. The first type takes advantage of natural (communication) tool

affordances (Kirschner, 2002; Brecht et al., 2006). In this approach, the teacher is in charge of

scripting the collaboration by socially scaffolding the students while working with the tools.

However, a more explicit technology-mediated coordination may be useful depending on the

characteristics of the educational situation (Dimitriadis et al., 2007). Another type of scripting

environment includes tools deliberately designed to structure collaboration. Some examples are

structured dialogue interfaces (Weinberger et al., 2005) or devoted tools for particular situational

subject areas (Dillenbourg, 2002). The third type refers to configuring existing LMSs (Muñoz-

Merino, Delgado-Kloos, Seepold, & Crespo-García, 2006) so that the scripts are organized through

navigation functionalities.

On the other hand, a crucial aspect regarding scripting CSCL is that scripts need to be carefully

designed. Incorporating collaboration approaches into educational situations involve risks,

especially when teachers and students are new to collaborative learning (Johnson & Johnson, 1999).

Moreover, over-scripting the situations may be counterproductive. Dillenbourg (2002) states that


22

while a certain degree of coercion is required for efficiency reasons (increasing the probability of

fruitful interaction), excessively constraining collaboration may disturb natural interaction

mechanisms. This fact together with the unexpected circumstances that can appear in CSCL

situations lead to a set of flexibility requirements pointed out by Dillenbourg et al. (2007).

Furthermore, (Häkkinen et al., 2007) points out the concern that the adoption of scripting

environments may not fit the specific situational needs and even create a conflict regarding the role

of the teacher. This definitely depends on who designs the functionality of the learning

environments that support scripting situations. Next section subsection explores PD as a promising

approach to deal with this concern.

2.2.4 Participatory Design and CSCL

As we advance in subsection 2.2.1, CSCL is a field soundly multidisciplinary. Therefore, it is

characterized by the coexistence of a priori very different expectations, requirements, knowledge

and interests posed by both teachers (collaborative learning practitioners) and technologists (ICT

experts), among other stakeholders such as institutions and students (Kirschner et al., 2002). This

fact implies a demanding requisite related to need of communication and mutual understanding

between all these actors (Dimitriadis et al., 2003). This would facilitate explicit thinking about the

border between the technical solutions and their context of use (Häkkinen, 2002). Along these lines,

the PD field appears as an interesting approach to consider when designing CSCL situations.

PD proposes a diversity of theories, practices, etc. with the goal of working directly with users

and other stakeholders in the design of social ICT systems (Muller et al., 1993). That is, PD

methodologies define processes where users and developers work together during a long period of

time, while they interchange values and identify the real requirements of the application. Muller et

al. (1993) include a brief guide to PD practices intuitively situated in a taxonomic space that is

summarized in Figure 2.2. The horizontal axis of the figure indicates the phase within the

development life cycle at which each practice may be useful. The vertical axis illustrates whether

the software professionals participate in the users’ world or it is the other way round. Of course,

both types of collaboration are valuable as well as the mixtures between them.

We argue that, in the CSCL case, PD is not only necessary when performing the identification

and analysis of requirements for the development of the CSCL solutions. On the contrary, we

recognize the importance of participatory modes of designing that go beyond plain communication:

the implied stakeholders should actively participate in the design of the learning environment

functionality according to the necessities of each particular educational situation. In this sense, we

underline the role of the teacher as the actual connoisseur of the concrete situation requirements.

That is, the design of CSCL situations requires of PD approaches where the users (in this case the


23

teacher) participates in the technologist world along the whole development cycle, but with special

emphasis in the late tuning.

Figure 2.2 Taxonomy of PD practices (from (Muller et al., 1993)) Superscripted letters in the figure denote

appropriate group size for the practice: T (tiny, 2-4 participants), S (small, 6-8 participants), M (moderate, up to 40 participants), and B (big, up to 200 participants)

2.2.5 Discussion: research focus and derived challenges

Recapitulating what the introduction of this chapter anticipates, this dissertation focuses on the

design of CSCL situations supported by ICT-based learning environments. Along this section we

have analyzed the requirements imposed by the characteristics of CSCL as well as their

implications regarding the ID and PD fields to the design of such technology-embedded educational

situations.

Thus, a critical concern in CSCL is to promote the elicitation of productive interactions as the

core learning mechanisms to reach educational objectives. From the design perspective it concerns

planning a set of instructions that scaffold the collaborative situation. When these sets of

scaffolding instructions are embedded in learning environments, they are called scripts. Our focus is

specifically on the scripts reflecting pedagogical methods that structure CSCL situations at the

macro-level, i.e. scripts that basically specify flows of activities and the groups (and roles) that

perform these activities.


24

The multidisciplinarity characteristic of CSCL entails the need of participatory modes of

designing. In the case of the scripting environments we put the accent on the role of the teacher,

who is the main stakeholder (and not the technologist) that should decide on the behaviour of the

scripting environment according to the particular requirements of the educational situations. This

PD requirement implies an important concern related to time and cost efforts in the development of

tools devoted to specific scripts (until now a new CSCL application should be developed to

implement each script).

Therefore, the challenges derived from this research focus are formulated as follows:

Challenge 1. It is not trivial to design scripts that potentially elicit the desired interactions.

There are many risks that can appear when putting scripts into practice. They are mainly

related to the lack of experience regarding CSCL (of some teachers and students), the

excessive coercion associated to over-scripting and how the teachers’ conceptions of

learning and the characteristics of the educational situation are consistent with what is

specified in the scripts.

Challenge 2. A modelling language should be used to formalize the scripts so that they are

computer-interpretable. To overcome the problem related to the PD requirement, a

promising approach is to use a computational notation system (e.g. an extensible markup

language (XML) “dialect” (W3C, 1996)) to represent the scripts so that they are

automatically interpreted by software engines integrated in learning environments such as

LMSs. This would allow the design of a computer-interpretable script for each situation

considering its concrete requisites without the need of developing a new scripting

environment.

Challenge 3. Computational representations are not familiar to most of the teachers. The

majority of the teachers do not have advanced technological skills, nor do they always have

the support of technical experts to create or adapt the computationally represented scripts.

The intricate mechanisms and components that comprise the descriptions of scripts

(interrelations of groups, synchronization of collaborative activity sequences, etc.) make these

challenges even more demanding. Concerning Challenge 3, a hopeful solution relies on providing

teachers with graphical authoring tools that generate computationally-represented scripts. These

graphical-based editors should use visualizations, concepts and representations familiar to the

teachers (Griffiths, Blat, García, Vogten, & Kwong, 2005; Griffiths et al., 2005; Harrer & Malzah,

2006; Botturi & Stubbs, in press). However, this solution should be combined with crucial

ingredients that undertake the other two challenges. Challenge 2, on the one hand, demands the

study of the existing EMLs in order to select or propose a computational notation suitable to

represent scripts characterized by complexity.


25

Additionally, with regard to Challenge 1, the direction identified in the previous subsections

refers to the use of design process-oriented methodologies more pragmatic and less rigid than

traditional ID approaches. In this sense, the design processes should be grounded in practice instead

of in general theories (Dimitriadis et al., 2003; Karagiorgi et al., 2005; Kenny et al., 2005). Hence,

the processes should convey a design foundation that provides teachers with well-known

pedagogical strategies that encourage students to interact meaningfully while, at the same time,

these processes should enable the adaptation of the strategies according to specific context,

objectives and peculiarities of the target audience. A key point on the whole is to make the

pedagogical rationale of the critical elements behind the scripts explicit (Strijbos et al., 2004;

Häkkinen et al., 2007; Dillenbourg et al., 2007). Interestingly, very much in line with this

inquietude, design patterns are used in TEL and other disciplines such as Architecture and Software

Engineering as formulations of well-known solutions that can be applied and creatively adapted to

many different situations. Thus, this dissertation investigates the use of patterns to tackle Challenge

1. Next section is devoted to their exploration.

2.3 Design Patterns

Despite the fact that the word “pattern” has been used for centuries with slightly different

meanings (e.g. patterns for dressmaking (Holkeboer, 1987)), their use is more known in the fields of

Architecture (Alexander et al., 1977) and Software Engineering (Gamma et al., 1995). A pattern

provides a means of organizing information regarding a contextualized common problem and the

essence of its broadly accepted solution, so that it can be repetitively applied. A collection of

interconnected (related) patterns which enables the generation of a coherent whole (e.g. a town) is

called a Pattern Language (PL) (Alexander et al., 1977). Recently other domain specific patterns

have been proposed, including TEL and CSCL (Goodyear, 2005; Retalis et al., 2006; Derntl et al.,

2006). However, not all the TEL patterns follow the same approach, nor have the same purpose.

Before providing a unifying view of several approaches with different scopes regarding the use of

patterns in TEL, we introduce the traditional notion of patterns in Architecture and Software

Engineering.

2.3.1 Patterns in Architecture

The first PL was called “A Pattern Language: Towns, Buildings, Constructions” and was

published in 1977 by the architect Christopher Alexander (Alexander et al., 1977). In this book

Alexander defines a pattern as follows. “Each pattern describes a problem that occurs over and

over again in our environment, and then describes the core of the solution to that problem, in such

a way that you can use this solution a million times over, without ever doing it the same way twice.”

Patterns provide a structure for integrating the analysis and solution that can be found to a problem,


26

in a way that is sensitive to context. They suggest or offer guidance but require embellishment (E-

LEN, 2004a).

Alexander, driven by the moral preoccupation regarding the need for a good environment and

for the living structure of built environment (Alexander, 1999), introduces 253 patterns in the

architecture domain. He presents patterns for everything from designing independent regions, to

cities, buildings and even single rooms. By connecting these patterns with common forces and other

relations he transforms this collection of patterns into a PL. The interrelated patterns can be applied

sequentially so that the PL provides a consistent way of creating a comfortable environment for

people to live in.

Alexander aims that all design and construction would be guided by a collection of communally

adopted planning principles: patterns. He differentiates the terms design patterns and construction

patterns. While design patterns refer to understanding the geometry of a building and the

relationships between parts, construction patterns examine the materials and processes needed in

order to put the designs into practice. On page 207 of (Alexander, 1979), Alexander defines a PL as

“really nothing more than a precise way of describing someone’s experience of building”. Different

designers would select diverse subsets of patterns on different occasions and therefore use a

different PL for a building or part of it.

Apart from a vehicle of communication in which structured ideas can be discussed, shared and

modified, Alexander points out three essential features of a pattern language (Alexander, 1999):

- Moral preoccupation; something important and vital that goes, ultimately, to the nature of

human life. In architecture this preoccupation refers to the need for a good environment, and

for the living structure of built environment.

- Aim of creating coherence (morphologically coherent in the things which are made with it).

- Generativeness; create coherence of the created whole, with the power to generate such

whole in a million forms, with infinite variety in the details but with a guarantee of well-

formed results (Alexander et al., 1977).

On page 74 of (Alexander, 1999) two interesting comments are included regarding the

generative approach of patterns: “… one of the characteristics of any good environment is that

every part of it is extremely highly adapted to its particularities. That local adaptation can happen

successfully only if people (who are locally knowledgeable) do it for themselves.” “In our own time,

the production of environment has gone out of the hands of people who use the environment. So,

one of the efforts of the pattern language was not merely to try and identify structural features

which would make the environment positive or nurturing, but also to do it in a fashion which could

be in everybody’s hands, so that the whole thing would effectively then generate itself.” It is


27

noteworthy how related are these ideas to the challenges that this dissertation faces (cf. subsection

2.2.5).

In Alexander’s latest 4 books of the series “The Nature of Order” (Alexander, 2003a;

Alexander, 2003b; Alexander, 2004a; Alexander, 2004b), he deepens in the “search for beauty” and

defends the existence of criteria and methods to determine if something is objectively beautiful and

what degree of live has. The core of the new theory presented in those books is the theory of

structure preserving transformations, which provide means for any step-by-step process to reach

configurations that are most capable of supporting life. A structure-preserving transformation is one

which preserves, extends, and enhances the wholeness of a system (Alexander, 2003b). This theory

relies on the idea of “centers” (Alexander, 2003a). A center is a kind of psychological whole entity

that is felt as a center in the visual field. Each living center is always created by configuration of

other centers (Alexander, 1999). During more than 20 years, Alexander examined objects for life

and wholeness. He identified 15 structural features which appear again and again in things which

have life: levels of scale, strong centers, boundaries, repetition, positive space, good shape, local

symmetries, deep interlock and ambiguity, contrast, gradients, roughness, echoes, the void,

simplicity and inner calm and not-separateness.

Example of Alexandrian patterns can be found mainly in (Alexander et al., 1977). For clarity

purposes and in order to present each pattern connected to other patterns, Alexander uses the same

format for each pattern (Alexander et al., 1977; E-LEN, 2004a). The relations with other patterns

are indicated in the introductory paragraph, setting the context of the pattern, and the final

paragraph, explaining how it can be embellished or completed. The essence of the problem and the

solution are highlighted in bold type. The solution is also shown in form of a diagram, with labels to

indicate its main components. This diagram is a key element to promote a good understanding of

the solution. Moreover, the patterns are marked with zero, one or two asterisks (*), which show how

“alive” we believe the pattern is. Two asterisks denote patterns that state invariants, i.e., these

patterns are essential and that the associated problem cannot be solved in a significantly different

way. One asterisk means that they are on the right track, but they can be improved. For patterns

with no asterisk, perhaps there are better ways of solving the problem, but they have not been found

yet.

2.3.2 Patterns in Software Engineering

At the beginning of the nineties the software community started using Alexander’s technique to

capture and communicate expertise in software design in general and object-oriented design in

particular. Although the movement started in universities and conferences such as OOPSLA

(Object-Oriented Programs, Systems, Languages, and Applications), the first book that was publicly

available was “Design Patterns” by Gamma, Helm, Johnson, and Vlissades called the Gang of Four


28

(GoF) (Gamma et al., 1995). It was published in 1995, and the authors presented a catalogue of 23

patterns that are a prescription of how to solve particular problems that come up in software

development.

Patterns in Software Engineering provide a common vocabulary, a common base of

understanding regarding what is important in software (vs. Architecture) and a large corpus of

solutions that makes developers more effective, i.e., they represent best practice, proven solutions,

and lessons learned, which aid in evolving software engineering into a mature engineering

discipline (Tesanovic, 2001). They name, abstract, and identify the key aspects of a common design

structure that make it useful for creating a reusable object-oriented design. Thus, patterns in

software identify the participating classes and instances, their roles and collaborations, and the

distribution of responsibilities. Each pattern focuses on a particular object-oriented design problem

or issue. It describes when it applies, whether it can be applied in view of other design constraints,

and the consequences and trade-offs of its use. On page 3 (Gamma et al., 1995) defines patterns as

“descriptions of communicating objects and classes that are customized to solve a general problem

in a particular context”.

Among other aspects, Alexander’s and Gammas’ patterns diverge in their formats and tackle

different types of problems. Table 2.1 collects their similarities and differences (Gamma et al.,

1995; Alexander, 1999). The table does not intent to be complete but to comparatively summarize

their main characteristics.

Although Gamma’s patterns are the most popular among the software community, there are

now many other types of software patterns proposed in the literature (Buschmann, Meunier,

Rohnert, Sommerlad, & Stal, 1996; Riehle, 1996; Tesanovic, 2001). These patterns includes

architectural patterns (high-level strategies concerned with large-scale components as well as global

properties and mechanisms of a system), design patterns (micro-architectures of subsystems and

components), idioms (low level, paradigm-specific or language-specific programming techniques

that fill in low-level internal or external details of a component’s structure or behaviour). In

addition, there are approaches around domain-specific patterns and organizational patterns that

describe software process design (Coplien & Harrison, 2005).

Now that the main ideas regarding patterns in Architecture and Software Engineering are

reviewed, next subsection explores some important initiatives that aim at applying the pattern

approach to the educational domain, and especially to TEL and CSCL.


29

Table 2.1 Comparison between Alexander’s and Gamma’s patterns

Alexander’s patterns Gamma’s patterns

Both adopt similar definitions of pattern: at the core of both kind of patterns is a solution to a problem in a context Patterns in buildings and towns: solutions in terms of walls and doors

Object-oriented design: solutions in terms of objects and interfaces

Emphasize the problems they address Describe the solutions in more detail The solutions do not describe a particular final design, they are general principles or templates that can be used in different situations

Definition

Vehicles of communication

Both are based on observing existing systems and looking for patterns in them Identification There are many classic examples of

buildings Few software systems can be considered classic

Both use structures or formats for describing patterns

Structure / format

Fields of the format: a picture showing an archetypal example of the pattern, name of the pattern, an introductory paragraph setting the context for the pattern (explaining how it helps to complete some larger patterns), ‘***’ to mark the beginning of the problem, a headline in bold type to give the essence of the problem in one or two sentences, the body of the problem (the forces) – its background, evidence for its validity, examples of different ways the pattern can be manifested, the solution in bold type (this is the heart of the pattern, stated as an instruction, so that you know what to do to build the pattern), a diagrammatic representation of the solution, ‘***’ to show the main body of the pattern is finished, and a paragraph tying the pattern to the smaller patterns that are needed to complement and embellish it.

Fields of the format: name and classification (creational, structural or behavioural), intent (problem that the pattern addresses), also known as (other well-known names of the pattern) motivation (a scenario illustrating a design problem), applicability (situations where the pattern can be applied), structure (a graphical representation of classes in the pattern), participants (classes and objects and their relationships), collaboration (participants collaborating to carry out responsibilities), consequences (trade-offs and results of using a pattern), implementation (things to be aware of when implementing a pattern), sample code (code fragment illustrating one implementation), known uses (example of patterns found in real systems), related patterns (other closely related patterns).

Natural language

Both rely on natural language and many examples to describe patterns rather than formal languages (although Gamma et al. discuss about defining a formal representation of patterns to make automating patterns possible as they state in (Gamma et al., 1995).

Moral preoccupation

Under what circumstances is the environment good?

Not clear whether there is or not a moral preoccupation (Alexander, 1999)

Generativeness Generate complete buildings (pattern language)

Do not generate complete programs (catalogue of related patterns) (Currently there are proposals of generative PLs, which are designed to shape system architecture and can generate systems or parts of systems (Tesanovic, 2001))

Processes There is an order in which the patterns should be used

There is no order suggested to apply the patterns (Currently there are proposals that indicate processes to apply patterns (Coplien & Harrison, 2005))


30

2.3.3 Patterns in TEL and CSCL

A rather new and promising approach in TEL is to identify and collect best practices and

formulate them as design patterns that are gathered in repositories which will be eventually enlarged

with the addition of new patterns. Some projects which follow this objective are (PoInter, 2001; E-

LEN, 2004b; PPP, 2005; TELL, 2005b). In this context, patterns reflect the knowledge of experts in

a particular educational domain (e.g. CSCL) and they capture common solutions to recurrent

problems in an educational scenario. Since designing effective TEL scenarios is a complex

problem, design patterns based on sound research can help to guide the design process. Five main

advantages for adopting a patterns approach in e-learning design are pointed out in (E-LEN, 2004a)

and briefly reproduced below:

- Patterns are both empirical and normative, but not prescriptive. Capture expert practice in

specific context.

- Patterns have an internal structure that is good for action-oriented evidence-based advice.

Capture the most important aspects of a problem and a solution in a standard format with a

formalism, which documents and justifies the rationale.

- There is expressive and normative power in the relation between patterns.

- The pattern-based approach is inherently democratic and inclusive.

- Patterns can help to enrich the language of educationalists and to provide a common

nomenclature for designers. They can facilitate communication within interdisciplinary and

multi-perspective teams.

Patterns can be identified and constructed using mainly two methodologies: inductive pattern

mining (from specifics to generalizations), by analyzing common solutions in a set of educational

situations, or deductive pattern mining (from generalizations to specifics), by capturing the essence

of generic models for solutions to recurrent problems that experienced learning designers identify

(Baggetun et al., 2004; Retalis et al., 2006).

2.3.3.1 Different interconnected scopes of TEL patterns and related fields: focus on CSCL Figure 2.3 aims at relating existing pattern proposals that refer to different scopes of the TEL

field. In general two main types of patterns can be distinguished depending on their use. Firstly,

“patterns for analysis” deal with analyzing the usage of TEL systems in training or academic

contexts, in order to help the implied stakeholders continually improve them (PoInter, 2001). The

PoInter project is concerned with investigating the appropriateness of patterns as a means of

communicating information about how people interact with each other through technology (PoInter,

2001). This type of patterns may be also classified as patterns of interaction or patterns of behaviour

(Rada & Hu, 2002).


31

Patterns for TEL

Pedagogical

(CS)CL

(Technological) LMS

Design

Analysis of interactions/ behaviour

Analysis

Learning objects

CSCW

Figure 2.3 Relations between some proposals regarding patterns in TEL

Secondly, “patterns for design” are devoted to the design of TEL environments. This is the wide

scope of E-LEN project (E-LEN, 2004a; E-LEN, 2004a), which proposes patterns for implementing

an institutional e-Learning centre. Of a more specific scale are patterns devoted to particular

didactic areas, for example (Learning Patterns, 2005) is a project investigating patterns for the

design, development and deployment of games for mathematical learning.

Within this “patterns for design” scope, patterns for designing learning scenarios (pedagogical

patterns) and patterns for designing technological solutions that supports these scenarios

(technological patterns) can be differentiated. On the one hand, pedagogical patterns try to capture

expert knowledge of the teaching/learning practice in different educational situations, including

blended learning (Derntl & Motschning-Pitrik, 2004). This type of patterns propose solutions for

problems such as motivating students, choosing and sequencing materials, or evaluating students

(PPP, 2005). Some patterns for (CS)CL can be classified as a type of pedagogical patterns. It is

remarkable that a number of patterns developed within the E-LEN project are related to CSCL (E-

LEN, 2004b). However, the project devoted specifically to the development of patterns for CSCL is

the EU-funded e-Learning TELL project (TELL, 2005b), in which the GSIC/EMIC group has

participated. A deliverable of this project is a book with a collection of patterns representing key

aspects of CSCL (TELL, 2005a).

Some of the outcomes of E-LEN are presented in (Baggetun et al., 2004; Ronteltap, Goodyear,

& Bartoluzzi, 2004) and mainly in (Goodyear et al., 2004). This latter work suggests that design

patterns offer a useful method for sharing design ideas in participatory educational design work and

includes a pattern for “Discussion Group”. (Goodyear, 2005) deepens in these issues and illustrates

a pattern language for “Debate”. In these papers Goodyear conceptualises the problem space of

educational design differentiating the learning task, the organisational form and the space of tools


32

and artefacts. Once these three dimensions are well designed, students will configure their own

personal space, play an activity and construct a learning community, which are not (totally)

prescribed or determined (and thus cannot be predicted) by the educational designer.

On the other hand, technological patterns proposed in (Avgeriou et al., 2003) record design

experience with regard to the construction of LMSs (some of these patterns consider CSCL

requirements, e.g. those related to communication facilities). Similarly, patterns for ITS

architectures are collected in (Devedzic & Harrer, 2005). Focusing on learning objects reusability,

Jones (2004) proposes the use of patterns to produce reusable designs for creating learning

resources that are adaptable.

Patterns from other disciplines or fields can also be useful in the design of TEL. For example,

CSCL can be greatly benefited of knowledge in “groupware” or CSCW (cf. subsection 2.2.1). An

example of patterns for groupware is GAMA, a PL that provides patterns for supporting dynamic

teams using computer technology (Schümmer, 2003; Schümmer & Lukosch, in press). Patterns in

collaborative system design, development and use are also proposed in (Guy, 2003) and (David,

Delotte, Chalon, Tarpin-Bernard, & Saikali, 2003). For instance, one of the examples included in

(David et al., 2003) is about the management of a dialog among many participants. The pattern

reflects a generic process regarding floor control (give hand, ask hand, free hand) that can be

particularized into different processes that share the same principle of execution, i.e., use this kind

of floor control (e.g. a process for managing shared resources).

It is necessary to remark that there are not clear boundaries among the different approaches.

Patterns that result from interaction analysis are related (or can belong) to the domain of CSCL and

CSCW. Patterns for designing ANSCL (Asynchronous Network Supported Collaborative Learning)

systems (Georgiakakis & Retalis, 2006) can be considered both patterns for CSCL and patterns for

LMSs.

Having in mind important initiatives around TEL patterns, this dissertation aims at identifying

the types of patterns and the types of connections that link them with the objective of enabling the

generation of potentially effective scripts as an approach to solve the Challenge 1 (cf. subsection

2.2.5).

2.3.4 Discussion: the generative problem and the generation of potentially effective scripts

Baggetun et al. (2004) point out that TEL patterns are closer to Alexander’s notion of design

patterns than to Gamma’s view. There is a moral preoccupation around these patterns: under what

circumstances is TEL more efficient and effective? We agree with this statement but we also assert

that the generative power of TEL patterns is closer to Software Engineering patterns in the sense

that what TELL patterns describe should be actually supported by software.


33

Alexander discusses the “generative problem and the generation of a living world” (relating this

problem to software) on page 79 of his keynote address in a conference on Object-Oriented

technologies (Alexander, 1999):

“I have, for many years, thought that this could only be solved by a genetic approach – an

approach where deep structure, spread through society, creates and generates the right sort of

structure, very much as genetic code creates and generates organisms and ecological systems –

indirectly by letting loose life-creating process.

That is what I still believe. But, today, I am convinced that the equivalent of the genes that act

in organisms will have to be – or at least can be – software packages acting in society. If these

software packages are life creating, and accepted, and widely enough spread throughout the world,

there is a chance we might get a grip on this problem: provided that the software is freeing,

liberating, allows each person individual control and decision making power to do the right

thing…”

In the case of TEL patterns, and particularly patterns for CSCL scripts, the generative problem

should go beyond the dissemination of these patterns in repositories, what is done until now (PPP,

2005; E-LEN, 2004b; TELL, 2005b). In contrast, it should be faced by means of providing users

(teachers, learning designers) with authoring tools that incorporate such patterns. According to

Alexander’s metaphor, the genes would be the functionalities of the tools that facilitate the creation

and generation of the right sort of collaboration scripts. If the functionalities of the tools, based on

the patterns, promote effective collaborative learning and are accepted and widely enough spread

through the educational institutions, there is a chance we might enhance learning effectiveness:

provided that the authoring tool allows users control and decision making power to design the right

script for their particular leaning situation.

This approach would provide a design solution grounded in scripting good practices that can be

particularized according to the concrete characteristics of an actual situation, where the “centers”

that appear again and again in the (potentially effective) scripts are the critical elements that likely

elicit the desired interactions. Moreover, not only would this solution overcome Challenge 1, but

the orientation inspired by Alexander also covers Challenge 3. The teachers should be the target

audience of the pattern-based authoring tools and, therefore, the technical details of the script

computational representation need to be hidden. Approaching Challenge 2, the following section is

devoted to review EMLs, as notation candidates to formalize the scripts.


34

2.4 Educational Modelling Languages

Learning Technology standards and specifications are growingly supporting the professional

activity of instructional designers (Rodríguez-Estévez, Caeiro-Rodríguez, & Santos-Gago, 2003).

The most widespread approach refers to metadata specifications (e.g. LOM) to describe reusable

chunks of learning content (so-called Learning Objects or LO) that can be used and referenced in

TEL environments (Hodgings, 2000). Recently, the standardization efforts are moving from content

delivery resources to EMLs, which are focused on specifying teaching-learning processes that stress

the importance of the activities. The definition of an EML is pointed out on page 8 of a survey of

EMLs (Rawlings et al., 2002):

“An EML is a semantic information model and binding, describing the content and process

within a ‘unit of learning’ from a pedagogical perspective in order to support reuse and

interoperability.”

This survey analyzes whether potential (non-proprietary) EMLs comply with the stated

definition. In the analysis it is underlined the concept of ‘unit of learning’ (also called ‘unit of

study’ or ‘unit of instruction’), which describes the learning design, the resources and the services

needed in order to achieve one or more interrelated learning objectives. A unit of learning

commonly refers to a course, module, lesson or curriculum. The conclusions of the survey

(accomplished in 2002) are that only OUNL-EML (Hermans, Manderveld, & Vogten, 2004) and

PALO (Rodríguez-Artacho & Verdejo-Maíllo, 2004) languages actually match up the definition of

an EML, with the nuance that OUNL-EML exceeds the expressive possibilities of PALO.

According to the survey, among other features, PALO is limited to the specification of individual

tasks.

OUNL-EML is the base of the IMS LD specification (IMS, 2003b), which is currently accepted

as a de facto standard for formalizing teaching-learning processes (Koper et al., 2004). On the other

hand, other approaches to ID languages involving similarities to EMLs are presently being proposed

as competing or complementary approaches to IMS LD. We present them in this section after

introducing the main issues around IMS LD.

2.4.1 IMS Learning Design

IMS LD Learning Design (IMS, 2003b) realised by the IMS Global Consortium (one of the

major bodies developing interoperability specifications for TEL) in 2003 (IMS, 2001), is the most

established EML at present (Koper & Tattersall, 2005; Burgos & Griffiths, 2005). The aim of IMS

LD is to enable the creation of complete, abstract and portable description of the pedagogical

approach taken in a course (or part of a course), which can be realized by a conforming systems.

The key idea is that it represents the learning activities (performed by learners) and the support


35

activities (performed by teachers), including those comprising multi-role teaching-learning

processes and personalized learning routes (Koper et al., 2004).

2.4.1.1 Elements, features and organization of the specification The conceptual model of IMS LD is shown in Figure 2.4. A Learning Design (an LD) is a

description of a method enabling learners to attain particular objectives by performing (learning and

support) activities in a certain order in the context of a learning environment. The environment

consists of the appropriate learning objects (e.g. pictures, books, software simulation) and services

(e.g. forums, chats) to be used during the performance of the activities. A method contains the play,

which is modelled according to the metaphor of a “theatrical play” with consecutive acts and role-

parts, which indicate the activities performed by each role participating in an act. IMS LD

integrates other specifications, such as IMS QTI (IMS, 2006) and IMS Meta-data (IMS, 2002), to

support the portable representation of units of learning (when they IMS LD compliant they are

usually written with capital letters). A Unit of Learning (UoL) is a content package (IMS, 2004a)

including an LD and a set of physical resources or their location. Further detailed information of

IMS LD elements can be found mainly in the specification (IMS, 2003b) and in a handbook

devoted to it (Koper & Tattersall, 2005).

Figure 2.4 The conceptual model of IMS LD (extracted from (IMS, 2003b))


36

As other specification and standards, IMS LD promotes the reuse and interoperability of LDs in

different TEL environments (Koper et al., 2004). This is facilitated thanks to the fact that IMS LD is

a generic language for formally expressing (computationally representing) educational situations

making use of a wide range of pedagogies, in such a way that it enables repeated execution in

different settings and with different participants (Tattersall et al., 2005). Therefore, the specification

includes an Information Model (complete description of the elements of the conceptual model), its

XML Binding (the binding of the information model conforming to the XML 1.0 Specification of

the W3C (W3C, 1996)) and a Best Practices and Information Guide (with examples as use cases).

Additionally, it considers three levels of implementation and compliance (IMS, 2003b; Koper et al.,

2005):

- Level A contains all the core aforementioned vocabulary needed to support pedagogical

diversity.

- Level B adds properties and conditions to level A, which enable personalization and more

elaborate structures. By also using global elements and monitoring services, it enables to

direct the learning activities as well as record personal, group or collective outcomes.

- Level C adds notifications to level B, which, also enables to control the flow of activities or

to send alerting e-mails.

Although LD does not prescribe any methodology for the creation of UoLs, Designer’s Guide,

included in (IMS, 2003b), proposes general stages for creating a UoL. (Sloep, Hummel, &

Manderveld, 2005) details these stages according mainly to three phases. The first phase involves

the analysis of a specific educational problem, whose result is a narrative description of the

educational situation. Then, the narrative is translated into a UML (Unified Modelling Language)

activity diagram (Arlow & Neustadt, 2001) in the design phase. The diagram is the basis for the

XML instance IMS LD compliant document. In the fourth phase the resources are developed (if

necessary) and added to the design. Thus, a UoL is created.

2.4.1.2 Interoperable IMS LD compliant tools The abovementioned basic design procedures are useful if the users that create the UoL are IMS

LD experts. However, other types of users require more abstract procedures that facilitate the design

of UoLs. This is to a large extent related to the type of authoring tool that may implement those

procedures (Griffiths et al., 2005). Griffiths et al. (2005) classify the tools according to two

dimensions according to their type of user (technical expert, instructional designer, teacher) and

their degree of pedagogical specialization:

- Higher vs. lower level tools (or distant from specification vs. close to specification). This

dimension is related to the level of expertise on IMS LD required by the user of the tool.


37

That is, how much the tool interface is influenced by IMS LD or how many IMS LD details

it hides.

- General purpose vs. specific purpose tools. This dimension deals with the pedagogical scope

of the tools. Teachers using a clearly defined pedagogic approach would not need all the

capabilities of the IMS LD specification. This implies that authoring tools more tightly

focused on that particular pedagogical approach might present to their users only the needed

functionality, reducing significantly the complexity of authoring.

Among the IMS LD compliant editors, RELOAD (Davis, 2002), CopperAuthor (van der Vegt,

2005) and CoSMoS (Miao, 2005) are examples of general purpose editors that are close to the

specification. Their users are IMS LD experts that are not focused on a particular pedagogy.

Similarly, MOT+ editor (de la Teja et al., 2005) and ASK-LDT (Sampson et al., 2005) are intended

also for expert learning designers rather than teachers, although they provide graphical

representations that facilitates to a certain extent the authoring task. On the other hand, LAMS

(LAMS, 2006) is a specialized editor because it offers a set of predefined learning activities, shown

in a comprehensible way for teachers, that can be graphically drag and drop in order to establish a

sequence of activities. Nevertheless, although LAMS is inspired by the same LD philosophy, it is

not LD compliant at the present time.

On the other hand, there are several IMS LD compliant players or LMSs available or under

development at present. Some of them are based on CopperCore engine (Martens et al., 2005), such

as RELOAD LD player or Gridcole (Bote-Lorenzo, 2005), and others include their own IMS LD

engine, such as .LRN (E-LANE, 2004).

Though the amount of developments around IMS LD is significant, its adoption in real practice

is however in its infancy (Burgos & Griffiths, 2005). This is not strange since the specification has

been released recently, as well as the corresponding tools. However, this fact makes unclear the

possibilities and limitations of IMS LD for representing specific requirements of educational

scenarios, including CSCL situations (Caeiro-Rodríguez et al., 2003). This open up the path to other

proposals of related EMLs approaches.

2.4.2 Other approaches

In addition to IMS consortium, ISO/IEC JTC1/SC36 develops standards in the domain of TEL.

Particularly, WG2 is devoted to collaborative technology (ISO/IET, 2002). WG2’s efforts are

focused on standardizing the Collaborative Workplace (data collection and reuse for collaborative

environments), Learner to Learner Interaction Scheme (peer-to-peer and group), and Agent/Agent

Communication (agent-based interfaces in collaborative environments). However, its progress in

the last four years has been limited and no official outcome has yet become available. In fact, its


38

proposal regarding Learner to Learning Interaction Scheme has similarities to IMS LD and the WG

has recently agreed to re-name this project to “Data Model for Text-based Communication by the

Study Group”.

Besides, other ID languages are emerging as complementary approaches (e.g. providing visual

representations) or alternatives to IMS LD (Vignollet et al., 2006; Botturi & Stubbs, in press). A

selection of them are comparatively categorized in (Botturi, Derntl, Boot, & Figl, 2006) according

to a classification framework that defines IMS LD as a notation system that is formal, textual, and

of a single perspective. We would classify almost similarly the newly proposed Learning Design

Language (LDL) to model collaborative activities (Martel, Vignollet, Ferraris, David, & Lejeune,

2006b), which also includes a graphical representation associated to each concept.

On the contrary, some available notations do not provide a stringent set of rules for describing

designs but more open, typically visual (in contrast to textual), ways of informally describing

educational situations, such as E2ML (Educational Environment Modelling Language) (Botturi,

2006) or the proposal of an AUTC (Australian Universities Teaching Committee) project (Oliver,

Harper, Hedberg, Wills, & Agostinho, 2002; Bennett & Lockyer, 2004). E2ML, in a similar spirit to

UML, is a multiple-perspective language. It makes use of different diagrams to provide different

views of the same designs (e.g. chronological vs. structural perspectives). In the same way, the

ongoing PoEML (Perspective-oriented EML) proposal tackles the design of educational units from

different perspectives aiming, at the same time, at achieving precision regarding the implementation

details (Caeiro-Rodríguez, Llamas-Nistal, & Anido-Rifon, 2006a; Caeiro-Rodríguez, Llamas-

Nistal, & Anido-Rifon, 2006b).

2.4.3 Discussion: IMS LD as a candidate to computationally represent CSCL scripts

The competing approaches to IMS LD have been proposed during the realisation of this

dissertation. That is the reason why we initially select IMS LD as a potential candidate to

computationally represent the CSCL scripts, so that it represents a solution to Challenge 2 (cf.

subsection 2.2.5). (We use simply LD instead of IMS LD henceforth in order to achieve a more

readable text, when the context does not lead to misunderstandings with related approaches.) The

scripts describe collaborative learning situations that, according to the definition of the

specification, LD supports. Moreover, LD is an open specification agreed upon by domain and

industry experts (members of the IMS consortium), what envisages promising interoperability

prospects. If the scripts are formalized using LD they could be executed in any LD compliant

environment, with the associated benefits in terms of costs (reusability, adapted reproducibility,

etc.) that it provides.


39

However, as we have previously advanced, the LD support for realizing scripts is not clear. This

is partially motivated by a lack of significant examples and efforts that show the possibilities of LD

for CSCL. Although partial work has been already accomplished (Gorissen et al., 2005; Koper et

al., 2005), a more complete and systematic analysis is needed. On the other hand, the audience on

which we focus the problem of designing computer-interpretable scripts are teachers that are not

necessarily technologists, nor LD experts. In this sense, if the LD notation is selected as appropriate

to express scripts, the mentioned LD compliant authoring tools would not be suitable to cover

Challenge 3.

2.5 Conclusion

In CSCL one way to use the technology (in this case ICT) so that it facilitates and improves the

teaching and learning experience is by means of (macro-)scripts supported by learning

environments. However, the design of such CSCL situations is not trivial and involves three main

challenges that are identified along the chapter. The conclusions about the challenges and the

envisaged potential solutions are developed next.

Challenge 1 is around the demanding aspects implicated in the design of potentially effective

scripts. On the one hand, the scripts should structure the organization of activities and groups so that

the situation promotes the elicitation of fruitful interactions that lead to learning outcomes. On the

other hand, the risk of excessive coercion and collaboration in general (especially when proposed

by novice CSCL practitioners) should be strongly considered. The same complexity that demands

flexibility claims a meticulous previous design of the CSCL situations.

Therefore, re-inventing and designing scripts for a particular educational situation is costly and

risky. In this sense, a solution would be unravelling content, tools, specific learning tasks, etc. from

the structure of the script (it may represent the essence of the collaborative learning design) so that

the structure (or the different script components and mechanisms reflecting designs solutions) can

be applied and reused in different learning situations and contexts. The effort involved in

developing separated generic reusable elements of scripts is justified if they have been extensively

tested and applied in a broad range of different settings. In this way, scripts based on these

commonly used elements (good or best practices) represent potentially effective scripts. These good

practices can be formulated as patterns so that they are written in structured formats that make

easier their understanding, (re)use, classification and relation among them.

Not only are the time and cost demands imposed by the design of the situations but also for their

implementation in the learning environments (e.g. LMSs). Challenge 2 proposes to use a

computational notation system to formalize the scripts so that they are interpreted by the

environments (without the need of software development efforts). The complex mechanisms and


40

components that comprise the descriptions of scripts impose difficult requirements to a candidate

notation. Among the EML proposals, the declaration of intent and interoperability prospective of

LD specification lead us to select it as the most interesting aspirant.

Altogether foresees a promising approach that enables participatory modes of designing in

which the teacher is the main stakeholder deciding on the behaviour and functionality of the

scripting environment according to the particular requirements of the educational situations. The

incorporation of patterns (for scripting CSCL) in design processes would provide more pragmatic

methodologies, grounded in practice, than traditional ID approaches. These design processes would

provide teachers with customizable well-known pedagogical methods according to CSCL critical

elements. Furthermore, the patterns would embody representations and abstractions that are easier

to understand by teachers (interested in CSCL) than the details of some IMS LD elements. If the

pattern-based design processes are graphically integrated into (high-level) authoring tools, teachers

without advanced LD knowledge would be able to create their own scripts, thus overcoming

Challenge 3.

CHAPTER THREE

CONCEPTUAL MODEL FOR CSCL

SCRIPTING PATTERN LANGUAGES

This chapter tackles the objective of identifying the types of patterns and relationships between patterns that can be used in the design of CSCL scripts. Its contribution consists in a conceptual model for CSCL scripting pattern languages and a specific pattern language that is compliant with the model. This contribution aims at providing to the community a starting point towards an agreed structure for the production of patterns for CSCL scripting. In this dissertation it specifically serves to situate the rest of its contributions. In order to validate the feasibility of the model, the chapter also discusses three authentic situations that apply a sequence of interconnected patterns selected from the proposed pattern language.

The hierarchical structure for CSCL scripting patterns characterized by the conceptual model is introduced in (Hernández-Leo et al., 2006d), while (Hernández-Leo et al., 2006) presents a real experience that applies a script generated with the proposed pattern language.

3.1 Introduction

How can design processes grounded in practice for creating CSCL scripts be provided? The

previous chapter identifies patterns as a promising way of formulating and sharing experience

regarding the design of potentially effective scripted collaborative learning situations.

Design patterns capture reusable knowledge about a problem and its associated broadly

accepted solution. They are useful to understand the critical elements that should be considered

within a design process. Patterns are decoupled when they are applied. However, they work

together with other interconnected patterns to generate emergent contextualized wholes. A Pattern

Language (PL) embraces a set of patterns relevant to a specific design space and the rules that put

together the patterns in meaningful ways so that they provide guidance when building a space-

related whole.


42

The previous chapter reviews the pattern-based approaches adopted in Architecture, Software

Engineering and, also, in TEL. Besides, it provides a unifying view that link several representative

proposals with regard to patterns in TEL and related fields differing in scope and perspective. This

chapter is devoted to the identification of the types of patterns and connections between patterns

that can be used for generating CSCL scripts. These types of patterns and relationships are

formulated as a conceptual model (or meta-language) for describing CSCL scripting PLs. That is to

say, CSCL scripting is the design space of the patterns and rules that can be situated in the proposed

conceptual model. This conceptual model aims at providing to the scientific community an

important starting point towards an agreed high-level structure for the production of patterns and

PLs that enable the generation of CSCL scripts. Each institution or community of practice may have

its own patterns of effective scripted CL situations (which somehow typify the community). The

sharing and communication of these good practices within and between communities can be

fostered if they are framed within the same conceptual model.

To illustrate the feasibility of this proposal, Appendix A includes a CSCL scripting PL (with

own and adopted patterns) that can be described with the conceptual model. The PL comprises 18

patterns. Each pattern documents the relationships to other patterns. The map of relationships

sketches many ways in which the patterns may be put together when creating different CSCL

scripts. Depending on the context derived from a particular educational situation, different patterns

and connections of patterns may or may not apply. Nevertheless, it is important to point out that the

PL is not complete as a set in the sense that their patterns cannot be used to generate any CSCL

script. Each community can augment the PL with its patterns or propose a different one (that might

borrow some of the considered patterns).

Furthermore, the chapter exposes three different scripted CL situations that can be generated

using the proposed PL. These situations express and illustrate the relationships between the patterns

and exemplify how the diverse types of CSCL scripting patterns can be applied.

Therefore, this chapter is structured as follows. Section 1.2 introduces the methodology applied

to propose the model and the illustrating PL. The conceptual model for CSCL scripting PL is

presented in section 3.3. It is presented in form of an aggregation model (according to the

granularity and scope of the types of CSCL scripting patterns) and description of types of pattern

connecting rules. A hierarchical structure depicting how the PL included in Appendix A fits in with

the conceptual model is also introduced in this section. In addition, it contains a discussion of how

this model is linked to other potential models of related PLs and points out some general guidelines

for applying the PLs that can be described with the conceptual model. Section 3.4 is devoted to

exposing the three situations based on interrelated patterns embraced in the PL. One of them is

explained in more detail since it is designed from scratch using the PL (the other two are used as

case studies to identify some of the proposed patterns). Section 3.5 discusses the potential

CONCEPTUAL MODEL FOR CSCL SCRIPTING PATTERN LANGUAGES

43

computer-supported applicability of CSCL scripting patterns. Finally, the conclusions of this

chapter are presented in section 3.6.

3.2 Methodology applied for proposing the model and the pattern language

Chapter One describes the research methodology adopted in this dissertation in detail. The

phases involved in this chapter are: the informational, the propositional and the analytical phase.

Some of the conclusions resulting from the informational phase are reported in Chapter 2,

which includes a review of important pattern-based approaches in TEL. These conclusions together

with the significant influence related to the participation in the GSIC/EMIC multidisciplinary

research team (GSIC/EMIC, 1994) and the TELL project (TELL, 2005b) are considered in the

propositional phase. In this phase the conceptual model for CSCL scripting PLs and the patterns

included in the illustrating PL are proposed. The analytical phase includes a concept

implementation regarding the description of the PL with the proposed conceptual model. It also

analyzes the generative characteristic of the PL by providing educational situations drawn from real

practice which apply a set of interrelated patterns (belonging to the proposed PL).

Nevertheless, the tasks included in the different phases are actually realized in several iterations.

Before proposing the final version of the conceptual model, most of the patterns in Appendix A are

formulated by us or adopted from other authors. Some of these patterns are identified using the

information provided in the literature, following a deductive or top-down approach (Baggetun et al.,

2004) but others are discovered in case studies, using a more inductive or bottom-up approach

(Brouns et al., 2005). These case studies are two of the three educational situations that we provide

to illustrate how the CSCL scripting patterns can be applied. This approach is commonly used to

show the feasibility of a PL (Coplien & Harrison, 2005; Lukosch & Schümmer, 2006). The third

situation is strictly generated using the PL from the scratch. Describing the pattern-based generation

of the situations is also helpful to reflect and illustrate the types of relationships between the

patterns that form meaningful sequences.

Next subsections look at the techniques we use to formulate the patterns, that is: the problem,

forces and solution for each pattern. We capture the patterns in a pattern form similar to the

Alexandrian format, as it is approached in the TELL project and in other proposals such as (Coplien

& Harrison, 2005; Goodyear, 2005). The pattern format organizes the important components of a

pattern. The body of each pattern starts with an introductory paragraph setting the context in which

that pattern applies. This paragraph explains how the pattern is related to other larger patterns. A

problem may arise in that context; therefore, the essence of the problem comes next in bold type.

After that, the pattern elaborates on the background of the problem, the evidence for its validity and

a description of the forces that define the problem and conduct to the solution. Then, the pattern


44

presents a proven solution, also in bold type, which solves the stated problem in the stated context.

A diagrammatic representation of the solution is often enclosed, which makes easier its

understanding. Finally, the pattern concludes with a paragraph relating the pattern to the smaller

patterns which are needed to embellish it.

We also use asterisks to show how mature the patterns are. Two asterisks denote that they

benefit from many years of experience and research and we are sure of their maturity and value.

One asterisk means that they are on the right track, but they can be probably improved. Patterns

without asterisks indicate that more research should be accomplished to state that the proposed

solution reflects the “best” way of solving the problem.

3.2.1 Capturing the experience reported in the literature

Some of the patterns collected in Appendix A have been identified and constructed according to

the experience broadly reported in the literature. In other words, the resulting patterns represent

best/good practices (when scripting CL situations) that have been extensively tested and applied in a

broad range of different situations (including different content and disciplines) and on which there

are many publications on research or practical results. Therefore, the resulting patterns actually

derive from practice (didacticism used in the practice) rather than from general learning theories

(Johnson & Johnson, 1999; King, 2007; NISE, 1997).

One very well-known example in this sense is the “Jigsaw” strategy, which is introduced by

Aronson & Thibodeau (1992) and Aronson & Patnoe (1997) but also applied by other authors such

as Clarke (1994) or DiGiano et al. (2003), among many others. A generalization of the strategy is

formulated as the JIGSAW pattern in Appendix A (Pattern 1.1). Briefly, it proposes that in order to

solve a complex problem that can be easily divided into independent sub-problems, each participant

in a (small) group (“Jigsaw Group”) studies or work around a particular sub-problem. The

participants of different groups that study the same problem meet in an “Expert Group” for

exchanging ideas. These temporary groups become experts in the subproblem given to them. At

last, participants of each “Jigsaw group” meet to contribute with its “expertise” in order to solve the

whole problem. Some of the educational objectives this strategy favours are: to promote the feeling

that team members need each other to succeed (positive interdependence), to foster discussion in

order to construct students’ knowledge and to ensure that students must contribute their fare share

(individual accountability).

Other patterns included in Appendix A that are formulated following this approach are:

PYRAMID (Pattern 1.2) or SNOWBALL (Davis, 2002; Gibbs, 1995), THINK-PAIR-SHARE or TPS

(Pattern 1.3) (NISE, 1997; Millis & Cottell, 1998), BRAINSTORMING (Pattern 1.4) (NISE, 1997;

Millis & Cottell, 1998), SIMULATION (Pattern 1.5) (Paulsen, 1995; Fablusi, 2000) or THINKING


45

ALOUD PAIR PROBLEM SOLVING (Pattern 1.6) (NISE, 1997; Millis & Cottell, 1998; Slavin,

1995).

We argue that this mining method is not purely top-down nor bottom-up. It is however an

intermediate approach in which the patterns are discovered after numerous implementations of

learning scenarios in different settings, and whose benefits are reported in the literature. A similar

approach is exposed in (Retalis et al., 2006), where they follow a reverse-engineering process for

identifying embedded good design practices in e-learning systems and, at the same time, analyze the

way users employ those systems in authentic scenarios. The resulting patterns for CSCL systems

can be found in (TELL, 2005a; Avgeriou et al., 2003). One of them, MANAGEMENT OF ON-LINE

QUESTIONNAIRES (cf. Pattern 3.2 of Appendix A), has been adopted for our CSCL scripting

pattern language.

3.2.2 Using case studies as a starting point

Other patterns presented in Appendix A are distilled using case studies related to scripted CL

situations as a starting point. Some of them are a result of the TELL project (TELL, 2005b; TELL,

2005a) and, therefore, the process we follow is based on the procedure employed in the project.

A case study serves as a starting point for the initial selection of titles and topics for potential

patterns. Additionally, this first case study analysis and our previous experience capturing patterns

in the literature lead us to reflect in the different types of patterns and relationships between them.

The case study we use in this sense analyzes an experience that takes place within a course on “the

use of ICT resources in education” (NNTT, its acronym in Spanish) at the Faculty of Education,

University of Valladolid, Spain (Ruiz-Requies et al., 2006). However, the design of this experience

also benefits from a previous case study also carried out by the members of the GSIC/EMIC group

(GSIC/EMIC, 1994). This earlier case study concerns a course on “Computer Architecture” (CA) at

the School of Telecommunications Engineering, University of Valladolid, Spain (Martínez-Monés

et al., 2005). It is noteworthy that though the content and discipline of the case studies are different,

NNTT reuses several design issues applied and deeply evaluated in CA.

During the enactment of the NNTT case study, we iteratively write and evaluate the patterns.

Several meetings involving the different stakeholders related to the case study are carried out to

discuss the content of the patterns. Therefore, the teachers involved in the case study and the

researchers monitoring its results largely influence the formulation of the patterns. Literature on the

topic of the problems tackled by the patterns is also consulted with the aim of complementing the

analysis of their forces and the associated solutions. In this phase, we continue looking for links

between the patterns so that together form meaningful designs. This process also enables to detect

connections between patterns written and not written yet (Goodyear et al., 2004).


46

Thus, the identification and construction of the patterns is basically accomplished by: a careful

questioning of an expert designer about why he designs in a certain way and by distilling them from

our observations about specific instances of good design (E-LEN, 2004a).

Furthermore, a workshop for reviewing the patterns is carried out as part of a meeting of the

TELL project. Feedback from the patterns is received in this meeting and in following

asynchronous messages. Each pattern receives two reviews from two different participants in the

project.

Using the NNTT case study as a starting point, the following patterns included in Appendix A

are proposed: ENRICHING THE LEARNING PROCESS (Pattern 1.7), INTRODUCTORY

ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), PREPARING FRUITFUL

DISCUSSIONS USING SURVEYS (Pattern 2.3), ENRICHING DISCUSSIONS BY

GENERATING COGNITIVE CONFLICTS (Pattern 2.4), GUIDING QUESTIONS (Pattern 3.3)

and FACILITATOR (Pattern 4.1). As mentioned in previous subsection, JIGSAW (Pattern 1.1) and

PYRAMID (Pattern 1.2) are formulated using the information provided by the literature, but they

are also applied in this experience. Since NNTT benefits from the practice of the CA case study,

CA also exhibits some of these patterns. Section 3.4 details how these patterns are applied in the

pedagogical designs of both cases studies.

Other patterns considered in the CSCL scripting pattern language are proposed by other authors

within the TELL project and, thus, also formulated using this procedure. They are THE

ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) and STRUCTURED

SPACE FOR GROUP TASKS (Pattern 3.1).

As aforementioned, all these patterns can be related forming a PL that outlines many different

possibilities of designing a script. The methodology presented in this section lead us to identify the

different types of patterns and relationships between them that can appear in potential CSCL

scripting PLs. Next section introduces a conceptual model for PLs covering the CSCL scripting

design space, and discusses how the considered type of patterns and relationships actually appear in

the illustrating PL.

3.3 CSCL scripting pattern languages model

It is clear that CSCL scripting is not an isolated design space. In contrast, it is highly

interrelated with other spaces involved in educational design. Figure 3.1 shows how the package

concerning the patterns for designing CSCL scripts is related with other types of patterns considered

in at least other two packages. Higher level patterns of CSCL scripting patterns are those related to

high level pedagogy (e.g. problem based learning, inquiry learning) (Goodyear, 2005). In this case,


47

the pedagogical approach directly related to the CSCL scripting patterns is described in the

SCRIPTED COLLABORATION pattern (pattern 11 from (E-LEN, 2005)).

Figure 3.1 CSCL scripting pattern language conceptual model package and relationships with other models

Moreover, patterns devoted to diverse didactics for specific subject matters are also relevant

when designing a script for a specific discipline. The learning problems, the typical misconceptions

and difficulties that the students face in particular subjects are well documented. While the

strategies related to eliciting desired social interactions are common among the scripts, the insight

and experience of, for example, mathematics education (Learning Patterns, 2005) or programming

languages education (Holland, Griffiths, & Woodman, 1997; PPP, 2005) are also helpful in the

design of scripts for such domains.

Therefore, CSCL scripting PLs are embraced by larger PLs regarding pedagogical approaches

and are complemented with other PLs capturing educational experience of specific knowledge

domains. This is similarly approached, for example in Architecture patterns (Alexander et al.,

1977), where a PL for building a porch completes a larger PL for designing a house; and in

Organizational patterns for agile software development (Coplien & Harrison, 2005), which

proposes that a “Project Management PL” complements a “People and Code PL”.

Having explained that, next subsection focuses on the CSCL scripting PLs package and details

the conceptual model for these PLs.

3.3.1 Aggregation model and types of connecting rules

The conceptual model for CSCL scripting PLs proposed in this dissertation comprises an

aggregation model expressed as a UML class diagram and the types of connecting rules that relate

the patterns situated at the same and different levels of aggregation.


48

Figure 3.2 represents the aggregation model of CSCL scripting PLs. Conforming to UML; the

diagram represents aggregation (and composition) relationships and specializations of abstract

classes. The aggregation model fits in well with the target design space, since the good practices

applied when creating CSCL scripts can be grouped at different granularity levels: set of activities

that are organized in CL flows vs. single activities vs. the resources (materials and tools) that

supports the single activities. Patterns at the different levels are complementary and need each other

for completeness, so they can be aggregated or related forming a hierarchical structure (which

represents the structure of pattern languages for CSCL scripts). Though next subsection shows a

graphical hierarchical structure illustrating how the PL included in Appendix A fits in with the

conceptual model, we use some patterns in the explanation of the different aggregation levels.

Figure 3.2 Aggregation model of CSCL scripting Pattern Languages

Some authors already distinguish between macro scripts and micro scripts (Fischer, Kollar,

Mandl, & Haake, 2007). As deeply exposed in Chapter Two, coarse-grained (or macro) scripts

describe general flows of collaborative (or not) learning activities (e.g. those following the Jigsaw

strategy). Fine-grained (or micro) scripts give detailed support within specific activities (e.g. scripts

for argumentative knowledge construction (Weinberger, Fischer, & Stegmann, 2005)). In the same

sense, the highest (coarser) aggregation or granularity level for CSCL scripting patterns is related to

the CL flow: the sequence of activities that make up a learning process. Some examples of patterns

at this level are JIGSAW and PYRAMID patterns (cf. Pattern 1.1 and Pattern 1.2 of Appendix A),

whose solutions form a generalization of actual learning flows. Other patterns directly related with

the learning flow but not necessarily proposing a flow structure are also situated at this level.

Another granularity level refers to the activities themselves. An example of a pattern at this level is

DISCUSSION GROUP (Pattern 2.2). In addition, we propose a third (lowest) granularity level that


49

includes the resources (materials and tools) needed to support the activities. Some examples are

the patterns proposed in (Georgiakakis & Retalis, 2006) such as MANAGING ON-LINE

QUESTIONNAIRES (Pattern 3.2). These tree types of granularity correspond to three aggregation

levels shown in Figure 3.2.

Besides, there are some aspects (such as roles or common collaborative mechanisms, namely

group formation, floor control, awareness) that can be connected directly to some of the patterns at

any of the aforementioned granularity levels. For example, roles can be defined globally at the level

of the whole learning flow, within activities or/and within collaborative tools (e.g. usage of the

FACILITATOR pattern, Pattern 4.1). Thus, the patterns that state a principle about these aspects are

usually integral parts of learning flows, activities or resources. That is the reason why this level has

a composite relationship with the others, which, in contrast, can exist independently.

We have already insisted on the idea that the connections rules between patterns are as much

part of a PL as the patterns themselves. Each pattern provides a possible context for any patterns

that appear at the same level or below it. In other words, a pattern serves to embellish higher-level

patterns, to work alongside patterns at the same level, and to provide a context for lower-level

patterns. This organization provides a powerful way of expressing educational design knowledge

and formulating comprehensible guidance. Particularly, the types of connecting rules between

CSCL scripting patterns at the same and across levels of aggregation are presented in Table 3.1.

Table 3.1 Types of connecting rules (relationships) between patterns at the same and different levels of aggregation

Types of connecting rules between patterns at different levels

- Complete (embellish) - (Complement)

Types of connecting rules between patterns at the same level

- Complete (embellish) - Complement - Is alternative to - Specialize

Four different types of connecting rules are identified:

- Complete (embellish): following the idea of composition indicated by the aggregation model

and other pattern-based approaches (Alexander et al., 1977; Derntl et al., 2006), patterns at

higher levels need patterns at lower levels for completeness. Resources complete activities

that, in turn, complete CL flows. Roles and common collaborative mechanisms complete

resources, activities or CL flows. Patterns at the same level can also complete each other.

This is clearly manifested in patterns at the CL flow, which can be combined in such a way

that a phase of the learning flow is refined by replacing the phase with another pattern at this

level. In all these cases the pattern that is completed with other patterns determines the range

of the whole.

- Complement: some patterns at the same level can complement each other. A pattern that

complements another pattern does not refine or modify the principles of the second pattern


50

(as it happens with the “complete” connecting rule). On the contrary, these patterns form

two parts of a larger whole, not previously embraced by any of the patterns. As an exception,

patterns at the activity level may complement those at the learning flow level by adding

(instead of refining) new phases to the flow.

- Alternative to: some patterns can tackle the same category of problem within a context

whose special details derive in alternative solutions. Alternative patterns can be interchanged

but they cannot be used in a complementary way probably because of contradictoriness or

redundancy (Harrer, 2006).

- Specialize: patterns at the same aggregation level can formulate principles that range from

general design ideas to their reusable specialization (David et al., 2003). Patterns that

formulate general design ideas have more “degrees of freedom” than patterns that formulate

their reusable specializations. “Degrees of freedom” in this context can be understood as the

number of design options that the solution of the pattern does not suggest and are free to

creatively vary.

These dependencies between the patterns provide guidance for their application and prevent

bringing together scripting strategies that do not make sense from the pedagogical perspective or

even inhibit the effects of each other (Harrer, 2006). Next subsection uses a hierarchical structure

(representing the aggregation levels) to illustrate how the PL included in Appendix A fits in with

the conceptual model.

3.3.2 An illustrating CSCL scripting pattern language: hierarchical structure

The patterns listed in Appendix A are organized according to the proposed conceptual model

forming a hierarchical structure (cf. Figure 3.3). The structure is represented as a graph (Salingaros,

2000): the patterns are identified with nodes, which are related by edges (the relationships between

the patterns). The graph shows that there are many paths through the patterns that guide their

application. Because of representational limitations not all the possible connections between

patterns are drawn in the figure. However, the complete relationships presented by the proposed PL

are documented in the pattern format fields devoted to the context and the paragraph relating the

pattern to smaller patterns that each pattern includes.

To illustrate and express the identified types of semantic relationships, we analyze in detail the

connections between some patterns included in the CSCL scripting PL next.


51

Figure 3.3 Hierarchical structure of the CSCL scripting PL showing how the PL fits in with the conceptual model. The numbers on each node reference the patterns as listed in Appendix A. Because of representational

limitations not all the possible relationships are drawn

There are two clear manifestations of the “is alternative to” and “specialize” relationships.

FREE GROUP FORMATION (Pattern 4.2) is alternative to CONTROLLED GROUP

FORMATION (Pattern 4.3). Both are CL mechanisms devoted to group formation. Assembling

groups is needed to complete the suggestions of patterns at the CL flow, activity level and also at

the resource level (cf. STRUCTURED SPACE FOR GROUP TASKS, Pattern 3.1). However the

specific characteristics of the problem that arise from (slightly) different contexts (e.g. a large

demanding assignment vs. an assignment that benefits from diverse or conflict knowledge) conduct

to divergent solutions that are mutually exclusive.

On the other hand, PREPARING FRUITFUL DISCUSSIONS USING SURVEYS (Pattern 2.3)

and ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4)

are specializations of DISCUSSION GROUP (Pattern 2.2). That is, they adapt DISCUSSION

GROUP to a more specific context and thus related problem, undergoing specialization.

DISCUSSION GROUP is within a context that requires organizations forms for knowledge sharing,

questioning and critique. Pattern 2.3 and Pattern 2.4 add to this context the issue that the

participants may consider their previous results or ideas before sharing the knowledge. In this sense,


52

the solutions of the specialized patterns are more concrete in their suggestions, offering less design

options to the users so that the essence of the pattern is not killed.

PREPARING FRUITFUL DISCUSSIONS USING SURVEYS (Pattern 2.3) and ENRICHING

DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4) form also a very

illustrative example of how two patterns at the same level complement each other shaping whole

not previously embraced by any of them. While Pattern 2.3 proposes preparing a survey that the

students answer before the discussion to organize their ideas, Pattern 2.4 suggests providing

students with the answers of their classmates so that they reflect on potentially different approaches

and thus generate new questions and issues to be discussed. The answers to the surveys organized

following the purpose according to Pattern 2.3 can be complementarily used as indicated by Pattern

2.4 for a different purpose. Together, Pattern 2.3 and Pattern 2.4 represent a debate strategy that

spans a two different phases, whose range is not considered separately by each of them.

There are many other examples manifesting the “complement” relationship in the PL. For

instance, the GUIDING QUESTIONS (Pattern 3.3) can be presented to the students according to

MANAGEMENT OF ON-LINE QUESTIONNAIRES (Pattern 3.2). A FACILITATOR (Pattern

4.1) complements the pattern CONTROLLED GROUP FORMATION (Pattern 4.3) since this may

be the role in charge of assembling the groups. Moreover, the learning flow of a whole educational

unit might comprise a pattern at the CL flow level, e.g. JIGSAW (Pattern 1.1), complemented with

another flow, e.g. PYRAMID (Pattern 1.2), that follows or precedes the flow suggested by the first

pattern (Pattern 1.1). Again, the limits of the whole resulting from this concatenation of patterns is

determined by the two patterns (one of them indicates the start of the learning flow and the other the

end).

The “complete” connecting rule also appears when combining two patterns at the CL flow. In

this case, a phase suggested by a pattern is organized according to another pattern (which can be

eventually the same. For example the “Expert group” phase of the JIGSAW (Pattern 1.1) may be

structured following the PYRAMID (Pattern 1.2). The base learning flow, indicating the start and

the end of the learning flow is not modified. In contrast, the proposal of the JIGSAW is refined,

being in the resulting design the PYRAMID an integral part of the adapted JIGSAW.

This type of relationship clearly intervenes between patterns at the different levels. The

knowledge of THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) may be

considered when completing the design of any of the activity types orchestrated in JIGSAW

(Pattern 1.1). Similarly, STRUCTURED SPACE FOR GROUP TASK (Pattern 3.1) refines Pattern

2.5 by indicating the type of resources that may be useful for supporting the activity. A

FACILITATOR (Pattern 4.1) may also complete the activity design ideas indicated by THE


53

ASSESSMENT TASK AS A VEHICLE FOR LEARNING or the behaviour of certain

collaboration tools according to STRUCTURED SPACE FOR GROUP TASK (Pattern 3.1).

The hierarchical structure depicted in Figure 3.3, which emphasizes the rules that connect the

patterns, provides guidance regarding the order in which the patterns are to be applied. We argue

that all the CSCL scripting PLs that conform to the conceptual model proposed in subsection 3.3.1

follow similar hierarchical structures. This fact allows the definition of general guidelines for

applying those PLs.

3.3.3 General guidelines for applying the PL described with the conceptual model

How can the patterns of a CSCL scripting PL described with the conceptual model be applied?

The main ideas, shared among diverse authors of other pattern-based proposals for different design

spaces (Alexander et al., 1977; Coplien & Harrison, 2005; Lukosch et al., 2006; Schümmer &

Lukosch, in press), is to start at the top (higher granularity level) and work towards the bottom.

When a pattern points to several subtending patterns, they can be also applied or not in any order

but considering the semantic differences indicated by the four types of relationships. Of course, the

decision of applying or not applying a pattern depends on context. A PL can be considered as a map

collecting numerous meaningful paths, but the exact chosen path is subject to the circumstances.

The patterns should be helpful and feasible in a particular situation in order to be selected. The

point is not to use as many patterns as possible, but to choose the patterns that solve the problems

that actually appear in an educational situation. If the context, problem and forces of a pattern match

an actual situation, then the pattern is worth considering.

The progression through the PL requires knowing the patterns beforehand or reflecting upon the

different design possibilities before their selection. In fact, this analysis is enabled by having the PL

at hand, which also provides a basis for discussion. The result is a set of interrelated patterns that

together generate a sequence (a story) that shapes the design of a specific script.

Evidently, each PL has a limited number of patterns. A PL represents the experience of

someone (or a community of practice) when creating CSCL scripts (Alexander et al., 1977).

However, the educational context of an external person using the PL may have specific needs that

request new patterns or for different versions of the patterns. In addition, a user of a certain PL (for

example the PL included in Appendix A) might need to apply a pattern in a way that is not

considered in the map of established relationships. The special characteristics of the educational

domain and the unpredictable characteristics of CL situations impose flexibility requirements when

applying PLs. In general, teachers should be able to consider a new pattern or try a pattern that is

out of sequence, if their intuition leads in that direction. The constant evaluation of the effects of the

resulting script enables the evolution of the PLs.


54

All in all, if we provide a teacher with a CSCL scripting PL, the desire is that the selected

sequence of patterns keeps the essence of the constraints that are intrinsic to the pedagogical

principle of each pattern in the sequence. The extrinsic constraints, in contrast to the intrinsic ones,

necessarily derive from the application of the patterns to particular situations. These extrinsic

constraints represent the (arbitrary) design decisions reflected in a script that are suitable for being

modified by the teacher and other involved stakeholders, such as students (Dillenbourg et al., 2007).

This idea is in line with the idea of preserving structure discussed around Architecture patterns

(Alexander, 2003b). Alexander proposes a process that involves step-by-step applications of

patterns in such a way that the whole increases by structure-preserving transformations. These

transformations gradually add symmetries (described as centers) that enables the unfolding of the

whole.

Some ideas of Alexander (2003b), also considered by Coplien & Harrison (2005), can be

adopted as general iterative guidelines when applying the CSCL scripting PLs.

- Consider your educational situation as a whole, get a feeling for how the course is working,

and try to identify its “weak points”. Maybe you have recently applied another pattern,

which left you in a new context or produced explicit forces which are not resolved yet.

- Focus on what can be done to enhance the script, considering the target objectives. Are you

striving for a specific skill development? Or are you seeking for motivating your students?

Read the patterns so that you can find help to resolve these questions.

- Find a place where the application of a new pattern – the consideration of a new role, the

addition of a new structured activity, changing the learning flow, - will achieve your goal.

Will any of the patterns help you? Do you know of other strategies that will help? Apply the

patterns (or other strategies) locally.

- Look at the structure of the PL. Each pattern indicates which patterns may come next as

complements or refinements.

- Reflect on the application of the patterns. Do they work? Evaluate the effects of the pattern-

based script so that the conclusions provide you feedback for further designs.

This subsection argues that the teacher is the main stakeholder in charge of applying CSCL

scripting PL. Next subsection discusses this issue elaborating on other possible categorizations of

CSCL scripting patterns.

3.3.4 Discussion: other categorizations of patterns

Our conceptual model bears similarities to the conceptualization of the educational design space

proposed by Goodyear et al. (2004). Patterns for designing good learning tasks are in our proposal

patterns at the activity granularity level. Moreover, the supportive space embracing the resources

(tools and artefacts) corresponds to the resource level. Organizational forms that favour the


55

emergence of convivial learning situations are what actually propose the patterns included in the

collaborative learning flow level and vertical level (roles, group formation). We argue that the

learning flow and the other CL mechanisms (common to the flow but also to the activities and

supporting resources) are relevant entities that deserve distinction. Furthermore, the aggregation

structure that provides the relationships “resources complete activities, which complete flows” and

“roles and other common CL mechanisms complete resources, activities and flows” makes our

proposed conceptual model more comprehensive. Previous subsections and the application of the

PL presented in section 3.4 provide foundation in this sense.

Other categorizations of patterns could be also distinguished. For example, Alexander et al.

(1977) differentiate the terms design patterns and construction patterns. Alexander’s design

patterns refer to understanding the geometry of a building and the relationships between parts, while

construction patterns examine the materials and processes needed in order to put the designs into

practice. We distinguish between design CSCL scripting patterns, which are used to devise the

educational design of a script (patterns at the CL flow and the activities levels); and construction

CSCL scripting patterns, which support the implementation of the designs in actual practice

(patterns at the resource and vertical level). The audience of design CSCL scripting patterns is

mainly teachers and learning designers, who create CSCL scripts. Nevertheless, these patterns may

be used by systems designers in requirements analysis tasks. In contrast, construction CSCL

scripting patterns are more intended for system developers (or content providers), although they

should be also considered by teachers and even students, who are the actual users of the scripts.

On the other hand, Vesseur, Lutgens, Broek, Koehorst, & Ronteltap (2005) differentiate five

phases (or levels) in developing and planning education in which design patterns can be useful:

preparation of a (case of study) pilot, preparation of the course, the course itself, assessment of the

course and evaluation of the pilot. The phase on which CSCL scripting patterns (from the

perspective approached in this dissertation) are focused is more related to the preparation of the

learning design than to putting the learning design into effect. Thus, CSCL scripting patterns are

mainly at the “preparation of the course” level. The roles that may use the patterns considered in the

other phases of planning education include the researcher and the evaluator.

This discussion leads us to the ideas developed at the beginning of the section. CSCL scripting

PLs are not isolated. Quite the opposite, they are connected to other PLs. These PLs can be situated

at the same design phase (e.g. PLs dealing with didactics of specific subject matters) or at a

different one (e.g. PLs for planning the evaluation of a course). It is also remarkable the fact that not

all the patterns that may be considered when designing scripts might have a pure scripting purpose

(cf. for example FACILITATOR, Pattern 4.1 of Appendix A). In fact, patterns that may belong to a

CSCL scripting PL can also fit in (with different connecting rules to other patterns) with other (non-

scripting) PLs. If all the possible PLs for educational design are put together, overlapping between


56

patterns and PLs will become apparent. An example of this appreciation is that isolated patterns (not

included in our proposed PL) and other PLs fit in with the CSCL scripting PLs conceptual model

and, thus, could be adopted to enlarge the PL of Appendix A. This discussion also supports the

validity of the conceptual model.

3.3.5 Discussion: other sample patterns that fit in with the proposed conceptual model

Interestingly, there are PL proposals that can be embraced by CSCL scripting PLs. As

aforementioned, this fact is manifested in other domains, such as Architecture, where a PL for

designing a house comprises a PL for a porch (Alexander et al., 1977).

It is noteworthy that DISCUSSION GROUP (Pattern 2.2 of Appendix A) is, in turn, part of a

PL devoted to “Debate” (Goodyear, 2005). This PL can be considered a smaller PL embraced by a

CSCL scripting PL. The patterns of the Debate PL can be situated in the activity and resources

levels as well as in the roles and common collaborative mechanisms level. Most of its patterns refer

to fine-grained activities involved in a debate such as PROPOSE THE MOTION or VOTE. Other

patterns are around roles, namely SPEAKER FROM THE FLOOR or CHAIRPERSON. Finally, the

“Debate PL” also considers principles about resources related to tools or materials, e.g. BULLETIN

BOARD and SOURCES OF EVIDENCE.

Similarly, MANAGEMENT OF ON-LINE QUESTIONNAIRES belongs to a PL for LMSs

(Avgeriou et al., 2003). The rest of the patterns of this PL and related PLs, e.g. a PL for

asynchronous CL systems proposed in (Georgiakakis et al., 2006), can be also situated at the

resource level of our conceptual model.

One of the patterns collected in (DiGiano et al., 2003) is a version of the JIGSAW pattern. The

other patterns included in this work can be framed in the CSCL scripting PLs conceptual model as

well. To name a few examples, PIPELINE WORKFLOW as a pattern at the CL flow level,

TOUCHING THE ELEPHANT is at the activity level, EXCHANGE TEMPLATE is at the resource

level, and CONTRIBUTION LAYERING is at the common mechanisms level.

The following examples of actual scripts, generated using the PL of Appendix A, aim at further

expressing the different aggregation levels and relationships that connect the patterns. They also

illustrate how the hierarchical structure conceptually guides the process that can be followed when

applying the patterns. The reader should also note that the patterns selected for each example form

themselves a small PL.


57

3.4 Examples of applying the CSCL scripting patterns

Some of the patterns in Appendix A are drawn from real experiences at our University. This

section looks at three examples, based on real case studies, which consider these patterns (and

others captured in the literature) in order to illustrate how the patterns can be applied.

The experiences are described as stories in the form of sequences that show the process of

growth. The sequences offer different paths (out of the many possibilities for the PL) that result in

particular wholes (scripts). These examples should provide a better feel for the PL and the patterns’

interconnections.

It is important to remember that these sequences are real. They come from our experience, and

they typified the rich ways in which patterns build on each other, as well as the way in which the

language can become alive. In this sense, the PL of Appendix A describes somehow the culture of

our community of practice.

As advanced in the section 3.2, the first and the second experiences are CA (Martínez-Monés et

al., 2005; Jorrín-Abellán et al., 2006) and NNTT (Ruiz-Requies et al., 2006), both used as a starting

point to propose some of the patterns. The results of these experiences are successful in terms of the

target educational objectives. The third experience is called Computer Networks Protocols (TTG, its

acronym in Spanish), designed explicitly applying the CSCL scripting patterns from scratch. This

section is therefore longer, since it also includes the results of the experience evaluation that shows

the fruitful results of following the good practices formulated by the patterns.

3.4.1 Computer Architecture (CA) course

The CA course is part of the core body of knowledge in the Telecommunications Engineering

curriculum in Spanish universities. The specific course in which the following script is applied is at

the University of Valladolid, Spain. It is placed in the fall semester of the fourth year (out of five)

and is made up of 30 lecture hours and 60 laboratory hours. Within the curriculum, the course is the

last of a branch on computing topics that covers programming fundamentals, operating systems, and

computer architecture. The script designed for the course aims at providing contextualized,

integrated and meaningful knowledge; to promote active, intentional and collaborative learning

(Martínez-Monés et al., 2005; Jorrín-Abellán, 2006).

The whole course is defined as a project that develops along the semester, whose objective is

the design and evaluation of computer systems. The class of 100–120 students is divided into three

sessions of 40 students, in which the elementary unit consists of groups of two students (pair). To

enable distinct perspectives of the main subjects within the classroom, five clients are defined,

giving answer to different market sectors and system requirements. Each pair is assigned one out of


58

the five clients for the whole course. This way, in each laboratory group, different clients are being

studied throughout the course, following the principles of JIGSAW (Pattern 1.1).

The Jigsaw-based structure is completed with the suggestions of SIMULATION (Pattern 1.5).

The teacher takes the roles of the five clients and the director of the manufacturer companies, while

students assume the roles of a consulting firm and a computer manufacturer. Along the script there

are several project assessment tasks, as indicated by THE ASSESSMENT TASK AS A VEHICLE

FOR LEARNING (Pattern 2.5), which correspond to three subprojects of 4 weeks each. The final

report is to be submitted a month after the course has finished, and therefore, during this period,

there are no lectures where students could meet face to face. In this way, the teacher, which

becomes a FACILITATOR (Pattern 4.1) during the whole course, makes sure that they have time to

execute and plan how they will carry out the subprojects.

The educational design aims at promoting interaction within and between the pairs assigned to

different clients. Each subproject studies different specific issues of the whole problem and presents

two milestones. In the first one, basic decisions are taken, and in the second milestone, each pair has

to submit a technical report to the client (teacher). In each milestone, every laboratory group

(“Jigsaw group” phase of the JIGSAW) holds a debate. The debate is designed as a collaborative

review of the work of the students, arranged as suggested by PREPARING FRUITFUL

DISCUSSIONS USING SURVEYS (Pattern 2.3), and where the problems of the different clients

can be shared and discussed at a laboratory group level following the ideas of ENRICHING

DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4). At the end of the

whole script, a technical report (corresponding to the last subproject) is collaboratively produced

among all pairs that deal with the same case study in each laboratory group (forming accordingly a

PYRAMID Pattern 1.2).

Since the subprojects are demanding assignments that take place along the whole course, FREE

GROUP FORMATION (Pattern 4.2) is proposed to the students when assembling the pairs.

Moreover, the structure of the technical reports is provided to the students as GUIDING

QUESTIONS (Pattern 3.3) that help them to focus on important issues to be covered.

To support preparation and realisation of the debates a telematic tool for automatic

MANAGEMENT OF ON-LINE QUESTIONNAIRES (Pattern 3.2) called Quest (Gómez, Rubia,

Dimitriadis, & Martínez, 2002) is used. As required by the joint application of Pattern 2.3 and

Pattern 2.4, Quest supports synchronous debates in the classroom based on the results of previously

submitted questionnaires that are filled by the students with their opinions about the topics under

discussion.

To offer a STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1) where the students can

share their reports, the teacher encourages the use of the Basic Support for Cooperative Work


59

(BSCW) (OrbiTeam, 2007). BSCW is a well-known shared workspace system based on web

interface, was used for asynchronous document sharing and threaded discussions. Also it offers

additional features for handling shared documents such as a commenting tool (one student may

“post” comments on another’s document), and an awareness tool (it informs students by e-mail

about the uploading of new documents, their modification, etc.), among others.

The patterns applied in the design of this script are used in order to promote the acquisition of

competencies and skills (e.g. those related to group work), besides specific contents of the course.

Next example deals with a course on a totally different topic (even discipline) which also adopts the

design principles captured in the PL.

3.4.2 The use of ICT resources in Education (NNTT) course

The global objective of the NNTT course is to allow students of the Faculty of Education at the

University of Valladolid to create didactic units in collaboration (Jorrín-Abellán, 2006; Ruiz-

Requies et al., 2006). It is made up of 40 hours including lectures and laboratory sessions. Each

class comprises from 36 to 42 students. Along the course the students should create a didactic unit

that could be eventually used in a real school and design the ICT resources that will support such a

didactic unit. An actual school also participates in the case study by providing specifications and

evaluation of the partial results of the students. As the CA course, the educational situation is

blended and interleaves normal face-to-face activities with technology-supported collocated or

distance activities. Besides knowing and applying the specific contents of the course, the script

employed in this course targets students develop their critic capabilities and skills related to

working in group and coping with strong workloads.

Before actually creating didactic units, the students are encouraged to work around the three

main topics of the course. This phase, which spans from second to fifth weeks of the course, is

scripted according to a two-level PYRAMID (Pattern 1.2). The previous (first) week is devoted to

present the course and the planned script following the indications of INTRODUCTORY

ACTIVITY: EXPLAINING THE LEARNING DESIGN (Pattern 2.1). The first level of the

Pyramid is, in turn, structured in accordance with JIGSAW (Pattern 1.1). The CL flow resulting

from the combination of these two patterns at the learning flow level is refined as follows. Taking

into account the FREE GROUP FORMATION pattern (Pattern 4.2), the students are assembled in

pairs that persist during the whole course. In the “Individual phase” of the JIGSAW every pair

studies one of the three topics in such a way that each topic is assigned to six or seven pairs. The

teacher facilitates a set of articles associated to the different topics and asks the students to discuss

in pairs the ideas of the articles (related to their assigned topic). Then, students should elaborate a

report for assessment purposes but also as a learning task (ASSESSMENT TASK AS A VEHICLE

FOR LEARNING, Pattern 2.5) which pushes them to reflect on a series of questions that they


60

should answer in the report. These questions are explicitly provided by the teacher, as suggested by

GUIDING QUESTIONS (Pattern 3.3).

In the “Expert Group” phase of the JIGSAW the six (or seven) pairs that have worked on the

same topic join in a single group to read and discuss the reports written by their partners. The

teacher gives them some ideas about how they should organize the DISCUSSION GROUP (Pattern

2.2) and acts as a FACILITATOR (Pattern 4.1) helping students get started and giving feedback

when necessary. To facilitate sharing of reports a STRUCTURED SPACE FOR GROUP TASKS

(Pattern 3.1) is used. In this case, the employed tool is Synergeia (ITCOLE, 2005), a version of

BSCW focused on learning that provides a shared, structured, web-based work space in which

collaborative learning can take place, documents and ideas can be shared, discussions can be stored

and knowledge artefacts can be developed and presented. Again, in a following activity the group

should jointly write a report in an activity designed according to Pattern 2.5 and Pattern 3.3.

In the “Jigsaw Group” phase (of Pattern 1.1), new groups are formed. Every group comprises a

pair “expert” on each topic. This assembling of pairs into larger groups is accomplished by the

teacher using the criteria of physical proximity in the lab. In this case, the context and forces of the

situation lead to the selection of CONTROLLED GROUP FORMATION (Pattern 4.3) instead of

FREE GROUP FORMATION. In this phase, the students read and present the (second) report

(outcome of the “expert group”) and elaborate a new common report integrating the three different

topics. After that they complete a questionnaire designed with the purpose of PREPARING

FRUITFUL DISCUSSIONS USING SURVEYS (Pattern 2.3).

The second (and last) level of the PYRAMID is devoted to ENRICHING DISCUSSIONS BY

GENERATING COGNITIVE CONFLICTS, Pattern 2.4) in a global debate where all the students

participate. The MANAGEMENT OF ON-LINE QUESTIONNAIRES (Pattern 3.2) is carried out

with Quest.

Throughout the whole script, several concurrent activities need to be synchronized among

different groups. Therefore, the teacher plans some enriching additional activities (e.g. training in

how to design a questionnaire, type of enrichment II) for groups that finish earlier, as indicated by

ENRICHING THE LEARNING PROCESS (Pattern 1.7).

As in the CA course, the script used in NNTT exists before the PL of Appendix A. However,

the presentation of the design of both courses as sequences of patterns illustrates the different types

of CSCL scripting patterns and connections between them. Next subsection introduces a new script

for a course on computer networks. In this case the script is designed explicitly applying the CSCL

scripting patterns from scratch.


61

3.4.3 Computer Networks Protocols (TTG) course

Several approaches for teaching computer network protocols concepts and mechanisms by

means of lab exercises and projects have been reported in the literature. Significant examples

include the use of networking tools (traffic monitors, traffic analyzers, etc.) in real scenarios

(Stevens, 1995), the use of network simulators (Al-Holou, Booth, & Yaprak, 2000), protocol

implementation and network programming project-based activities (El-Kharashi, Darling,

Marykuca, & Shoja, 2002), and even the performing of learning activities with no computer and

network support at all (Surma, 2003).

The script applied in this subsection represents an experience carried out in an undergraduate

course on computer network protocols that combines several of those approaches: networking tools

in real scenarios, network programming, and simulation. The course belongs to the studies of

Telecommunication Engineering at the University of Valladolid. More particularly, the script is

focused on the lab exercises related to the first of the mentioned approaches: the use of networking

tools in real scenarios of traffic interchange for the learning of, in this particular case, concepts and

mechanisms of the Transmission Control Protocol (TCP).

During previous years (before applying the script), those lab exercises simply comprised the

analysis of the traffic that communicating applications running in lab hosts interchanged among

each other (or with applications running in other Internet hosts) through the lab network. The set of

communicating applications included, in a similar way as proposed by (Stevens, 1995), common

client/server utilities (FTP, telnet, web...) as well as user-guided traffic sources and sinks based on

the sock application. The students were requested to set up the applications in a predefined way, to

capture the traffic with the tcpdump tool (Tcpdump, 2006), and to analyze captured traffic

identifying what TCP mechanisms previously explained in theoretical lecturers are involved

(connection establishment, flow control, slow start, congestion avoidance, Nagle algorithm, etc.)

Seeing the protocols “in action” enabled the students to contrast their knowledge of the protocol

mechanisms with their real behaviour thus detecting potential misunderstandings.

Nevertheless, this approach showed to have several weaknesses. Firstly, the large set of TCP

mechanisms under study demanded from the students the setting up of an also large set of traffic

interchange scenarios. Therefore the students’ effort was devoted, to an important extent, to the

performing of those repetitive configuration tasks. This fact precluded them from focusing on the

understanding of the protocols mechanisms themselves and the circumstances under they are

intended to be useful. Secondly, the time required for the analysis of a large set of scenarios

involving single mechanisms made difficult (within the time frame of the course under study) to

devote time for the setting up and analysis of scenarios involving several TCP mechanisms

simultaneously. This fact hindered the understanding of the mutual influence among TCP

mechanisms and, consequently, their differences and corresponding purposes.


62

With the goal of alleviating those problems, the two teachers of the course plan to encourage

students to design (not only to analyze) their own scenarios of traffic interchange that trigger one or

a combination of TCP mechanisms. This kind of scenario-design activities helped the students to

understand the circumstances under a TCP mechanism is intended to be useful. Moreover, the

teachers design a script that pushes students to collaborate in groups with other students so as to

complete the traffic analysis and scenarios design tasks. By means of collaborative activities,

students are expected to generate further knowledge as well as to share the work load, thus

providing new time slots for introducing new exercises.

In addition to the above advantages, basically related to the course contents and organization,

the script introduced in the course is expected to generate opportunities for the students’ acquisition

of competencies and skills such as argumentation capacity, responsibility for the own learning,

involvement in group work, etc.

Next subsection (3.4.3.1) firstly introduces the course that constitutes the context of the script.

Then subsections 3.4.3.2 and 3.4.3.3 detail its design, focusing on the type of scenarios and the

specific script proposed to the students. Afterwards, subsection 3.4.3.4 explains how the experience

has been evaluated and what conclusions can be drawn from the evaluation results.

3.4.3.1 Course background TTG is placed in the fall semester of the third year (out of five) of the studies for achieving the

degree of “Telecommunications Engineering” at the University of Valladolid, Spain. It is the third

out of five courses belonging to the core body of knowledge of the degree’s curriculum that are

devoted to the study of computer network protocols following a bottom-up layered approach. The

first of those courses introduces basic concepts of computer networks and focuses its attention on

media access control and logical link layer techniques and protocols. The second course is mainly

devoted to network layer aspects (routing, congestion control, etc.). Finally, the third course, the

focus of this study, deals with transport-level protocols and, specifically, with the TCP. The

remaining two courses cover application-level protocols.

The course spans a 15-week-long semester and involves 30 lecture hours (one two-hour session

per week) and 60 laboratory hours (two two-hour sessions per week). The 60 laboratory hours are

divided in three different types of exercises: 22 hours involve exercises of analysis and design of

scenarios of traffic interchange, 22 hours are devoted to network programming with the BSD

sockets API (Stevens, 1998), and the last 16 hours propose activities based on the use of the ns-2

(ISI, 2007) network simulator. The script is focused on the activities performed during the first 22

hours.


63

A total of 160 students approximately participate in the course. Due to physical space

restrictions at the lab, they are divided in four groups of 40 students. Consequently, each lab session

is repeated four times a week. The students are arranged in groups of two (a pair) that share the

same lab resources (machine, user account, etc.) and work together during the whole course.

3.4.3.2 Scenarios to be analyzed or designed throughout the script Scenario-based learning is defined as “...learning that occurs in a context, situation, or social

framework. It’s based on the concept of situated cognition, which is the idea that knowledge can’t

be known or fully understood independent of its context” (Kindley, 2002). Applying this same idea,

knowledge on computer network protocols cannot be fully understood if it is not situated in the

context in which those protocols are used: applications that communicate through a computer

network. Within this context, students are requested to perform two types of activities: analysis of

predefined scenarios of traffic interchange (provided by the teachers) and design of new scenarios

that illustrate a required set of protocol mechanisms. Analysis activities enable the students to

confront real protocol behaviour with the knowledge acquired in theoretical lecturers (thus detecting

potential misunderstandings). Design activities “force” the students to focus their attention in the

circumstances in which a particular mechanism is useful. This helps the students to differentiate the

protocol mechanisms and promotes a better understanding. Table 3.2 enumerates the TCP

mechanisms to study during the 22 first lab hours of the course as well as the type of scenarios that

the students are requested to analyze and to design. The enumerated TCP mechanisms are grouped

in “sets” (left-hand column) related to the script that will be described in the following section.

3.4.3.3 Designing the script Considering the educational situation as a whole and the weak points identified in experiences

of previous years, the teachers of the course select a sequence of patterns belonging to the PL of

Appendix A. The focus is on finding ways of resolving problems that appear in such a situation,

having in mind the target educational objectives.

Starting at the top level, the teachers select JIGSAW (Pattern 1.1) as the basis to structure the

CL flow of the script. They also decide to complement this pattern with PYRAMID (Pattern 1.2),

by refining the “Jigsaw Group phase” with a two-level Pyramid structure. The result is a CL flow

that combines JIGSAW and PYRAMID patterns (cf. Table 3.3). The formulation of both patterns in

Appendix A indicates the context in which they can be applied and what they suggest along with

the educational benefits that they foster. This information indicates that the selected patterns are

helpful and feasible for the conditions of the faced situation. For example, the adequateness of

JIGSAW is, among other issues, manifested by the fact that the list of TCP mechanisms (and thus

scenarios of traffic interchange) can be easily divided into sets.


64

Table 3.2 TCP mechanisms to be studied in the course and the related scenarios of traffic interchange to be analyzed and/or designed

Set TCP mechanism Scenarios TCP connection establishment and termination

Analysis: telnet session with a “local” machine and web session with a “remote” machine.

TCP Half-close Design: use of sock tool as traffic source and sink with options needed to “force” a half-close scenario.

Common

TCP Reset segments Analysis: telnet session with “unreachable” TCP port.

Retransmission time-out (error control)

Design: scenario of retransmission during connection establishment (e.g. disconnecting network cable). Design: scenario of ARP (Address Resolution Protocol) traffic capturing during TCP connection establishment (due to expiration of ARP cache) A

2MSL (Maximum Segment Lifetime) time-out (error control)

Analysis: scenarios proposed in (Stevens, 1995) for the measurement and characterization of the 2MSL time-out.

B

Nagle algorithm (reduction of overhead caused by small TCP segments)

Analysis: telnet session in which a sentence is typed (impossible to identify the mechanism due to human key pressing rhythm). Analysis: telnet session in which the same typed sentence as above is “copied and pasted” at once (the Nagle algorithm can now be identified). Design: use of sock tool so as to illustrate Nagle algorithm behavior (fast generation of small data chunks that will be grouped by the mechanism). Analysis: identification of the “delayed ack” mechanism and its positive/negative influence in the Nagle algorithm (ack delaying enables Nagle algorithm to generate larger segments thus reducing protocol overhead).

C Sliding Window (flow control)

Design: using the sock tool, a traffic source requests TCP to send a large chunk of data. A traffic sink “reads” data from TCP in smaller chunks. It is interesting to analyze the size of the advertised sliding windows of the TCP receiving entity when the buffer size is modified, when “read” operations are separated by a certain amount of time, when those operations start a certain amount of time after the connection is established, etc. (the flow control imposed by the sliding windows is observed as well as its relationship with the receiver’s buffer size). Design: the above scenario is requested to be analyzed again but running the traffic sink in a Sun Sparc workstation using Solaris operating system (in order to discover different TCP behavior in different implementations).

D Slow start and congestion window (congestion control)

Analysis: using the sock tool, a traffic source and a traffic sink interchange TCP data within the lab network (the slow start mechanism is difficult to appreciate due to low delays). Design: use of sock tool as traffic source in order to illustrate the slow start mechanism behavior (e.g. using a “remote” web server as traffic sink. The mechanism can be appreciated if delays are large enough). This scenario is useful to illustrate under what circumstances the congestion window decreases the traffic sending rate. Design: the above scenario is requested to be analyzed again but running the traffic source in a Sun Sparc workstation using Solaris operating system (in order to discover different TCP behavior in different implementations).

Congestion-window vs. sliding window

Design: identify which of the two mechanisms limit the data sending rate and under what conditions

Nagle algorithm vs. sliding window

Design: identify which of the two mechanisms force the traffic source to interrupt the sending of data using as the traffic sink a “far” (in terms of end-to-end delay) web server Combined

Retransmission time-out vs. ARP time-out

Design: scenario of retransmission during the data transfer phase of a TCP connection between two machines located in different networks (not in the lab network, in order to analyze the differences)


65

Table 3.3 Sequence of lab sessions and the corresponding individual and collaborative learning activities

Session Activity description Outcome Supporting tools and resources

1

2

3

4

Jigsaw phase: “individual work”

Explanation of the educational design

Each pair works on the common set of TCP mechanisms

None to deliver

None

5

6 Questionnaire on the common and assigned set of TCP mechanisms (at the end of session #7)

Quest tool

7

Each pair works on the assigned set of TCP mechanisms (A, B, C, or D)

Written report on the assigned set of TCP mechanisms (eight-page document to be finished a week before session #10)

BSCW tool to store, share, and comment the reports

8 Discussion on the test results moderated by the teacher

List of controversial points to be discussed with other “experts”

None

9 Jigsaw phase: “experts group”

Pairs that have worked on the same set of TCP mechanisms (“experts”) meet together and discuss on the test results and controversial aspects (generated during the discussion)

Document with a list of discussed topics and agreed conclusions (one-page document delivered at the end of the session)

BSCW tool to store, share, and comment the list of discussed topics and agreed conclusions

10 Jigsaw phase:

“jigsaw group” (I)

Each pair in a “super group” explains to the others the mechanisms of its assigned set.

Prerequisite: each pair has read the written report of the other members of the “supergroup”.

None to deliver

BSCW tool for accessing the reports of the rest of “super group” members.

11 Jigsaw phase:

“jigsaw group” (II)

Analysis of “combined scenarios” according to a “pyramid” structure:

Pyramid level 1

Each pair works individually

Pyramid level 2

The whole “super group” compares and discusses the obtained results and tries to obtain a common consensus

At the beginning of session #12 students answer a questionnaire individually. The questionnaire contains 4 questions on each set of mechanisms (A, B, C, and D) as well as 4 questions related to the relationships among different mechanisms

Quest tool


66

The students belonging to the same pair work together on the common set of TCP mechanisms

until the fourth session and employ three sessions more to study a different set of TCP mechanisms

(these tasks are all part of the “Individual phase” of the JIGSAW). For assessment purposes but also

with the aim of training students’ writing skills, the teachers ask the students to write a report on the

assigned set of TCP mechanisms (A, B, C, or D). Following THE ASSESSMENT TASK AS A

VEHICLE FOR LEARNING (Pattern 2.5) the teachers give students a lot of control over how they

carry out the elaboration of the report, making sure that they have sufficient time to plan and

execute it well. The reports must be shared at least among the pairs that will join in a “Jigsaw

group” in the tenth session. Therefore, a STRUCTURE SPACE FOR GROUP TASKS (Pattern 3.1)

that facilitates sharing these resources is provided to the students. The teacher selects BSCW as the

actual tool to support this functionality (OrbiTeam, 2007). Of special interest here is the fact that

BSCW logs every action performed on the shared workspace, providing data that were used as a

source of the analysis, as explained in the following subsection.

On the other hand, the teachers plan to organize a DISCUSSION GROUP (Pattern 2.2) about

their progress in the study of the TCP mechanisms before finishing the report. That discussion aims

at clarifying detected misunderstandings and to generate new questions on the TCP mechanisms

intended to motivate further work and collaboration. In this way, the teachers apply PREPARING

FRUITFUL DISCUSSIONS USING SURVEYS (Pattern 2.3) and, consequently, arrange an on-line

survey with questions related to the different TCP mechanisms, so that the students answer the

survey before the discussion and organise their ideas and arguments supporting them. The answers

of the survey are then used by the teachers with the aim of ENRICHING DISCUSSIONS BY

GENERATING COGNITIVE CONFLICTS (Pattern 2.4) in the actual discussion, which takes

place in session #8. The students have the opportunity to read the answers of their classmates and to

notice that their results may be wrong, thus generating new doubts that can be discussed. The

questions proposed in the survey are used by students and teachers as GUIDING QUESTIONS

(Pattern 3.3) of the important issues to be discussed. The MANAGEMENT OF ON-LINE

QUESTIONNAIRES (Pattern 3.2) is facilitated by Quest (Gómez et al., 2002).

Following the number of interconnections with other patterns indicated in JIGSAW and

PYRAMID, the teachers decide to complete the learning flow with the principles proposed by

patterns at other levels. In the first session the teachers, according to INTRODUCTORY

ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), plan to explain the whole learning

design so that the students are aware and understand the sequence of lab sessions and its goals.

Moreover, the teachers ask the students, as the outcome of the discussion, a list of controversial

points to be also used as GUIDING QUESTIONS in the “Expert group” phase of JIGSAW. In this

phase, the pairs that have worked on the same set of TCP mechanisms join in order to perform a

DISCUSSION GROUP about the controversial points related to their set of mechanisms. The


67

discussion is not formally structured by the teachers using a micro-script but, following the

FACILITATOR pattern (Pattern 4.1), they monitor, get in occasionally un-stuck and intervene

when necessary. The outcome of the discussion is a list of discussed topics and agreed conclusions.

In the “Jigsaw group” phase of the JIGSAW each pair explains to the others (which have studied

a different set of mechanisms) the mechanisms of its assigned set. After that, following a PYRAMID

structure they work on “combined scenarios”. At the first level of the Pyramid the work is

accomplished in pairs, and then the results are discussed and compared in “super groups” at the

second level of the Pyramid. The teachers also act according to FACILITATOR pattern during these

sessions. Additionally, to foster even more the participation of the individual students in every

phase of the script, at the end (in session #2) a final (individual) test about all the TCP mechanisms

is planned.

It is also worth mentioning, that students are involved, at the beginning of the course, in the

decision on how to grade the outcomes generated during the 22 hours of lab work, as well as in the

decision on how to form “super-groups” and how to assign mechanisms sets to each pair. This

involvement is based on a learning technique called “learning contracts” (Anderson, Boud, &

Sampson, 1996). This technique defends that by negotiating with the students different

organizational issues, they try to better understand the course plan and also show a higher

implication in the course development. As a result of this negotiation the teachers apply for the

composition of the groups the principles of FREE GROUP FORMATION (Pattern 4.2) and not of

CONTROLLED GROUP FORMATION (Pattern 4.3).

To conclude, we summarize in Table 3.4 the competencies and skills that the application of this

script supports. Those competencies and skills are aligned with the prescriptions of the educational

guidelines of the future European Higher-Education Space (EC, 2006).

This script is applied in the fall semester of 2005. Next subsection evaluates the effects of this

pattern-based script so that the conclusions test the value of the applied sequence of patterns and

provide feedback for further designs.

Table 3.4 Main expected competencies and skills

Competencies Skill

Thinking and arguing - Argument building

Problem solving and planning design - Situation analysis - Experiments design

Responsibility for the own learning

- Plan and guide own learning - Understand, take part in, and be sensible to others’ contributions - Work collaboratively

Working with large amount of information

- Information selection - Data observation and interpretation

Communication - Explanation of ideas to others


68

3.4.3.4 Evaluation methodology and results Evaluating the success of an experience such as the resulting of applying the previously

described script is a complex task due to the nature of the factors that must be taken into account,

some of them not easily quantifiable: individual characteristics of learners and educators, social

issues, impact of technology environment, etc. Therefore, the mixed evaluation approach based on

that proposed in (Martínez-Monés et al., 2003), that combines quantitative and qualitative

evaluation methods, has been selected for that purpose. Quantitative methods allow detection of

general trends related to students’ opinions and attitudes, while qualitative methods allow the

evaluator to understand these trends better by introducing context issues and considering

participants’ perspective (Martínez-Monés et al., 2005).

Table 3.5 explains the nature and purpose of the data sources that provide input information for

the evaluation of the project. Data collection and processing is supported by a set of computational

tools: questionnaires are edited and answered by means of the Quest tool; and analysis of qualitative

explanations contained in questionnaires is supported by the Nud*IST tool (SQR, 1997).

Table 3.5 Labels used in the text to quote the data sources of the “Network management” experience

Data source Type of data Purpose

Questionnaire at the end of the experience

Quantitative ratings and qualitative explanations of those ratings

Collection of students’ opinion on the experience: context, resources, organization, collaboration.

Observations

Registration of interactions among students by teachers during collaborative activities as well as their qualitative annotation

Identification of expected and un-expected modes of interactions among students.

BSCW logs Access to shared documents (for reading, annotation, modification, etc.)

Quantification of information interchange among collaborating groups

This evaluation results are structured according to the questions posed to the students at the end

of the experience. The answers to those questions (both quantitative and qualitative) provide some

useful indications for drawing preliminary conclusions on the success or failure of the experience.

The discussion on those indicators and the corresponding conclusions is enriched with data coming

from the other data sources enumerated in Table 3.5. Relevant conclusions as well as the supporting

evidences provided by evaluation data are summarized in Table 3.6. These preliminary conclusions

are sufficient to provide some hints about the value of the sequence of patterns applied in the

experience. We do not need more detailed conclusions for the objectives of this chapter. However, a

deeper analysis, such as the one carried out in Chapter Six, would be necessary to fully understand

the educational experience.


69

Table 3.6 Relevant evaluation conclusions and associated evidence

Aspect of interest Preliminary conclusion Evidence

Structure of the experience

The structure of the experience is perceived as a positive aspect in general.

Only 6% of the students have a negative opinion on the way the experience is structured, e.g.: “working with other groups has derived in the raising of many doubts. I do not like depending on others’ work”. Oppositely, most of the answers emphasize positive aspects of the script, e.g.: “working in super-groups is new, interesting, and funny”, “it is easier to understand certain TCP concepts when they are explained by experts on them”, “explaining something to your classmates demonstrates whether you understood it or not”, “it has reduced the work load”...

The experience is successful in promoting collaboration.

Observations (cf. Table 3.5) show that students have collaborated following teachers’ instructions with slight differences in the time each group devoted to the activities of session #10, and #11. BSCW logs have shown that before session #10 all documents are read at least 4 times by different groups (all pairs read the reports written by the other members of the same “super-group”). Some of the reports are read by up to 10 times by different groups.

Students consider that collaboration provides important advantages, although it requires a greater effort and creates doubts on the “correctness” of what students learn.

Students are asked on the advantages and drawbacks of introducing collaborative activities in the experience. Among advantages, students indicate: “it is more motivating than just listening to a teacher’s explanation”, “it helps to understand how important is to prepare your part properly so as to help your group mates”, “it reduces competitiveness”, “it is good for real life”. Negative opinions include mostly variations of “common doubts cannot be solved”, and “it requires more effort and time”.

Collaboration

Computational support of collaborative activities is adequate.

Students are requested to grade (in the range 0-10) the importance of BSCW and Quest tools for their work. Quest receives an average of 6.77 (deviation of 1.57) and BSCW an average of 7.86 (deviation of 1.86).

Competencies and skills

The experience “forces” the students to reach consensus, to discuss/listen with/to others, to build arguments, and to be responsible for their own learning.

Students are asked to enumerate what skills are required in order to complete the proposed tasks. The above skills are the most cited.

As a last interesting evidence found in evaluation data it is worth mentioning that many students

show their concern with the fact that becoming “experts” in a subject (as indicated by JIGSAW)

might imply less knowledge in the other subjects. Nevertheless, when processing the results of the

questionnaire that the students individually answered at the end of the experience (cf. Table 3.3) in

the range 0-10, the overall variance is only 1.72. Furthermore, if we analyze the results of the

questionnaire obtained by each type of “expert”, and taking into account that the questionnaire

includes four questions on each of the TCP mechanisms set (A, B, C, D, as well as four questions

on the “combined” scenarios), the number of failures corresponding to each group of questions is

quite similar. More concretely, the variance among the average number of wrong answers (range of

0-4), calculated for each group of questions and for each type of “expert” is always lower than 0.06.

These results indicate that the script minimizes the apparent problems of JIGSAW.


70

Summarizing, the evaluation results provide indications on that the script designed using the

CSCL scripting PL (of Appendix A) framed within the conceptual model proposed in section 3.3

has fostered collaboration in an effective way. Particularly, the script has contributed to a better

understanding of the applicability of the mechanisms, to a sharing of the workload, and to the

fostering of important competencies and skills for engineers.

Nevertheless, students also point out the fact that this kind of experience implies a different way

of working (possibly requiring from them more implication and a higher level of dedication) and

that sharing “expertise” may sometimes produce the feeling that some concepts have not been

understood properly. As discussed in subsection 3.3.3, this conclusion leaves us in a new context,

making explicit further forces, from which the need of a new pattern emerges. All in all, the

evidences achieved in this evaluation illuminate the fact that the sequence of patterns selected from

the proposed PL for this specific educational situation is meaningful, helpful and feasible.

3.4.4 Discussion: moral preoccupation, coherence, generativeness and creativity

CSCL scripting patterns encapsulate design experience rendering it available for re-use in

concrete educational design problems. The three scripts exposed in this section manifest that

achieving educational benefits by scripting CL is the moral preoccupation of CSCL scripting PLs.

These examples also make visible that the significance of patterns increases from their position in a

structure, and particularly a sequence, of other patterns. The proposed model for CSCL scripting

PLs conceptually illuminates the aim of creating (morphological) coherence of the potential PL

situated in the model (Alexander, 1999). This property is exemplified in the scripts of this

subsection.

In this way, the examples also manifest the generative power of the proposed PL, which enables

the creation of coherent wholes (scripts) in many different forms, with infinite variety in the details

but with a confidence of well-formed results. Though most patterns coincide in the three scripts, the

specific selected paths connecting the patterns though the PL are different. Moreover, their specific

application is different in each course, since the patterns are particularized and adapted to the actual

characteristic of each situation. As discussed in subsection 3.3.3, patterns are intended to serve as

inspiration when designing full-fledged scripts. They offer a balance between rigour and

prescription that provides useful guidance without largely constraining creativity. Quite the

contrary, patterns represent a vehicle of creativity and communication, which enables sharing and

discussing general ideas related to good practices.

On the other hand, it is true that the “degree of freedom” offered by the patterns is not always

the same. This issue has been already discussed in subsection 3.3.1, where it is pointed out that

patterns can be derived from general design ideas, which provides more degrees of freedom, to


71

patterns that formulate their reusable specialization. Degree of freedom in this context is defined as

the number of design options that are free to creatively vary without killing the principles suggested

by the solution of the pattern. From our point of view this is not in contradiction with the

Alexanders’ statement included on page 167 of (Alexander, 1979):

“A pattern language gives each person who uses it, the power to create an infinite variety of

new and unique buildings, just as his ordinary language gives him the power to create an infinite

variety of sentences.”

The sense of creativity that a PL supports is clarified in (Salingaros, 2000):

“A set of connected patterns provides a framework upon which any design can be anchored.

The patterns do not determine the design. By imposing constraints, they eliminate a large number of

possibilities while still allowing an infinite number of possible designs. The narrowing of

possibilities is, after all, an essential part of a practical design method.”

It is clear that ENRICHING THE LEARNING PROCESS (Pattern 1.7) has more degrees of

freedom or fewer constraints that JIGSAW (Pattern 1.1). But JIGSAW supports creativity: it needs

to be adapted to a particular situation, content and disciplines, the task of each activity of the CL

flow needs to be defined as well as the resources needed to support each activity. Teachers (or

learning designers) can even add more activities to those already proposed by the pattern and

probably any other issue that they can imagine (which may be stimulated by the ideas captured in

the pattern). Needless to say, a pattern with more degrees of freedom may be more reusable, but its

reuse may be less “automatic”.

In this line, (Gamma et al., 1995) on page 356 points out:

“When Alexander claims you can design a house simply by applying his patterns one after

another, he has goals similar to those of object-oriented design methodology who give step-by-step

rules for design. Alexander doesn’t deny the need for creativity; some of his patterns require

understanding the living habits of the people who will use the building, and his belief in the

“poetry” of design implies a level of expertise beyond the pattern language itself. But his

description of how patterns generate designs implies that a pattern language can make the design

process deterministic and repeatable.”

CSCL scripting patterns do also require teachers understanding the pedagogical principles that

are under the patterns. The progression through the PLs requires knowing the patterns beforehand

or reflecting upon the different design possibilities before their selection. Teachers are responsible

of selecting and applying the patterns in order to generate a script. Of course, there is an infinite

amount of possible designs but there is also a probability of achieving (quite) similar scripts.


72

The scripts exposed in this section consider the use of software tools to support the activities.

However, they are not automatically interpreted by learning environments (e.g. LMSs). It is the

teacher who is in charge of socially mediating the indications of the script (Dimitriadis et al., in

press). Yet the global objective of this dissertation focuses on computer-interpretable scripts, which

enables the automatic management of groups, activities and provision of tools, among other things,

useful in many educational situations. Next section discusses the potential computer-supported

applicability of CSCL scripting patterns focusing on the “life cycle” of scripts, what helps us to link

the contribution provided in this chapter with the objective undertaken in next one.

3.5 Potential computer-supported applicability

Different types of tools may benefit from the use of patterns for CSCL scripts or are needed to

facilitate their use. This section discusses the characteristics of some of them classified according to

the different stages (designing, instantiating, executing) of a CSCL script “life cycle” (cf. Figure

3.4).

LMS

STUDENTS

...

Awareness tool

Repository

Resources searcher

Authoring tool

Groups/roles management

tool

TEACHER (CL practitioner)

Execution

Instantiation Des

ign

and

auth

orin

g

CSCLscript

Figure 3.4 “Life cycle” of CSCL scripts and related tools

Within the phase in which CSCL scripts are conceived and created (design and authored), four

types of tools may be distinguished. CSCL scripting patterns and scripts (generated by applying the

patterns) may be collected in repositories. The main challenges for this type of tool is related to

facilitating collaboration for the joint development of scripts, and to “labelling” patterns and scripts

to ease their sharing. The reuse of CSCL scripting patterns can be fostered by incorporating them in


73

authoring tools so they may provide advice along the design process. In addition, CSCL scripting

patterns whose solutions propose structures of scripts can be represented computationally and

implemented in authoring tools as a kind of templates that can be easily completed in order to create

computer-interpretable scripts.

To create CSCL scripts, teachers also need to select tools (not to generate them) that are to

support the activities. In this line, semantic search of tools using ontologies is being researched by

Vega-Gorgojo, Bote-Lorenzo, Gómez-Sánchez, Dimitriadis, & Asensio-Pérez (2005). Therefore,

some patterns at the resource level (this is applicable to tools and learning materials) can act as a

mediator between the resource searchers and the user.

During the instantiation of a CSCL script, tools for managing roles and groups are also

necessary. This type of tools should easily enable the creation of multiple groups or roles and the

further binding of individuals according to the knowledge captured in the patterns and the pattern-

based structure of a script, which may be quite complicated.

Regarding the interpretation (i.e. execution) of CSCL scripts, the most important types of tools

are players and LMSs. A system that interprets CSCL scripts should consider the information

collected in the patterns. That is, it should be able of interpreting scripts at the learning flow level

or/and at the activity level, provide the needed resources, etc. (Bote-Lorenzo, 2005). The

information captured in patterns may be also used for feedback or adaptation purposes. In addition,

CSCL scripting patterns can be used by awareness tools. For instance, a CL flow awareness tool

(considering the patterns at the CL flow level) will allow participants to be aware of the

collaborative learning flow during execution: which activities have been accomplished, which are

the next ones, in which activities are involved the rest of the participants, etc. In many CL

situations, having such awareness is crucial since participants may change their groups depending

on the phase of the learning flow and may need to know the progress of their future team partners.

3.6 Conclusion

Among the different types of tools involved in the life cycle of scripts, the most directly related

to the main utility of CSCL scripting patterns are authoring tools. Authoring tools can integrate

interactive pattern-based design processes that guide teachers in the design of potentially effective

scripts. The incorporation of interconnected patterns in this type of tools provides a comprehensive

set of well-known design ideas, in a structured way, so that these ideas and the relations between

them are easy to understand.

Alexander’s PL consists of 253 patterns, ranging in scale from an independent (geographical)

region to an ornament. But he also proposes smaller PLs for particular projects (such as building a

porch, for which he provides a PL consisting of just ten patterns). In a similar way, this chapter


74

situates the scope of CSCL scripting PLs within a larger design space that considers the

relationships of this type of PLs with other PLs related to different pedagogical approaches and

didactics of specific subject matters.

In this chapter a conceptual model of PLs for CSCL scripting is proposed. It describes the

different types of patterns and relationships among them that can be used for generating CSCL

scripts. The types of patterns are described in an aggregation model that differentiates the patterns

according to their granularity: CL flows which comprises activities that are supported by resources

(tools and material). Roles and common CL mechanisms are modelled as elements that are integral

parts of flows, activities and resources. The patterns associated to the different elements of the

model can be related with connecting rules indicating that a pattern completes another pattern (by

refining the principles of the completed pattern) or that a pattern complements a second pattern

(forming a new larger whole). Patterns at the same level may also complement or/and complete

each other, may represent alternative patterns (mutually exclusive) and specialize other patterns.

The connections between the patterns provide guidance for their meaningful application and prevent

bringing together patterns that do not make sense from the pedagogical perspective. In this sense,

this chapter describes some general guidelines for applying the PLs that can be described with the

proposed conceptual model.

Considering the educational situation as a whole, the patterns should be selected so they are

helpful and feasible for the situation. The recommendation when applying a PL framed within the

proposed conceptual model is to start at the top (CL flow) and work towards the bottom. The

decision of applying or not applying a pattern depends on context. In addition, the resulting

sequence of selected patterns needs to be adapted according to the specific characteristics of the

learning situation.

To show the feasibility of the conceptual model a specific PL for CSCL scripting is proposed. It

comprises only 18 patterns but it illustrates the different types of patterns and relationships

considered in the model. The chapter also discusses how other patterns fit in with the model. New

patterns can be added to this PL or other PLs (reflecting the experience of other communities of

practice) can be proposed. The contribution of this chapter provides to the community a conceptual

model that represents an important starting point towards an agreed high level structure that enables

the sharing and communication of good scripting practices within and between communities.

Some patterns that form the PL are adopted from other authors’ proposals. Other patterns are

formulated by capturing the experience reported in the literature and/or using case studies as a

starting point. It would be very ambitious and unrealistic for a relatively short period of time to

develop a larger PL. Aspect that, on the other hand, is not necessary to reach the objectives of this

dissertation. Alexander et al. need 8 years to finish their published PL (Alexander et al., 1977).


75

Gamma proposes only some of his famous patterns in his Ph.D. thesis but he needs more time

before writing his book (Gamma et al., 1995).

It is also important to point out that the connecting rules considered in the conceptual model are

quite general and the specification of the particular details that may constrain even more the

relationships depends on particular PLs (e.g. “only the “Expert group” phase of JIGSAW can be

completed with BRAINSTORMING”, “INTRODUCTORY ACTIVITY: LEARNING DESIGN

AWARENESS complements JIGSAW by preceding it”). Future work in this sense may consider the

use of ontologies as a rich way to encode the semantics of the relationships between patterns

(Rosengard & Ursu, 2004; Sicilia, 2006). With this approach, we could explicitly represent the

meaning of terms (patterns) in vocabularies and the relationships between those terms. This would

also allow a powerful automatic organization and retrieval of the patterns.

The examples for applying the pattern exposed in this chapter illustrates that the proposed PL

and potentially any PL situated in the conceptual model comply with the properties of moral

preoccupation, coherence, generativeness and creativity. CSCL scripting patterns offer inspiration

and guidance based on equilibrium between rigour and prescription that enables creativity. They

impose constraints that are related to the intrinsic knowledge of the patterns. In this sense, they

prevent a number of design options while still allowing an infinite number of possible specific

scripts.

With regard to the generation of coherent scripts, this dissertation aims at providing users

(mainly teachers) with authoring tools that incorporate CSCL scripting patterns. Following the

metaphor of Alexander (Alexander, 1999), the genes are the functionalities of the tool that

facilitates the generation of potentially effective scripts adapted to particular situations according to

the decisions of the user. If these tools are accepted and widely spread through the educational

institutions and the scripts can be interpreted by as many as possible learning environments, we are

contributing to technologically enhance the teaching and learning practices. To reach this goal next

chapter undertakes the problem of computationally representing the scripts so that they can be

interpreted by software systems. As concluded in Chapter Two, IMS LD is a significant candidate

language for representing the scripts. The use of this specification, which is being adopted by

several LMSs, would foster the moral purpose of broadly spreading the application of well-known

CSCL practices.


76

CHAPTER FOUR

IMS LD SUPPORT FOR

COMPUTATIONALLY REPRESENTING

CSCL MACRO-SCRIPTS

This chapter tackles the objective of analyzing the suitability of IMS Learning Design for computationally representing CSCL macro-scripts, which describe flows of coarse-grained activities. In order to provide a systematic analysis of this problem, we point out the requirements of the scripts for their representation using LD. These requirements include common collaborative learning mechanisms (group composition, role and resource distribution and coordination) and flexibility demands (mainly flexible group composition). The possibilities and limitations of LD to support each of these needs is tested and illustrated by means of significant cases. The chapter collects the lessons learned from this analysis showing the capacity of LD notation to express CSCL macro-scripts but also considering the support of related specifications and tools.

The contributions of this chapter have been published in (Hernández-Leo et al., submitted), (Hernández-Leo et al., 2005), extended version of (Hernández-Leo et al., 2004), and (Hernández-Leo et al., 2005a), extended version of (Hernández-Leo et al., 2005b).

4.1 Introduction

How can CSCL macro-scripts be computationally represented so that they are interpretable by

learning environments? As Chapter One pinpoints, the main objective of this dissertation focuses on

macro-scripts rather than micro-scripts. Psychological and pedagogical concepts are involved in

their differentiation (Fischer et al., 2007). However, for the purposes of this chapter (devoted to

computational representations) the distinction is concentrated on their granularity: micro-scripts

describe the fine-grained actions that each participant should accomplish within CL activities, and

macro-scripts orchestrate flows of coarse-grained individual and CL activities. From the patterns

listed in the CSCL scripting PL proposed in Appendix A and discussed in Chapter Three, the


78

patterns at the CL flow level characteristically embody numerous features of macro-scripts.

Therefore, this type of patterns (what we call Collaborative Learning Flow Patterns or CLFPs

henceforth) are of special relevance regarding the pattern-based design of (machine-readable)

macro-scripts (simply “scripts” from now on to achieve a more readable text).

As pointed out in Chapter One, the global objective of the dissertation is to propose a design

process based on patterns for facilitation the creation of potentially effective scripts that can be

interpreted by general learning environments. Accordingly, to enable automatic scaffolding of the

teaching and learning process by means of scripts but without the need of developing specifically

devoted tools, this chapter proposes to formalize the scripts so that they can be interpreted by

engines integrated in learning environments. This approach has many time and cost advantages

related to efforts in development and enables tailorable systems. Chapter Two concludes that the

(XML-based) IMS LD language is the most interesting candidate to computationally represent the

scripts according to the definition of the specification and the interoperability prospects.

LD is broadly accepted as the de facto standard to formally model interoperable Units of

Learning (UoL). The specification is designed so that UoLs can describe any teaching-learning

process (Koper et al., 2004). Nevertheless, the LD support for implementing CSCL scripts is not

clear. The motivation of this problem is twofold. Firstly, since LD is a recent specification (IMS,

2003b), there is a lack of significant examples and efforts that show the possibilities of LD for

CSCL. Secondly, there is a lack of clarity regarding which characteristic of the scripts should be

expressed by the notation itself as opposed to which requirements can be supported by tools or even

other related specifications. Although partial work has been already accomplished (Gorissen et al.,

2005; Bote-Lorenzo, 2005; van Es & Koper, 2006; IMS, 2003a; Koper et al., 2005), a more

complete and systematic analysis is needed. As a consequence, some researchers are pointing out

presumed limitations of LD and alternative Educational Modelling Languages proposals to describe

CL situations are emerging (cf. subsection 2.4.2).

In this chapter we analyze the support of LD to implement the main requirements of CSCL

macro-scripts. Next section (4.2) describes the methodology applied in the analysis. Then, section

4.3 identifies the educational design requirements of CSCL scripts. Each requirement is illustrated

by means of references to CLFPs and actual scripts that significantly reflect them. A set of these

manifestations of the requirements are selected to be implemented using LD. This problem is

approached confronting the differences between the needs that can be satisfied by the LD notation

itself (section 4.4) and the needs that can be solved using tools or related specifications (section

4.5). Finally, section 3.6 concludes the chapter.

IMS LD SUPPORT FOR COMPUTATIONALLY REPRESENTING CSCL SCRIPTS

79

4.2 Methodology applied in the analysis

The analysis performed in this chapter is situated in the analytical phase of the global

methodology considered in this dissertation (cf. Chapter One). This analysis is approached as

follows.

First of all, significant requirements of CSCL macro scripts are identified in the literature. From

the numerous CSCL practices that manifest these requirements, we collect some examples that

significantly feature them. In this sense, the description of the requirements is complemented with

excerpts of actual practices illustrating them. The complete versions of the selected practices are the

CLFPs included in section A.1 of Appendix A and the well-known (in the CSCL community)

scripts listed in section B.1 of Appendix B.

The three renowned scripts employed in the analysis are Universanté, ArgueGraph and

ConceptGrid (Dillenbourg, 2002; Dillenbourg et al., 2007; Berger et al., 2001; Universanté, 2002;

Jermann & Dillenbourg, 2002; Kobbe et al., submitted). The fact that these scripts can be built

using the suggestions of CLFPs is important for the analysis. Universanté script is based on

JIGSAW structure (Pattern 1.1 of Appendix A). ConceptGrid is a variation of JIGSAW and might be

also considered as based on the SIMULATION CLFP (Pattern 1.3 of Appendix A). ArgueGraph

script can be constructed using PYRAMID (Pattern 1.2 of Appendix A) as a starting point. In this

sense, some of the requirements may be present both in CLFPs and specific scripts but others may

appear only when refining CLFPs with other types of patterns (cf. Chapter Two) and particularizing

them into full-fledged scripts.

From the excerpts of actual practices that significantly feature the requirements, we select

representative ones for their implementation with LD. The aim is trying to develop LD-represented

scripts that code the identified requirements. The involved LD elements and attributes as well as

illustrative XML excerpts that express the requirements (when applicable) are provided. Besides,

possible solutions relying on complementary specifications and supporting tools are discussed when

coding the characteristics related to the requirements is not feasible.

The work around the support of LD (level A) for computationally representing CLFPs is

accomplished first (Hernández-Leo et al., 2005; Hernández-Leo et al., 2005a). The main

requirements analyzed to that point are the definition of group hierarchies, rotation of roles,

specification of learning flows (coordination of activities) and CSCL tools. Then, the representation

of full-fledged scripts comprising further requirements is approached (considering also level B and

C of LD). This work is realized at OTEC in the OUNL during a three-month research stay

(Hernández-Leo et al., submitted).

Furthermore, the results of the evaluation phase reported in Chapter Six provide further insight

to the conclusions of this analysis. In that phase a multicase study considers different experiences


80

around other LD-represented scripts from different perspectives: teachers and designers authoring

the LD scripts, creation of a script not selected by us and enactment of another script with real

students.

4.3 Requirements of CSCL macro-scripts

As described in the previous subsection on methodology, this section introduces the educational

design requirements of CSCL scripts, which are identified in the CSCL literature. The main sources

are conformed by the review of CL available at (NISE, 1997), the CSCL scripting patterns collected

in Appendix A (especially the patterns at the CL flow levels, CLFPs), well-known CSCL scripts

and, particularly, the current research reports on framing their components and mechanisms (Kobbe

et al., submitted) as well as their flexibility demands (Dillenbourg et al., 2007). Based on these

sources, we classify the requirements for computationally representing the scripts into four different

types, namely group composition, role and resource distribution, coordination and flexibility. Each

requirement is illustrated with excerpts of CLFPs (when applicable) and actual full-fledged scripts.

4.3.1 Group composition

Appropriate and effective composition of groups is crucial in collaborative learning (NISE,

1997; Johnson & Johnson, 1999). Depending on the specific situation, a CL practitioner needs to

think over the composition of the involved group (or groups). This dependency may be implicit in

the learning flow structure (CLFP) of the script or it may appear when refining the structure into an

actual script.

We identify five central features related to group composition: hierarchy of groups, group size,

amount of groups, group formation policies and group formation at runtime. Next subsections

discuss each of these characteristics.

It is noteworthy that a script may not take into account all the features as necessary for success

(although a combination of them is possible). For example, some situations require a certain amount

of groups but they are flexible as far as the group size is concerned, while other situations give more

importance to group size. In addition, some scripts demand group compositions based more on

particular group formation policies than on actual group sizes (Kobbe et al., submitted).

4.3.1.1 Hierarchy of groups Scripts typically make use of groups forming hierarchies, i.e., groups may be composed of other

(smaller) groups or different (individual) roles.


81

This feature largely depends on the CL flow of a script. Thus, hierarchies of groups are implicit

in many CLFPs (cf. Appendix A) and consequently in their particularization into a script. A

significant example can be found in PYRAMID (Pattern 1.1 of Appendix A):

- Each individual participant studies the problem and proposes a solution. Groups (usually

pairs) of participants compare and discuss their proposals and, finally, propose a new shared

solution. Those groups successively join in larger groups in order to generate a new

agreed proposal. At the end, all the participants must propose a final and agreed solution.

Of course, group hierarchies also appear in specific scripts, such as Universanté (cf. Table B.1):

- There are country group and thematic groups. Each thematic group is composed of case

groups.

4.3.1.2 Group size Defining the desired number of group members is perhaps the most common suggestion of

scripts regarding group composition. They usually recommend keeping group size small for short

activities because, for example, there is not enough time for the group to become effective (NISE,

1997). However, larger groups might be adequate in longer situations (e.g. a course).

With regard to this feature, scripts may indicate the minimum, maximum or desired number of

participants. This aspect is determined in the actual script and not in its CL flow structure (if we do

not take into account that a group has at least two members), which can be (re)used in any length of

time and with many different tasks/activities, etc. An excerpt of a script (ArgueGraph, cf. Table

B.2) indicating group size is:

- An even number of at least 4 participants (works best with 20-30 participants) and a

tutor […] In the “survey” phase, all participants together form the class group and receive

one copy of the questionnaire. In the “conflict” and “elaboration” phase, all participants

are distributed evenly among groups of two…

4.3.1.3 Amount of groups As aforementioned, some scripts require a certain number of groups or at least a minimum or

maximum amount so that the dynamics they propose are afforded (Kobbe et al., submitted). Again,

scripts may indicate the minimum, maximum or desired amount of groups. This aspect may be

specified either in the CL flow structure (e.g. at least two experts groups, cf. JIGSAW in Appendix

A) or in a particular script:

- At least two different theme groups, two case groups (per thematic) and two country

groups (Universanté).


82

4.3.1.4 Group formation policies Depending on the situation, actually on the practical experience and research related to it,

groups should be heterogeneous or homogenous (NISE, 1997). The groups can be formed either by

the students themselves or by the teacher by referring to existing common features (e.g. gender,

age) or simply using a random assignment policy (cf. FREE GROUP FORMATION and

CONTROLLED GROUP FORMATION, Pattern 4.2 and Pattern 4.3 of Appendix A).

This aspect is determined in the particularized script. The Universanté script offers us an

example of a group formation policy:

- Each “case group” is formed of at least 1 participant per country.

4.3.1.5 Dynamic group formation Some scripts need groups to be formed at runtime. That is to say, the assignment of a person to

a group (and to a role) may depend on the result of a previous activity. A clear example of this

feature appears in ArgueGraph script:

- In the “conflict” and “elaboration” phases, all participants are distributed evenly among

groups of two, composed of participants with maximal difference in their responses to

the questionnaire. (In this script selecting peers with similar opinions instead of opposite

opinions would counteract the conflict generation principle of the script.)

4.3.2 Role/resource distribution

With the aim of fostering (positive) interdependence, CSCL scripts often make use of

distributing “tasks”, which is related to allocating roles and resources (e.g. the output or result of a

“collaborative (sub)tasks distribution” may be considered as a distribution of roles and/or resources

at runtime).

4.3.2.1 Role distribution In a script participants may assume one or more roles at the same time (e.g. one of the students

in a group is assigned to the role “scribe”… (Dalziel, 2003)) In addition, participants can switch

these roles with other participants (e.g. rotation of roles, scribe vs. speaker) (Kobbe et al.,

submitted).

Examples of role distribution in a CLFP and in actual scripts are:

- The two students are given specific roles that switch with each problem: Problem Solver

and Listener (TAPPS, Pattern 1.6 of Appendix A).


83

- Groups of four students have to distribute four roles among themselves (ConceptGrid

script, cf. Table B.3).

- Each person belongs to three different types of groups, which implies playing specific

roles depending on their “case”, “theme” and “country” (Universanté).

- An example of participants assuming several roles at the same time may be:

- In the particularization of the JIGSAW “Expert groups” phase, experts use simultaneously

a discussion forum where one of the experts is a moderator and the rest are

participants, and a collaborative conceptual map tool where one of the experts is the

writer and the others are annotators.

4.3.2.2 Resource distribution Another important feature of CSCL scripts is the distribution of resources (Kobbe et al.,

submitted). The amount of resources and their selected distribution may depend (or not) on the

number of groups, roles or participants.

For example, in order to foster knowledge exchange between group members JIGSAW proposes

that:

- Each participant in a group (“Jigsaw Group”) studies or works around a particular sub-

problem.

Similarly, SIMULATION (Pattern 1.3 of Appendix A) indicates:

- Each participant consults information about the problem to be simulated and prepares

the role of his/her character (with further information that may be available).

Therefore, particularizations of CLFPs into scripts usually keep the resources distribution of the

patterns and include new requirements concerning this feature:

- At least as many participants per country as there are case descriptions […] For each

case description one “case group” is formed […] All case descriptions are distributed

evenly among all case groups (Universanté).

- One copy of a questionnaire for each participant and another copy for each small

group. […] One argument sheet per question of the questionnaire and as many copies of

these sheets as are needed to provide one copy for each participant (ArgueGraph).

4.3.3 Coordination

Without doubt, coordination is an inherent characteristic of scripts. We distinguish between

coordination of (collaborative) leaning activities and coordination of “actions” within activities


84

(Ellis et al., 1994). In addition, coordination of artefacts is necessary: artefacts are often created by

an individual or a group and may be used in different activities and by different individuals or

groups (Miao et al., 2005).

4.3.3.1 Flow of (collaborative) learning activities The coordination of activities that make up a collaborative learning process is organized as a

flow of learning activities. The activities are mostly collaborative, but the learning flow can include

individual activities as well. The main problem of this type of flows falls into the synchronization of

groups and roles along the activities: a person may belong to a group in a certain activity and to

another group in the following one (then she probably needs to wait for the rest of the members in

her second group in order to start the second activity):

- Each participant in a group (“Jigsaw Group”) works around a particular sub-problem. The

participants of different groups that study the same problem meet in an “Expert Group” for

exchanging ideas. […] At last, participants of each “Jigsaw group” meet to contribute with

their “expertise” in order to solve the whole problem (JIGSAW).

Each CLFP proposes a flow of learning activities, which may be enriched or cut down and at

the same time particularized into a full-fledged script. For example, the flow of activities proposed

by JIGSAW is simplified in ConceptGrid (the “expert group” phase is cut down). The same flow of

activities is enriched in Universanté by including “theme group” activities.

4.3.3.2 Floor control While working together in the same activity, learners’ actions are sometimes guided or

constrained according to floor control mechanisms (e.g. a model of turn-taking in conversations or

when modifying an artefact). Floor control is typically needed in synchronous activities where

participants should know how and when they can interact. That is, floor control aims to manage

conflicts and structuring group work within activities (Boyd, 1996). Floor control policies may be

previously fixed (for example by the teacher) or may change dynamically (on the fly).

For example, when particularizing BRAINSTORMING (Pattern 1.4 of Appendix A) different

floor control policies can be considered (NISE, 1997):

- Different types of floor control can be used when generating ideas: methodically going

around the group, going around the group but letting students “pass” if they cannot

generate an idea (number of passes may be limited), a free floor control (not going

around the group, i.e. waiting for voluntary contributions).


85

Most collaborative activities included in CLFPs can use an actual floor control when they are

particularized into a script (e.g. achieving a consensus according to a voting floor control policy):

- In Universanté script, fact sheets and health strategies are shared artefacts that require floor

control mechanism to ensure data consistency.

- Achieving a consensus when answering the questionnaires in ArgueGraph scripts should

consider a floor control mechanism to ensure the consistency of the shared answer.

4.3.3.3 Flow of artefacts As aforementioned, artefacts (e.g. a document) are often created by an individual or a group.

They may be used in different activities and by different individuals or groups of the same script.

There are many explicit examples of artefacts flows in particular scripts:

- Since fact sheets are created until they are finally available to the teacher in Universanté,

they are used in discussions within theme groups, presented within country groups,

commented by the teacher, and modified by their original authors.

- ArgueGraph requires that the answers to the questionnaires (choices and arguments) of

individuals and small groups are displayed in different activities.

A similar problem appears when the teacher needs data (e.g. traces) in specific supporting

activities for regulation purposes (e.g. providing students with more information according to the

progression along the script).

4.3.4 Flexibility

The main drawback of scripts is their associated “risky” flexibility restrictions. Some of these

restrictions are intrinsic constraints of the script that justify its effectiveness (Dillenbourg et al.,

2007). These “intrinsic constraints” are in fact the essence captured in CLFPs (e.g. participants of

each “Jigsaw group” contribute with its “expertise” in order to solve the whole problem). Scripts

should keep this type of constraints and may add new ones (e.g. pair students with different

opinions). Therefore, flexibility concerning intrinsic constraints is not a strong requirement of

scripts.

However, when particularizing CLFPs into ready-to-run scripts “extrinsic constraints”, without

an underlying essential pedagogical principle, emerge. Inflexible extrinsic constraints can spoil

(needlessly) a satisfactory enactment of the learning situation. An example of extrinsic constraints is

the decision related to the duration of activities:


86

- In the ConceptGrid script, students can be allowed between 5 and 20 days for reading texts

and writing concepts definitions, the specific definition depends on the course calendar

(Dillenbourg et al., 2007). Even after defining it according to the course calendar (or the

class duration), unexpected situations may happen (e.g. teachers’ strike, students’ trip…)

and the time structure of the CSCL script should be consequently modified (because

otherwise it will be impossible to finish it).

Not only are modifications on the fly regarding the time structure required but changing

resources (content or tools), the order of the activities and the activities themselves is often also

crucial (e.g. adding a new text for clarification purposes). These flexibility requirements are more

complex in CSCL scripts than in other learning situations since the behaviour of groups is even

more unpredictable (e.g. need of providing additional materials or activities for faster groups so

they do not waste time waiting for the people to be joined in the following activity, cf. ENRICHING

THE LEARNING PROCESS, Pattern 1.7 of Appendix A). However, these requirements are actually

present in any learning situation (e.g. in an individual learning situation usually appears the need of

providing additional material and more time because something is not clear enough); and the

support of LD to specify these adaptive and personalized situations for individual learning are being

deeply analyzed by other researchers (Burgos, Tattersall, & Koper, 2006; Berlanga & García, 2005;

Burgos et al., 2006b; Burgos et al., 2006a; Zarraonandia, Dodero, & Fernández, 2006). That is the

reason why we focus our analysis on a common flexibility-demanding characteristic that

significantly appears in scripts: flexible group composition.

4.3.4.1 Flexible group composition A typical problem of CL situations, especially blended learning and synchronous virtual

situations, is the variability of students’ participation. It is common that a member of a group leaves

the course or cannot participate in a specific moment. It is often impossible for the teacher to guess

the precise amount of participants that are attending a particular session, whether they will be an

even or odd number or if some of them will join the class afterwards or cannot participate in a

specific moment.

- The current version of ConceptGrid runs with teams of 4, while the ideal group size range

from 3 to 5 (Dillenbourg et al., 2007). If a teacher has 23 attending participants she

should be allowed to form five groups of 4 and one of 3, or three groups of 5 and two of 4,

etc.

More complex situations come out when the participation varies at runtime. These situations

often require unexpected group composition modifications:


87

- Several students do not attend the “conflict phase” of ArgueGraph script and the teacher

(or the system) has already formed the groups (pairs). A reconfiguration of groups may be

necessary.

Each requirement described in this section implies a different challenge. However, they are all

shaped around the fact that they involve groups and multi-participant characteristics: specification

of groups (hierarchy, size, amount, formation policy and dynamic formation), distribution of roles

and resources (according to groups, roles and participants), coordination of activities, users’ actions

and artefacts (assuming that there are several participants, etc.), and flexible group composition.

Some requirements notably appear in the CLFPs. In these cases, a first study of the LD support

for expressing these requirements is accomplished in next section. Then, further analysis is carried

out with full-fledged scripts. It is noteworthy that all the requirements are significantly manifested

in Universanté and ArgueGraph scripts. Therefore, both scripts are selected for testing their

implementation with LD. Before codifying those scripts, their corresponding narrative use case

descriptions and activity diagrams are realized (cf. Appendix B) as indicated by the basic design

procedures for the development of UoLs (IMS, 2003a; Sloep et al., 2005).

4.4 Expressing the requirements using IMS LD notation

The problem of implementing the requirements of CSCL scripts using LD is approached by

examining the needs that can be satisfied by the LD notation itself and the needs that can be

supported by related specifications or supporting tools. This section emphasises the role of the LD

notation to express the requirements while next section discusses how tools and other specifications

may complement the implementation of the requirements.

As aforementioned, expressing some of the requirements is initially accomplished using the

CLFPs (cf. Appendix A). Then, the analysis is completed with Universanté and ArgueGraph scripts

(cf. Appendix B). The results of the analysis are collected in tables that also include selected

excerpts of suggested coding. They refer to the characteristics of the scripts that illustrate well the

use of LD elements and attributes for computationally representing the requirements (The complete

UoLs are available at http://gsic.tel.uva.es/collage/scripts and in the attached CD-ROM. In addition,

Appendix B shows screenshots of specific runtime executions (runs) of the UoLs representing the

scripts).

The main elements of LD are revised in subsection 2.4.1 of Chapter Two. For a better

understanding of the analysis detailed in the following subsections the reader might want to consult

the LD specification (IMS, 2003b).


88

4.4.1 Group composition

The capability of LD to support the requirements related to group composition are firstly

studied with PYRAMID CLFP, since it significantly manifest these requirements (as analyzed in

section 4.3).

A group is modelled in LD using the role element. Several persons can be associated to the

same role (thus forming a group). Besides, LD enables the design of processes that include several

roles. A collaborative learning situation can be described by associating multiple people and/or

multiple roles to the same learning activity.

Figure 4.1 shows the subtle way in which the hierarchy of groups defined by PYRAMID can be

generally specified with LD by means of nested roles (IMS, 2003b). Left part of the figure (a) refers

to the LD formalization of Pyramid roles. “Pyramid_N” is the role that will be played by the

participants in the last (N) level of the Pyramid. Role “Pyramid_i” refers to the first and

intermediate levels of the Pyramid (i ranges form 0 to (N-1)). Right part illustrates a possible result

of the dynamic role assignment while interpreting (but before running) a (three-level) PYRAMID-

based UoL.

(a) (b)

Figure 4.1 LD representation of the groups involved in the PYRAMID structure (a) and an example of the assignment of roles to users in a particular PYRAMID -based UoL (b)

The “Pyramid_N” role should be played by several individuals (minimum of 2 persons, min-

person=”2”), which actually form the largest group of the PYRAMID. Various Pyramid groups can

be dynamically created (created-new=”allowed” determines that multiple instances of a particular

role are allowed (IMS, 2003b)). (The alternative to dynamically creating the number of groups

when instantiating the UoL (occurrences of roles, created-new=”allowed”, can be created) is

defining the actual number of groups at design time: each declared role corresponds to a group, cf.

Role Pyramid_N

created-new=”allowed” min-persons=”2”

Role Pyramid_i

created-new=”allowed” match-persons=

”exclusively-in-roles”

1

2 . . M

Pyramid_3

Pyramid_2 (1)

Pyramid_2 (2)

Pyramid_1 (2)

Pyramid_1 (1)

Pyramid_1 (3)

Pyramid_1 (4)


89

Table 4.1) When a new occurrence (instance) of a role is created, the new occurrence is also the

parent of its defined sub-roles. Each individual in a “Pyramid_i” group can be bound exclusively to

one “Pyramid_i-1” sub-role (match-persons=”exclusively in roles”). A participant bound to the

occurrence (1) of the “Pyramid_1” role cannot be bound to any other occurrence of the same role

and is also bound to occurrence (1) of “Pyramid_2” role and to “Pyramid_3” role (cf. Figure 4.1).

Note that if a unique individual will be bound to each role in the first level of the Pyramid

(Pyramid_1), these roles are not strictly necessary. In this case, “Pyramid_N” and “Pyramid_1” are

equivalent. “Pyramid_N” represents a group or an individual depending on the environment

associated to the activity that this role plays in a particular moment (act), i.e. whether the service

included in the environment is collaborative or not.

Similarly, Universanté and ArgueGraph characteristics regarding group composition are

represented as indicated in Table 4.1 and Table 4.2.

Table 4.1 Computationally representing the group composition requirements of Universanté script

Requirements Involved LD elements

and attributes

Illustrative excerpts, supposing that there will be 2 countries and 4 participants per country, i.e.

2 thematic groups comprising 2 case groups Hierarchy of

groups (Each thematic group is composed of (at least) two case groups. There are also two country groups.)

Groups are modelled using roles, which can be bound to several persons. Roles can be nested, indicating that a role is divided in sub roles.

Group size (Since there are at least two countries, each case-group has (at least) 2 persons of different countries. Since there are 2 cases per theme, each thematic-group has (at least) 4 persons. Since there are four cases, each country-group has (at least) 4 persons.)

Role attributes min-persons and max-persons specify the required minimum and maximum numbers of persons bound to the role.

Amount of groups (At least two different thematic groups, two case groups (per thematic) and two country groups)

Each group can be modelled as a role. An alternative is using the role attribute create-new, which indicates that multiple occurrences of the role (and their sub roles) can be created at runtime (as previously illustrated with PYRAMID pattern)

<roles> <learner identifier="R-thematic-group-cancer" min-persons="4"> […] <learner identifier="R-case-group-breast_cancer" min-persons="2"> […] </learner> <learner identifier="R-case-group-lung_cancer" min-persons="2"> […] </learner> </learner> <learner identifier="R-thematic-group-aids" min-persons="4"> <learner identifier="R-case-group-pregnant" min-persons="2"> […] </learner> <learner identifier="R-case-group-drug_addict" min-persons="2"> […] </learner> </learner> <learner identifier="R-country-group-switzerland" min-persons="4"> […] </learner> <learner identifier="R-country-group-cameroon" min-persons="4"> […] </learner> […] </roles> To emphasize that a person bound to a thematic-group can be exclusively matched with one of its sub-roles (a case-group) the attribute match-persons of the role element can be used.

Group formation policies

(Each case group is formed of at least 1 participant per country)

This requirement cannot be formally specified but it can be added as information of the role in the referenced resource.

<learner identifier="R-case-group-breast_cancer" min-persons="2">[…] <information> <tem identifier="I-relation-case-country-groups" identifierref="R- relation-case-country-groups " /> </information> </learner> […] The resource "R- relation-case-country-groups" can be a text file indicating “Each case group is composed of at least 1 participant per country”

Dynamic group formation

(Not applicable)


90

Table 4.2 Computationally representing the group composition requirements of ArgueGraph script


and attributes

Illustrative excerpts, supposing that in ArgueGraph there will be only 4 participants,

i.e. 2 pairs Hierarchy of

groups, group size, amount of

groups (An even number of at least 4 participants (works best with 20-30) and a tutor. Participants will be distributed among groups of two.)

(cf. Table 4.1) <imsld:roles>

<imsld:learner identifier=”Student” min-persons=”4”> <imsld:title>Student</imsld:title> </imsld:learner> <imsld:staff identifier=”Tutor”> <imsld:title>Tutor</imsld:title> </imsld:staff> </imsld:roles> (See group formation polices and dynamic group formation)

Group formation policies,

dynamic group formation

(In one of the phases, all participants are distributed (by the tutor) evenly among groups of two, composed of participants with maximal difference in their responses to the questionnaire.)

Local personal properties can be used to model groups so that their value can be determined at runtime using global-elements. The persons with the same value of the property are in the same group. Conditions are in charge of coordinating the activities and artefacts according to this value of each participant’s property (i.e. according to their group).

The tutor dynamically determines at runtime to which group each student is bound (see second screenshot of survey phase, tutor, in Table B.9). Depending on the group, access to an activity containing a different instance of the shared questionnaire is provided (see screenshots of conflict phase, student, in Table B.9). <locpers-property identifier="LP-pair"> [...] <datatype datatype="string"/> <restriction restriction-type="enumeration">PairA</restriction> <restriction restriction-type="enumeration">PairB</restriction> </locpers-property> […] <if>  <is> <property-ref ref="LP-pair"/> <property-value>PairA</property-value> </is> </if> <then> <show> <learning-activity-ref ref="LA-fill-in-pairs-questionnaire-PairA"/> </show> <hide> <learning-activity-ref ref="LA-fill-in-pairs-questionnaire-PairB"/> </hide> </then>

4.4.2 Role/resource distribution

Rotation of roles cannot be explicitly specified using LD. However, Figure 4.2 shows a way of

implicitly describing the rotation of the roles involved in the TAPPS pattern: problem solver and

listener. The generic roles “Student-A” and “Student-B” are alternatively associated to the

explanation and listening activities within each act. These activities are connected through a

conferencing service.


91

<learning-design identifier="TAPPS-CLFP" level="B" uri=""> … <method> <play>

<activities> <learning-activity identifier="LA-solution-explanation"> <environment-ref ref="E-communication-tool" /> … </learning-activity>

<learning-activity identifier="LA-solution-listening"> <environment-ref ref="E-communication-tool" /> …

</learning-activity> </activities>

<environments> <environment identifier="E-communication-tool"> <service identifier="S-chat"> <conference conference-type="synchronous"> <participant role-ref="R-couple" />

</conference> </service> </environment>

</environments>

… <act> <role-part> <role-ref ref="R-student-A" /> <learning-activity-ref ref="LA-solution-explanation" /> </role-part> <role-part> <role-ref ref="R-student-B" /> <learning-activity-ref ref="LA-solution-listening" /> </role-part> </act> <act> <role-part> <role-ref ref="R-student-B" /> <learning-activity-ref ref="LA-solution-explanation" /> </role-part> <role-part> <role-ref ref="R-student-A" /> <learning-activity-ref ref="LA-solution-listening" /> </role-part> </act> …

</play> </method> </learning-design>

<roles> <learner identifier="R-class"> <learner identifier="R-couple" create-new="allowed" min-persons="2" max-persons="2"> <learner identifier="R-student-A" match-persons="exclusively-in-roles" /> <learner identifier="R-student-B" match-persons="exclusively-in-roles" /> </learner> </learner> <staff identifier="R-educator" /> </roles>

Figure 4.2 Excerpt of a TAPPS-based LD (arrows point referenced element definitions)

In accordance with the conclusions of the previous subsection, Table 4.3 and

Table 4.4 describe how the role distribution required by Universanté and ArgueGraph can be

expressed with LD. The distribution of resources is also detailed in these tables.

Table 4.3 Computationally representing the role/resource distribution requirements of Universanté script

Requirements Involved LD elements and attributes

Illustrative excerpts (or explanations), supposing that there will be 2 countries and

4 participants per country, i.e. 2 thematic groups comprising 2 case groups

Role distribution (Each person belongs to three different types of groups, which implies playing specific roles depending on their “case”, “theme” and “country”.)

Persons can be bound to one or several roles in the same run of the UoL. In this example each person should be bound to one country group and one case group (and thus to one thematic group). The moment in which they are playing each role is specified in the learning flow using the role-part element. To explicitly indicate that persons can be matched exclusively to one of the sub roles (e.g. case groups within a thematic group) LD provides the role attribute match-persons.

Resource distribution (All case descriptions are distributed evenly among all case groups.)

The resources can be associated to activity-descriptions or to environments, referenced in turn by other LD elements depending on the distribution needs.

In this example, an environment per “case description” (a learning-object) is defined. An activity-structure per case is also defined. Each activity-structure references one of the environments and a common learning-activity explaining the task. Each activity-structure is bound to a role in different role-parts of the same act. (See package UniversanteUoL.zip at http://gsic.tel.uva.es/collage/scripts or in the attached CD-ROM)


92

Table 4.4 Computationally representing the role/resource distribution requirements of ArgueGraph script

Requirements Involved LD

elements and attributes


i.e. 2 pairs Role

distribution (Pairs can be considered as different roles to be

distributed.)

(See group formation at runtime, Table 4.2)

Resource distribution

(One copy of a questionnaire (including argument sheets) for each participant and another copy for each small group.)

The created artefacts (answers to the questionnaires) can be stored in local properties (o global if it needs to be stored beyond the UoL). If the artefact is created individually, the property can be personal (locpers-property) and if it is associated to a group of type role (locrole-property)

The results of individual questionnaires are stored in local personal properties <imsld:locpers-property identifier="LP-individual-argument"> <imsld:title>Argument for my choice</imsld:title> <imsld:datatype datatype="text"/> </imsld:locpers-property> The persons in the same group (pair) share local properties for their joint answers. Using global-elements different persons can view and set the value of the property. <imsld:loc-property identifier="L-pairA-argument"> <imsld:title>Argument for PairA's choice</imsld:title> <imsld:datatype datatype="text"/> </imsld:loc-property>

On the other hand, resource distribution is tightly related to the flow of artefacts considered as a

coordination requirement. Next subsection explores this aspect.

4.4.3 Coordination

The learning flows described by CLFPs can be expressed in the LD method. A method

contains one or more plays, which are modelled according to a theatrical play with acts, role-parts

(linking roles to activities) and activity structures (wrapping several activities). These plays run in

parallel, independent from each other. Acts determine whether, when, and for what roles an activity

and resources are to be used (IMS, 2003b). Figure 4.3 illustrates how the learning flow described by

JIGSAW can be modelled using LD. It makes use of three acts whose boundaries are determined by

the synchronization points between the groups (same persons change roles/groups and need to

synchronize with the persons joining their group). In this sense, Figure 4.3 also shows that the

groups required in JIGSAW are modelled using roles (as explained in subsection 4.3.1). The persons

associated to the “Jigsaw group” role can work individually or collaboratively depending on the

activity.


93

method

play Jigsaw- CLFP

act role role-part assigned

1.1 (member in) “Jigsaw group” learning-activity: individual-study 1.2 Teacher support-activity: control-activity

Complete act when...

2.1 “Expert group” activity-structure: subproblem-activities 2.2 Teacher support-activity: control-activity


3.1 “Jigsaw group” activity-structure: globalproblem-activities 3.2 Teacher support-activity: control-activity


Complete play when last act has been completed

Complete method when play Jigsaw-CLFP has been completed

Figure 4.3 Expressing the JIGSAW learning flow with the LD method

Therefore, the general flow structures described by the CLFPs can be formalized using level A

of LD. More complex flows may appear in full-fledged scripts, as it is the case of Universanté and

ArgueGraph scripts that benefit from level B and C LD constructs (cf. Table 4.5 and Table 4.6).

Table 4.5 and Table 4.6 also indicate the floor control effect that can be achieved with “shared”

properties. This ensures the consistency of the created or modified artefact, however more

sophisticated floor control mechanisms may be required depending on the situation. Properties

together with global-elements (monitor services) are used to set and view the value of artefacts

along the learning process. These LD elements enable the description of flows of artefacts between

activities.


94

Table 4.5 Computationally representing the coordination requirements of Universanté script


and attributes

Illustrative excerpts (or explanations), supposing that there will be 2 countries and 4 participants per country, i.e. 2 thematic groups comprising 2

case groups Flow of CL activities

(Cf. Table B.4)

The flow of activities is expressed in the method. A method contains one or more plays, which are modelled according to a theatrical play with acts and role-parts. The plays run in parallel. Acts together with conditions (and also notifications) determine whether, when, and for what roles activities and resources need to be available. Different types of conditional expressions can be used to orchestrate the flow (IMS, 2003b).

This script requires a method with five acts. Each act contains a role-part per role of the “type” of role that corresponds to each phase. In the cases that the activities are performed by persons belonging at the same time to two groups (E.g. “within each thematic group, the members of each country group create a fact sheet”), it is necessary to add conditions with two expressions of type is-member-of-role. See the screenshots of the “Thematic group: the members of each country group create a fact sheet” in Table B.7. <if> <and> <is-member-of-role ref="R-thematic-group-cancer" /> <is-member-of-role ref="R-country-group-switzerland" /> </and> </if> <then> <show> <class class="C-fact-sheet-cancer-switzerland" /> </show> <hide> <class class="C-fact-sheet-cancer-cameroon" /> <class class="C-fact-sheet-aids-switzerland" /> <class class="C-fact-sheet-aids-cameroon" /> </hide> </then> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <div class="C-fact-sheet-cancer-switzerland"> <p>Please create fact sheet concerning the "Cancer" status in your country, Switzerland: </p> <ld:set-property ref="L-fact-sheet-cancer-switzerland"/> </div>

Floor control (Fact sheets and health strategies are shared artefacts that require floor control mechanism to ensure data consistency.)

The UoL uses local properties to model fact sheets shared by persons belonging at the same time to the same country-group and to the same thematic-group (persons who are members of two roles). A student (in a country group and with a particular theme) can set a local property that can be viewed and modified by another student (in the same country and thematic group). The UoL also uses local role properties to model health strategies shared by the people belonging to the same case-group (same role). A student (in a case group) can set a local role property that can be viewed and modified by another student (in the same case group). (As can be seen in Table B.7 the fact sheets are modelled using properties of type “text”. Another solution is using properties of type URI or “file” so that a document (the fact sheet) is uploaded. This document may be created with a collaborative text editor (which may implement floor control policies) or, if the participants working in the same fact sheet are F2F in the same country, with an editor in a shared PC.)

Flow of artefacts (Since fact sheets are created until they are finally made available to the teacher, they are used in discussions within theme groups, presented within country groups, commented by the teacher, and modified by their authors.)

Properties can be used to model individual and shared artefacts. Global-elements and monitor services are used to set and view the value of their own or that of others properties. These elements are referenced by the different activities that require the artefacts.

The value of the properties modelling the facts sheets can be set and viewed by the participants by means of global-elements in the several activities. In this excerpt users can modify their fact sheet. <html xmlns:ld="http://www.imsglobal.org/xsd/imsld_v1p0" xmlns="http://www.w3.org/1999/xhtml"> […] <div class="C-fact-sheet-cancer-switzerland"> <p>Please modify the fact sheet (Cancer status in Switzerland) according to the methodological comments: </p> <ld:set-property ref="L-fact-sheet-cancer-switzerland"/> </div> […] </html>


95

Table 4.6 Computationally representing the coordination requirements of ArgueGraph script


and attributes


i.e. 2 pairs Flow of CL activities

(Cf. Table B.8)

(Cf. Table 4.5) This script requires a method with four acts. Each act contains two role-parts one for the activities of the students and one for the activities of the tutor. The exception is the act corresponding to the “conflict” phase. This act comprises three role-parts one for each Pair and one for the tutor. Note that although the role referenced in the learners’ activities is not explicitly the Pair but the Students, the conditions manage to show the correct activity to the students (according to their associated Pair). <method> <play identifier="P-1" isvisible="true">[…] <act identifier="A-2"> <title>Conflict phase</title> <role-part identifier="RP-Student-2-PairA"> <role-ref ref="Student"/> <learning-activity-ref ref="LA-fill-in-pairs-questionnaire-PairA"/> </role-part> <role-part identifier="RP-Student-2-PairB"> <role-ref ref="Student"/> <learning-activity-ref ref="LA-fill-in-pairs-questionnaire-PairB"/> </role-part> […] </act> […] </play> […] </method>

Floor control (Achieving a consensus when answering the questionnaires should consider a floor control mechanism to ensure the consistency of the shared answer.)

Properties (local or global properties or local role properties) can model “shared” artefacts whose consistency is managed as follows. All participants with access via global-elements to the property can view and modify its value. The final value is the latest set.

The persons in the same group (Pair) share local properties for their joint answers to the questionnaire and related arguments. <-- Answer to the question: In a courseware, when a student makes an error is better to --> <loc-property identifier="L-pairA-answer1"> <datatype datatype="text"/> <initial-value>Select</initial-value> <restriction restriction-type="enumeration"> 1. Tell the student he made a mistake and give him the correct answer. </restriction> […] <restriction restriction-type="enumeration"> 4. Give the student some time to find out the mistake by himself. </restriction> </loc-property> <loc-property identifier="L-pairA-argument1"> <datatype datatype="text"/> </loc-property>

Flow of artefacts (The answers to the questionnaires (choices and arguments) of individuals and small groups are displayed in different activities.)

(Cf. Table 4.5) This solution (whose result is illustrated in Table B.9) does not consider the fact that for the students the answers of their colleagues may be anonymous. By using monitoring services of global-elements of type view-property of supported-person this cannot be achieved. However, this problem could be solved, for instance, by adding more local properties.

The value of the properties storing the choices and arguments of the individuals and the pairs can be viewed by the participants by means of global elements <html xmlns:ld="http://www.imsglobal.org/xsd/imsld_v1p0" xmlns="http://www.w3.org/1999/xhtml"> […] <p>Individual answer to the questionnaire:</p> <p>In a courseware, when a student makes an error is better to: </p> <ld:view-property ref="LP-individual-answer" property-of="supported-person" view="value"/> <p>Argument: <ld:view-property ref="LP-individual-argument" property-of="supported-person" view="value"/></p> […] </html>


96

4.4.4 Flexibility

The flexibility requirements as far as group formation is concerned are generally approached in

Table 4.7 and Table 4.8. The addressed characteristic deals with the unexpected number of students

that eventually participate in the actual execution of the UoL.

Table 4.7 Computationally representing the flexibility requirements of Universanté script

Requirements Involved LD elements and attributes

Illustrative excerpts (or explanations), supposing that there will be 2 countries and 4

participants per country, i.e. 2 thematic groups comprising 2 case groups

Flexible group composition

(What happens if we do not have the precise desired number of participants or if it varies at runtime?)

The possibility of detailing the maximum/minimum number of persons within each group (cf. subsection 4.4.1) allows flexibility to a certain extent. In this example, more than 8 students can be involved. It is possible to have more than 4 participants per country by associating 2 persons of this country (instead of only one) to one or several case-groups. Therefore, the flexibility regarding attendance (in a particular moment) that provides this UoL is that more than 4 participants per country could participate. The limit of flexibility appears when less than 4 participants per country participate, what counteracts the educational principles of the UoL. In addition, having too many participants (e.g. more than 8 per country) may not be recommended.

Table 4.8 Computationally representing the flexibility requirements of ArgueGraph script


and attributes


i.e. 2 pairs Flexible group composition

(What happens if we do not have the precise desired number of participants or if it varies at runtime?)

As mentioned in Table 4.7, the possibility of detailing the maximum and minimum number of persons within each group expresses flexibility regarding group composition variability. In this example, more than 4 persons can be involved and the small groups can be of more than two persons depending on the students’ participation in a certain moment. (E.g. if there are five participants, the teacher can form a group of two and a group of three). In addition, since the tutor forms the pairs at runtime, the particular needs of the moment will be considered. Later the tutor can also change the association of a student to a group coming back to the pair formation activity.

Additional conclusions referring to flexibility which consider the combined analysis performed

in the previous subsections are discussed in next subsection. It also reviews the results collected in

the tables for the detection of LD notation limitations for computationally representing the scripts.

4.4.5 Discussion and revision of the limitations regarding LD notation

Although modelling Universanté and ArgueGraph using LD is not easy and demands a deep

knowledge of the specification and several weeks of devoted work, the analysis performed in the

previous subsections show the many possibilities of LD for representing the requirements of

CSCL scripts. Nevertheless, there are some detailed issues that cannot be formally expressed

using the notation but which may be supported by tools or other specifications.

Regarding group composition, the use of the create-new attribute provides flexibility since a

specific number of groups is not predefined at design-time. However, the desired amount of groups

cannot be explicitly defined in the same way as it can be done with the group size (unless the

relationships between the values of min-persons attributes of roles and their sub-roles are used).

Universanté script requires that each thematic-group has at least 2 case-groups and that there are at


97

least 2 thematic-groups and two country-groups. If a thematic-group must include at least 4 persons

and its sub-role case-group can have only 2, then at least another occurrence of the case-group

should be created. Indicating in this way the number of thematic and country-groups is not possible

unless we defined a super-role. In order to represent CSCL scripts similar to this script, having

attributes to explicitly indicate the minimum and maximum number of new occurrences that can be

created during runtime would be useful.

In addition, the relationship between some roles to describe group arrangements as well as other

types of group formation policies cannot be formally specified (cf. Table 4.1). That is to say, it

cannot be described in such a way that automatic binding of persons to roles according to these

characteristics is achieved.

With regard to role distribution, rotation of roles can be implicitly represented by rotating

activities and defining “neutral” roles as proposed in Figure 4.2. Moreover, in the case of the

conference service four different types of system roles (participant, observer, conference-manager,

and moderator) can be distributed. Nevertheless, the solutions pointed out so far for role distribution

do not consider the reasonable situations in which persons assume several roles within the same

activity. The example that we indicate in section 4.3 to illustrate this problem is the following: in

the “Expert Group” phase of JIGSAW, experts may use simultaneously a discussion forum where

one of the experts is a moderator and the rest are participants, and a collaborative conceptual map

tool where one of the experts is the main writer and the others are annotators. In these cases the

solution relies on specifying these roles within supporting services. In this sense, LD defines within

the conference service four different types of system roles (participant, observer, conference-

manager, and moderator) that can be distributed. But what happens if different roles and other

services are required?

On the other hand, when creating a new occurrence of a role (allowed by the create-new

attribute), new instances of the associated local role properties are also created. This facilitates the

distribution of resources, which in this case could not be accomplished by referencing different

environments to different role instances (these instances are not available at design-time).

Furthermore, when relying on the create-new attribute, the conditions used in Table 4.5 for

checking if a person is a member of two groups at the same time cannot be used. Instead, a new role

modelling this situation should be declared and referenced in the corresponding role-parts (see the

alternative (partial) version of Universanté UoL using the created-new=”allowed” attribute and

local role properties (universante2-uol.zip) available at http://gsic.tel.uva.es/collage/scripts and in

the attached CD-ROM).

Though the LD support for expressing complex learning processes is questioned in the literature

(Miao et al., 2005; Torres, Dodero, Aedo, & Zarraonandia, 2006), the results of our analysis


98

indicate that for the studied types of scripts the LD expressiveness is satisfactory. Besides, the

conclusions concerning the support of LD to represent the flow of artefacts between activities are

also positive. Nevertheless, the flow of artefacts between services (tools) supporting the same or

different activities is another problem which is clearly not supported by the specification (Miao et

al., 2005; Helic, 2006; Palomino-Ramirez, Martínez-Monés, Bote-Lorenzo, Asensio-Pérez, &

Dimitriadis, 2007).

Moreover, further flexibility needs to those addressed in subsection 4.4.4 may emerge at

runtime. Some of these situations can be planned a priori considering adaptation issues enabled by

the properties and conditions included in the level B of LD (Koper et al., 2005; Burgos et al.,

2006). However, other unpredictable demands need to be undertaken by teachers and students

via modifying some script features during the enactment (Dillenbourg et al., 2007).

To sum up, many of the requirements are addressed by the LD notation whereas some are not

fully supported. Adding new constructs to LD would increase even more the complexity of the

specification (and would probably decrease its interoperability prospective), what it is not desirable.

At this point it is important to consider the differences and relationship between the LD

specification itself and its relation to other (interoperability) specifications and tooling.

4.5 Supporting the requirements using related tools or specifications

LD relies on other specifications and tools that, conveniently integrated, envisage an extensive

support for the implementation of any teaching-learning process and, particularly, CSCL script. For

example, to implement the questionnaire of ArgueGraph Script there are three possibilities: using

LD properties (cf. Table 4.6), interoperating with IMS Question and Test Interoperability

specification (Vogten et al., 2006) or referencing an external questionnaire tool within an

environment (Hernández-Leo et al., 2006b).

In this section we refer to four different types of tools:

- authoring tools devoted to the creation of the scripts (e.g. Reload (Milligan et al., 2005)),

- players, whose main component is an LD engine (e.g. CopperCore (Martens et al., 2005))

for running the scripts,

- administration tools in charge of managing roles (and groups) and assigning participants to

roles (e.g. Clicc of CopperCore),

- and supporting tools, which refer to any kind of tools used to support an activity or part of

an activity (e.g. a chat (Hernández-Leo et al., 2006b)).


99

4.5.1 Addressing the limitations using complementary specifications and tooling

According to the computational representations exposed in the previous section, administration

tools should show to the users (e.g. teachers) the textual information of the roles so that they are

aware of the suggested group formation policies or amount of groups (this information is not

formally specified and cannot be interpreted by the engine). It is also possible to envisage the

possibility of having a dedicated specification to represent this type of characteristics. The resource

referenced in the information element as well as the administration tool should be thus compliant to

this eventual specification. In this sense, administration tools should also adopt the IMS Enterprise

Services specification (IMS, 2004b), which defines how systems manage the exchange of

information that describes people, groups and memberships within the context of learning.

Administration tools are also needed at runtime either as supporting tools, such as a “grouping

service” providing the functionality to (automatically, semi-automatically or manually) assign

people to roles (Burgos et al., 2006), or just as a utility of the player. For example, in the Problem-

Based Learning example included in (IMS, 2003a) students dynamically decide who is going to be

the chairperson during runtime. Once decided, (s)he should be bound to the corresponding

(previously defined) role using, for instance, a utility of the player. In the case of the ArgueGraph

script an alternative to the use of properties is defining the Pairs as roles and using a grouping

service to bind the students to the roles according to their responses. Of course, a “dedicated activity

service” devoted to analyze students’ answers and automatically form the pairs (by setting

properties or binding users to roles) could be used as well. This solution significantly reduces the

workload of the teachers as compared to the solution indicated in Table 4.2.

Rotation of roles can be also realized if the player provides a mechanism that allows roles

switching. In the Literature Circles example (IMS, 2003a), each session, which implies role

rotation, is viewed as a different run of the UoL. Regarding role distribution, supporting

collaborative services may also define their own roles, which should contain references to roles in

the LD. If the number of groups is not known at design-time, the requirement of resource

distribution can be also supported using a “shared repository service”. A different instance of the

service (e.g. a different folder) should be provided to each occurrence of the role participating in the

service. An example of this solution is presented in (Hernández-Leo et al., 2006b). This idea is also

employed for providing a different instance of a “synchronous conference service” (a chat) to the

members bound to each occurrence of a role. In addition, supporting collaborative tools may

implement more sophisticated floor control mechanisms than the effect achieved using properties.

Creating the fact sheet can be accomplished with a “collaborative text editor service” or a

“collaborative whiteboard service” making use of, for instance, turn-taking policies.

With the aim of addressing the problem of specifying the flow of artefacts between tools (not

supported by the LD notation), Palomino-Ramirez et al. (2007) plan to use a workflow standard


100

language in a way that complements LD maintaining interoperability. It is interesting, for example,

in a situation in which an external tool for questionnaires is used. In this case, the compilation of the

choices and arguments should be done by the tool. In order to make available this compilation to the

participants in other activities it should be stored either in a predefined property or in another tool,

which can be eventually the same.

As aforementioned, since UoLs can be designed considering adaptation issues, various

flexibility issues can be pre-defined (e.g. activities to be added or removed by the teacher on the

fly). Besides, the design of LD runtime systems meets flexibility requirements (Tattersall et al.,

2005). The main idea behind this design relies on the distinction between an abstract description

(UoL) and its specific instantiations (runs). Consequently, the same UoL can be executed in

different settings with different participants. Administration tools should be also available at

runtime for managing unexpected group composition variations. Moreover, minor modifications to

runs are also possible by changing the details in the related UoL. Furthermore, Zarraonandia et al.

(2006) propose an alternative to introduce slight variations on a run in progress without modifying

the UoL.

Dillenbourg et al. (2007) argue that a supplementary representation of the script (additional to

the actual UoL to be executed) specifying their intrinsic and extrinsic constraints could be also

handled by script engines to support flexibility. In this way, the tools enabling scripts modifications

by interacting with the engines can consider this information (e.g. extrinsic constraints are allowed

to be modified by teachers and students since these changes do not kill the potential effectiveness of

the script). Further solutions for managing flexibility, such as associating several students to the

same (redundant) role (enabled by LD and LD tooling) or such as using artificial agents simulating

not attending students, are also proposed in (Dillenbourg et al., 2007).

Moreover, tooling (mainly players) user interface features are important for the actual use of

scripts (Dillenbourg, 2002; CRAFT, 2007). In this sense, an interesting topic to be studied is related

to the effects of interface features that result from the interpretation of scripts. And, consequently, a

problem would be how the tools can address this problem. Is a general common way of playing LD

valid for every script? Or is needed additional information (e.g. another specification) for describing

this type of aspects?

Finally, it is worth mentioning that another important supporting tool for CSCL scripts is an

“awareness service”. This kind of tool should provide information updated in real time by the

engine about the progress of the participants (and groups) in the learning flow. This is not only

essential for managing flexibility (accomplishing modifications according to the awareness

information) but it is also fundamental to provide an adequate context for each own work by


101

understanding the work of others. The LD monitor service facilitates this functionality if it is

previously specified in the UoL (Gorissen et al., 2005).

Along the chapter, we argue that a group-based activity can be described by associating a role

played by several persons or/and multiple roles to a learning activity that provides at least one tool

(LD service) that mediates collaboration. This is true; however LD only defines two basic

collaborative services: e-mail and conferencing. The accomplished analysis makes clear that

collaborative learning situations require more services (e.g. collaborative editors, document sharing

tools, discussion forums, shared whiteboards, etc.). Since the definition of all the possible required

services is an extremely demanding task which, on the other hand, decreases the interoperability

power of the specification (the compliant learning environments would need to implement all the

services), we claim that a feasible solution is generally specifying group-services (as similarly done

with the conference service) so that several common “configuration characteristics” can be

described at design time. Next subsection points out some ideas in this sense, which do not intend to

be a definitive proposal but an initial approach to reach a consensus in that direction.

4.5.2 Discussion: general specification of group-services

LD states that if multiple individuals are to collaborate or work together at the same time, this

has to be done through a service in their assigned environment which supports this collaborative

capability (IMS, 2003b). As aforesaid, due to interoperability reasons LD only defines four basic

services, two of which are collaborative (e-mail and conferencing). Besides, the specification states

that that other needed services should be specified by the designers. The problem is that LD does

not permit to describe collaboration-related capabilities when defining (or configuring) a new

service. Some of the interesting features that can be used to configure group-supporting services

are:

- Floor control policy that guides learners’ actions. If it is fixed and previously established

by the teacher/designer or it is dynamic (it is not previously established) or, simply, there

is not any floor control policy.

- The type of communication skills that is to be used in the collaboration to be supported by

the tool defined in the service (Osuna & Dimitriadis, 1999): writing, speaking, gesturing,

drawing, etc.

- The roles (of the LD) that participate in the same instance of the service. A different

instance of the service should be provided to each occurrence of the role (when create-new

attribute of the role element is set to allowed) participating in the service.


102

- If the whole workspace of the collaborative tool described in the service is public or shared

(every participant have access to the workspace), or private (each participant only has

access to his/her workspace) or a mixed workspace (Ellis et al., 1994).

- The type of interaction that is supported by the service. This element distinguishes between

direct interactions with a source and one or more receivers (e.g. a contribution to a

discussion in a chat), indirect interactions, mediated by a shared object (such as a

document or a piece of a puzzle) and participation-oriented interactions, that allow to

annotate “participation” of an actor in situations where no receptor has been identified (e.g.

a post in a discussion forum without answers) (Martínez-Monés, 2003).

- Roles that the service supports (e.g. writer and annotator in a collaborative conceptual map

tool).

- Type of awareness information (updated information about other people’s presence,

location, state of the task, etc.) needed and provided by the service and from what roles is

needed to provide information (Gutwin & Greenberg, 1999). Possible values of the type of

awareness are identity (who is participating?), action (what is she/he doing?), location

(where is she/he working), etc.

To address the specification of these common features of group-based services so that proper

tools support the collaborative activities, several solutions are possible. A first approach refers to

extending LD with an element that enables the general definition of a special type of service, called

(for example) groupservice, whose information model includes the aforementioned features (an

example of this approach is provided in (Hernández-Leo et al., 2005)). Another solution is

developing a new specification for the definition of the collaborative services and their

interoperation with LD (engines). Finally, the creation of an ontology for describing this type of

tools is another promising possibility, which is complementary to the previous ones. Searching tools

with the required features to adequately support the specific activities is addressed in (Vega-

Gorgojo et al., 2005).

Supporting tools may also implement finer-grained scripts (micro-scripts) so that interaction

processes within activities are scaffolded too. In fact, tools implementing particular floor control

mechanisms (e.g. turn-taking polices) also aim at structuring interactions at a micro level. This idea

is discussed in next subsection.

4.5.3 Discussion: “hardwired” in tools vs. computer-interpretable micro-scripts

Floor control polices are highly related to micro-scripts in the sense that they organize the

specific actions that should be accomplished within activities. This argument leads to the idea that

micro-scripts can be embedded in supporting tools associated to the activities of an LD-represented


103

macro-script, thus achieving composition of micro- and macro-scripts (Harrer, 2006; Haake &

Pfister, 2007). An alternative approach is computationally representing the micro-scripts as well so

that they are interpreted by an engine.

At this point the following question arises. Can LD be used for computationally representing

micro-scripts? To answer this question, similar research to the detailed analysis contained in this

chapter concerning macro-scripts needs to be carried out. According to a comment included on page

229 of (Koper & Tattersall, 2005), LD is mainly devoted to describe learning flows and it does not

support the specification of detailed learning actions within activities (activity granularity level).

Nevertheless, a positive answer is envisaged if we consider the following example related to

argumentative micro-scripts. The example is devoted to guide the construction of a specific

argumentation sequence within discussion activities (Weinberger et al., 2005) (see also the scripts

that can be generated using the “Debate” pattern language proposed in (Goodyear, 2005)). The first

message of a discussion thread is labeled “argument”. The answer to an argument is categorized as

“counterargument” and a reply to a counterargument is labeled as “integration”. This script could be

easily expressed with LD by means of grouped properties with titles “argument”,

“counterargument” and “integration” and coordination elements (mainly acts and conditions).

Alternatively, other languages specifically dedicated to formalize particular types of micro-scripts

(e.g. argumentative micro-scripts) may be developed. In any case, the interoperability between

macro and micro-scripts should be afforded.

4.6 Conclusion

The results of the analysis described in this chapter show the capacity of the LD notation to

express CSCL macro-scripts. The chapter discusses how the requirements can be supported by tools

and related specifications. Macro-scripts aim at structuring collaborative learning processes of

coarse-grained activities. Their requirements are shaped around the fact that they involve groups

and multi-roles characteristics.

These possibilities of LD to support the identified needs are tested and illustrated by means of

CLFPs and well-known full-fledged scripts. Although it is not possible to broadly affirm that any

CSCL script can be realised following the results of the analysis, these conclusions can be very

useful to implement scripts embracing similar manifestations of the studied requirements.

Recapitulating the different types of generalized requirements, we summarize now how they are

addressed by the notation itself and/or by other specifications and tools:

- The LD roles component and its related elements and attributes together with the joint use

of properties and conditions provide constructs to computationally represent several group

composition requirements. These requirements refer mainly to hierarchy of groups, group


104

size and dynamic group formation. The notation provides limited support for the formal

specification of the number of groups and group formation policies. However, these

requirements as well as an enhanced realization of the others can be supported by

administration tools (and also supporting tools such as grouping services or player utilities)

in combination with eventual exhaustive group composition specifications.

- Similarly, role distribution relies on the constructs offered by the roles component,

complemented with the coordination of role-parts, enabling that participants play the same

or different roles. In addition, supporting collaborative tools (group services) may define

specific roles implying different privileges when using the tools. Rotation of roles can be

realized by rotating activities or by using mechanisms provided by the players. The

distribution of resources is facilitated by the coordination of role-parts but also through the

possibility of referencing resources to different elements of LD such as activity-

descriptions or environments. The use of properties or supporting tools also provides

another means of resource distribution.

- Coordinating the flow of CL activities is feasible using the LD method and conditions. The

flow of artefacts between activities can be attained by employing properties, global-

elements and monitor services conveniently referenced by other LD elements. The

consistency of shared artefacts is ensured by jointly held properties. Moreover,

sophisticated floor control mechanisms can be described in the learning flow or achieved

by means of external supporting tools, whose description and interoperation with LD

should be covered by a new specification (or an extension of LD). In this sense, that or

another interoperability specification is needed to address the data flow between

supporting tools.

- Flexibility requirements are also tackled by both the LD notation and its implementation in

tools. The main attributes of roles that enable flexible group compositions are min-persons,

max-persons and create-new. Further flexibility is provided by the capabilities of LD to

support adaptation as well as the distinction between abstract descriptions (UoLs) and

specific instantiations (runs). This distinction affords new developments allowing the

introduction of slight modifications to runs in progress. The probability of reaching

success using scripts would be increased if engines handle the specifications of script

constraints that are allowed to be modified along the whole teaching and learning process

as well as specific user interface features related to particular scripts.

Concluding, computationally representing scripts using the LD interoperable notation provides

the following benefits. Firstly, they can be repetitively and automatically processed. As a

consequence, the scripts become resistant to technological changes. In addition, they can be reused


105

in different settings and with different participants. And, furthermore, they can be easily adjusted to

support other learning situations by using LD-compliant authoring tools. Chapter Six provide

further insight to the conclusions of this analysis by evaluating the creation and use of LD-

represented scripts in real situations.

Of course, LD-based scripts editors should hide the computer representational details laid out in

this chapter, if we actually want to foster the adoption of LD as the base machine-readable

modelling language of ready-to-run scripts. How design processes based on patterns (and

particularly on CLFPs) can undertake this challenge is the focus of next chapter.


106

CHAPTER FIVE

DESIGN PROCESS FOR THE

GENERATION OF IMS LD SCRIPTS

REUSING CLFPS

Assuming the contributions of the previous chapters as a starting point, this chapter tackles the objective of proposing a design process that facilitates the reuse of CLFPs (Collaborative Learning Flow Patterns, a particular type of CSCL scripting patterns) in the creation of CSCL macro-scripts computationally represented with LD. The aims of the design process include: achieving a satisfactory trade-off between particularizations of CLFPs so that the resulting LDs are contextualized according to particular CL situations and the loss of the meaningfulness captured in CLFPs; allowing teachers to focus on CSCL critical features that are involved in the elicitation of expected interaction processes; and not requiring high technical knowledge, particularly of LD. The design process is implemented in an authoring tool which proves its feasibility and enables its proper evaluation. The chapter also introduces a framework that conceptualizes different (existing and potentially yet-to-come) approaches that drive the creation of full-fledged UoLs by reusing different types of design solutions. This framework helps us to comprehensibly situate our proposal.

The design process for the generation of CSCL scripts reusing CLFPs is the central contribution of this dissertation, which has been published in (Hernández-Leo et al., 2005; Hernández-Leo et al, in pressa; Hernández-Leo et al., 2006e). The “create-by-reuse” framework in which the process is situated has been published in (Hernández-Leo et al., 2006a).

5.1 Introduction

How can the creation of potentially effective LD-represented CSCL macro-scripts be

facilitated? This is in short the main objective of this dissertation which is tackled in this chapter

benefiting from the contributions of Chapter Three and Chapter Four. The ultimate goal is to enable

participatory modes of designing in which teachers (interested in applying CSCL and without


108

advanced LD knowledge) decide on the behaviour and functionality of LMSs by designing their

own scripts according to the particular needs of their educational situations.

Chapter Three elaborates on how CSCL scripting patterns and their interrelations into pattern

languages enable design processes grounded in practice. From the different types of patterns that

can be used in the design of scripts, the patterns at the CL flow level (Collaborative Learning Flow

Patterns, CLFPs) are of special interest in the design of macro-scripts. CLFPs capture good

practices regarding the structure of coarse-grained activity flows and the involved groups. Thus,

CLFPs are the main patterns considered in the proposals of this chapter (as similarly done in

Chapter Four).

On the other hand, Chapter Four analyzes the capacity of the LD notation to express CSCL

macro-scripts. The conclusions of the analysis indicate that, though with minor limitations that can

be supported with existing and eventually-developed specifications and tools, LD can be

satisfactorily used to computationally represent the scripts. Having the scripts represented with LD

increases the opportunities for their re-use by enabling interoperability among compliant LMSs.

However the complex requirements that demand the descriptions of the scripts together with the

intricate LD computational constructs that enable their representations (cf. Chapter Four) makes

quite difficult for teachers the creation of their own LD scripts.

As advanced in Chapter Two, a promising solution to facilitate the creation of LD scripts relies

on providing authoring tools incorporating design processes that hide the LD notation and use

instead visualizations and concepts easy to understand and use by teachers. Our aim is that these

design processes help teachers to focus on the CSCL critical elements (e.g. learning objectives, task

type, level of pre-structuring, group size) of effective scripts. We exploit the incorporation of

patterns (CLFPs) in a design process to provide teachers with well-known design solutions. The

objective is that this design process achieves a satisfactory trade-off between particularizations of

CLFPs so that the resulting LDs are contextualized according to particular CL situations and the

loss of the meaningfulness captured in CLFPs. Moreover, we argue that this type of processes also

fosters the reuse of patterns more satisfactorily than their “simply” collection in repositories. Along

with the research methodology presented in Chapter One, a pattern-based design process is

proposed in this chapter and incorporated into a (high-level) authoring tool in order to illustrate its

feasibility and evaluate it in real situations with teachers and students (cf. Chapter Six)

It is important to point out that, according to the revision presented in section 2.4.1.2 of Chapter

Two, (at the time of proposing the contributions of this chapter) there is not any high-level (distant

from the specification) LD compliant authoring tools that are specialized in CL. However, there are

several initiatives working towards promoting the widespread adoption of LD that, in a similar

DESIGN PROCESS FOR THE GENERATION OF LD SCRIPTS REUSING CLFPS

109

spirit to our approach, advocate the use of design processes based on the reuse of complete or

incomplete learning design solutions at different levels of granularity.

Therefore, the chapter starts with section 5.2 which introduces a comparison framework for

conceptually analyzing and classifying reusable learning design solutions and processes that drive

the creation of ready-to-run UoLs. Then, section 5.3 concentrates on the role of CSCL scripting

patterns, and particularly CLFPs, in design processes. Consequently, section 5.4 proposes a CLFP-

based design process for the generation of LD scripts (i.e. collaborative UoLs) and exposes its

integration into an authoring tool (Collage). The creation of LD scripts using this authoring tool is

illustrated with several examples (also used in next chapter for the evaluation in real situations) in

section 5.5. A discussion section (5.6) briefly reflects on eventual solutions towards more general

approaches regarding the graphical visualizations and the language independence of script

computational representations. Finally, the conclusions of the chapter are included in section 5.7.

5.2 The “create-by-reuse” conceptual framework

In general, the adoption of LD by teachers in real educational practice greatly depends on the

provision of tools and processes capable of facilitating the creation of computer-interpretable UoLs

(Griffiths et al., 2005). These tools and processes should consider a broad range of types of teachers

with different pedagogical and technical backgrounds as well as diverse didactical contexts: types

of institutions and communities of practices.

As mentioned above, the main problem refers to the fact that technical formalism (XML) and

LD concepts are not familiar to the majority of the teachers. In this sense, the current trend in the

development of LD editors is to hide the LD details by using concepts (and their representations)

closer to the teachers’ concepts, what in several approaches is related to the idea of providing

teachers with specific reusable learning design solutions (Burgos & Griffiths, 2005). However, the

existing and yet-to-come approaches can be quite varied in that the reusable design solutions on

which they rely can be at different levels of granularity (an LD activity vs. the whole flow of

activities included in an LD) and completeness (a complete UoL vs. the bare bone structure of the

flow of the activities of the LD). Moreover, the diverse types of learning design solutions afford

different types of design processes for their reuse and customization (assembly vs. refinement

processes).

This section introduces a create-by-reuse framework that elucidates different approaches for the

creation of UoLs via the reuse of learning design solutions at different levels of granularity and

completeness. This framework is intended to provide criteria for comparing and classifying existing

and forthcoming proposals for creating UoLs, as well as their associated design processes based on


110

a certain level of reusability. In addition, the framework provides a “tool” for discussing the proper

level of reuse for user-friendly creation of UoLs according to teachers’ contexts and backgrounds.

5.2.1 Reuse of learning design solutions

Several proposals have been identified for creating UoLs by reusing pre-existing learning

design solutions at different levels of granularity and completeness. These two dimensions

(granularity and completeness) provide an interesting way of classifying and comparing some of

those relevant proposals (cf. Figure 5.1). Furthermore, this two-dimensional space provides a way

of grouping the existing and forthcoming proposals into four general (overlapping) sets:

- Exemplars are ready-to-run (complete) UoLs (LN4LD, 2005; Griffiths et al., 2005;

Griffiths, 2005). These UoL may embrace from one-activity session to a whole course (i.e.,

finer or coarser-grained exemplars). In fact, the final goal of any design process carried out

by a teacher (or learning designer) is obtaining an exemplar that fulfils the teaching-learning

requirements. In other words, an exemplar contains all the information required to be

enacted by an LD compliant LMS (Learning Management System).

- Templates are partly completed exemplars (Griffiths, 2005). There may be also templates at

different levels of granularity as well as at different degrees of completeness. Figure 4.2

shows, as an example for illustration, that an LD template without the specification of the

group size limits is more incomplete than the LD (the latter template with the specification

of the group size limits but still without the resources that are needed in order to achieve a

ready-to-run UoL). Accordingly, the LearningMapR initiative aims at enabling the selection

of templates and exemplars to be reused and customized as necessary or desired (Buzza,

Richards, Bean, Harrigan, & Carey, 2005). Though without considering compliance with

LD, the objectives of the AUTC project fit well with the presented orientation (AUTC,

2003; Hedberg, Oliver, Harper, Wills, & Agostinho, 2002): it seeks for the identification of

learning design exemplars considered as having the potential of fostering high quality

learning and could be redeveloped in more generic templates.

- UoL chunks are portions of exemplars. The granularity of the chunks may range from a

ready-to-use (complete) activity structure (including the activities, environments, resources)

to a learning object (fine grained). In contrast to exemplars, chunks are not “playable” on

their own. The “lego metaphor” is used by Berlanga et al. (2005) in order to explain their

approach to enable reusability and exchangeability of UoL chunks when supporting adaptive

learning design. A similar approach (but not LD compliant) is the proposal of Haake et al.,

(2007) who distinguish between atomic scripts (in the terminology of the “create-by-reuse”

framework it would be a chunk), which support a specific collaborative learning activity,


111

and composite scripts (exemplars), which support complex collaborative learning tasks

through sequences of atomic or composite scripts. - Building blocks or components are partly completed UoL chunks at different levels of

granularity and diverse degrees of completeness. Figure 4.2 includes as an example “an

abstraction of a pedagogic activity type”, which may be similar to the predefined activity

tools that LAMS (LAMS, 2006) offers to users as components that can be graphically

dragged and dropped to describe a sequence of activities. With the aim of dynamically

generating web course structures Giacomini-Pacurar, Trigano, & Alupoaie (2006) use a list

of LD building blocks (what they also name as primary pedagogical models or basic

models), which are associated according to inference rules stored in a knowledge base.

High level of granularity (fine grained)

Low level of granularity (coarse grained)

Incomplete

an incomplete

LD

Pedagogic activity

abstraction

a UoLan LD

a complete learning activity

Exemplars (Ready-to-run

UoLs of different

granularities)

Templates (Partly

completed exemplars)

Building blocks/

Compontents (Partly

completed UoL chunks)

UoL chunks (A UoL

portion of different

granularities)

Complete

Figure 5.1 Dimensions of the create-by-reuse framework: reusable learning design solutions at different level

of granularity and completeness

Nevertheless, the design processes for reusing the learning design solutions in order to create

UoLs (implicitly indicated in some descriptions of the reusable solutions) are even more important

than the reusable solutions themselves. Hence, further topics arise: What kind of design processes

can be applied? To which extent do the processes depend on the type of reusable solution?


112

5.2.2 Design processes for creating units of learning

Considering the previously defined reusable learning design solutions, this subsection presents

different types of design processes that enable the creation of full-fledged UoLs. The existence of

varied types of processes is caused by the different nature of the reusable solutions:

- Refinement: this process is needed to reduce the abstraction level (incompleteness) of

constituents by adding concrete information about numbers of participants, roles, activity

descriptions, resources, etc. This is the basic process to move from templates to constituents

that are closer to an automatically executable representation, which may take in several steps

of reducing abstraction.

- Assembly: this process is needed to reduce the granularity of a constituent by combining

several constituents together or integrating them into a coarser grained process structure.

This activity is especially suited for UoL chunks which are not “playable” on their own, but

have to be integrated into other structures to be operational. While the mere sequencing of

activities without dependencies between them is relatively unproblematic, more complex

learning processes, that require interrelations between artefacts flowing through several

activities or consistency of roles through phases, are more demanding. These relations have

been discussed with proposed solutions in Chapter Three of this dissertation and in (Harrer,

2006).

- Modification: this process may take place orthogonally to the other two. It usually reduces

neither abstraction nor incompleteness, but changes some information inside the constituent.

E.g. in exemplars the creation of a new UoL can be achieved by keeping the process

structure, while changing the concrete resources to move to another domain of learning.

Figure 5.2 shows these typical types of processes for creating complete UoLs. From right to left

a refinement process moving from abstract to less abstract constituents and from bottom to top an

assembly process, which creates a larger scope structure from fine grained constituents. A

modification usually would keep the position with respect to both abstraction and completeness.

The refinement and assembly design processes highlight the basic, stereotypical techniques to

move towards complete UoLs. In practice it is very well imaginable and – from the perspective of a

learning designer – highly desirable to have the option of mixing both approaches within one design

process.

Although learning objects are not “learning designs solutions” strictly speaking, they have been

considered in the framework as the finest grained chunks, which need to be assembled with other

components of different granularity (e.g. an activity building block) in order to reuse them for

creating a UoL. In this case, the result of the assembly is actually a refinement of the component:

the learning object (e.g. a document) completes the component (e.g. an activity building block).

“Refinement by assembly” can be thus understood as a type of mixed design processes.


113



Incomplete


UoLs of different

granularities)

Templates (Partly


Building blocks/

Compontents (Partly


UoL chunks (A UoL


granularities)

Complete Assembly process

Refinement process

Mixed process

Figure 5.2 Design processes for creating UoLs by assembling and refining learning design solutions

To show the usefulness of our classification of design processes, we apply this

conceptualization to a representative tool, namely LAMS (although LAMS models are not

completely compatible with LD), based on the approach of creating by reusing learning design

solutions. The typical design process supported by LAMS is the assembly of LAMS building blocks

(activity tools) into a process sequence by graphical linking of the activity tools. This type of design

process can be considered the “vertical” assembly design process of Figure 5.2. The result of the

assemblage (the process sequence) is a template that needs to be refined (with task descriptions,

etc.) into full-fledged units. This second step reflects a “horizontal” design process that together

with the previous assembly step forms a “mixed process” can be seen as an instance of the angular

design process in Figure 5.2.

Other examples cited in the previous subsection make use of “pure” design processes, for

example (Haake et al., 2007) propose assembling atomic scripts (chunks or exemplars) or/and

composite scripts (exemplars) into new composite scripts (larger chunks or exemplars, depending

on whether they are “playable” on their own). In contrast, the proposal of Buzza et al. (2005) will


114

enable pure refinement processes, if the user selects to reuse a template, or pure modification

processes, if an exemplar is selected.

To sum up, the create-by-reuse framework distinguishes reusable design solutions according to

their level of granularity and completeness and illustrates the basic types of design processes and

their combinations, used to integrate the reusable solutions. Therefore it provides a conceptual

frame to classify approaches with provide design processes based on reusable solutions to facilitate

the creation of UoLs. In addition, the framework provides an instrument that can be used when

discussing several related issues, such as: What is the proper level of reuse for teacher-friendly

creation depending on the institution, community, etc? Which types of learning design solutions are

potentially more reusable, the coarser and/or the more incomplete? How can a proper understanding

of the solutions before their actual reuse be facilitated?

The latter question suggests that the different possible approaches devoted to the selection the

design solutions so that they are appropriate for a particular learning situation deserves a new

dimension in the framework. Though for situating the proposal of this dissertation the current

version of the framework is sufficiently useful, in fact several additional interesting dimensions may

be considered in a more complete framework. Some of them are discussed in next subsection.

5.2.3 Discussion: other potential dimensions

Apart from the current dimensions of the create-by-reuse framework, it is possible to envisage

at least four topics that can be translated into new dimensions: approaches for the selection of

reusable design solutions, types of notations or representations used to present the solutions to the

teachers, languages for their computational representation (not only LD), and types of

pedagogically-based formulations behind the reusable solutions.

Regarding the selection of reusable solutions, different approaches are used: from the use of

metadata compliant with specifications (e.g. LOM or IMS Resource Meta-data specification,

(Duval, 2001; IMS, 2002)) or other types of categories to the use of ontologies (Knight, Grasevic, &

Richards, 2006). Being able to understand the learning design solutions and decide whether they

can be applied in the actual situation that the teacher is facing is crucial in the analysis stage

(previous to the proper design) (Buzza, Bean, Harrigan, & Carey. T., 2004; Griffiths et al., 2005).

For example, Buzza et al. (2005) propose using two types of categories for the selection of LD

templates and exemplars: categories for cognitive analysis based on Bloom’s taxonomy (Bloom &

Krathwohl, 1984) (what the designer wants the learner to know and do) and categories related to

instructional challenges (e.g. lack of student motivation to engage with particular topics).

Moreover, the way of presenting teachers the reusable elements, using from textual to visual

notations (Botturi et al., 2006; Botturi & Stubbs, in press; Waters et al., 2004), plays an important


115

role when promoting the understanding of the design solutions and the intuitiveness for their

customization. For instance, the reusable activity tools of LAMS are shown as graphical

components that can be dragged and dropped.

On the other hand, the computational language that we consider in the framework is LD,

however we also point out some approaches that are not LD compliant (e.g. (LAMS, 2006; Haake

et al., 2007)). Providing language independence of learning designs would foster even more their

reusability (Dodero, Tattersall, Burgos, & Koper, 2007). This independence envisages the need of

emergent approaches for creating learning designs when elements from more than one specification,

formalism or model have to be combined in a single UoL, or they have to be transformed before

being delivered to a specific non LD compliant LMS.

Finally, there can be different types of pedagogically-based formulations behind the reusable

solutions. These formulations, which might be related with the approaches used for selecting the

solutions, are typically significant to teachers of particular communities of practices.

One of the approaches refers to using educational taxonomies, such as the taxonomy of learning

activities used in (Conole & Fill, 2005). Griffiths et al. (2005) also point out other candidates

taxonomies to be used as a guide when creating UoLs: the above cited Bloom’s taxonomy (Bloom

& Krathwohl, 1984), the classification of learning activities proposed by Shuell (1992) or the

learning events developed by Leclerc & Poumay (2005). Frameworks for the description of

pedagogically specific LDs can be also useful in this context. The framework for the specification

of collaboration scripts proposed in (Kobbe et al., submitted) is an example. A different approach

advocates the use of primitives, i.e. events that do not necessarily embody a particular pedagogical

view of learning and teaching but which reflect the real situations in the classrooms. Examples of

primitives are “discuss this text” or “research this topic on the web” (Griffiths et al., 2005; Casey et

al., in press).

All these approaches are interesting as ways of providing pedagogically sound formulations that

can be used in the creation of UoL. However, it is not clear to which extent they really provide

reusable solutions beyond mere categorizations. In this sense we argue that pedagogical design

patterns, besides providing a conceptual common ground, represent a structured way of capturing

and communicating educational expertise (cf. section 2.3). In this dissertation we focus on patterns

as the formulations behind reusable design solutions addressing recurrent problems in certain

educational contexts. In our case, these contexts are those that require of CSCL scripting solutions.

5.3 The role of CSCL scripting patterns in design processes

As Chapter Three exposes in detail, a “CSCL scripting pattern” describes a common problem

and its corresponding broadly-accepted solution which can be used repeatedly in the design of


116

CSCL scripts. In particular, that chapter proposes a conceptual model for describing CSCL scripting

pattern languages. The model describes the different types of patterns and relationships between

them that can be used for generating CSCL scripts. The types of patterns are described in an

aggregation model that differentiates the patterns according to their granularity: CL flows which

comprise activities that are supported by resources (tools and material). Roles and common CL

mechanisms are modelled as elements that are integral parts of flows, activities and resources. The

patterns associated to the different elements of the model can be related with connecting rules

indicating that a pattern completes another pattern (by refining the principles of the completed

pattern) or that a pattern complements a second pattern (forming a new larger whole). Patterns at

the same level may also complement or/and complete each other, may represent alternative patterns

(mutually exclusive) and specialize other patterns. The connections between the patterns provide

guidance for their meaningful application and prevent bringing together patterns that do not make

sense from the pedagogical perspective.

While subsection 3.3.3 of Chapter Three indicates general guidelines for applying the PLs that

can be described with the proposed conceptual model, the research question in this chapter focuses

on: what is the role of patterns in design processes for the creation of LD-represented scripts?

5.3.1 Assistant vs. templates

Most of the approaches related to e-learning patterns propose to collect the patterns in

repositories so that interested people can go to the repository, read the patterns and reuse the design

experience projected into them. Another approach consists in explicitly incorporating the patterns in

authoring tools in such a way that they provide advice and suggestion along the design process

(McAndrew, Goodyear, & Dalziel, 2005).

In the case of the CSCL scripting patterns, both approaches are valuable. However, as

mentioned above, incorporating patterns in design processes (to be implemented in authoring tools)

potentially facilitates their reuse and the design of experience-founded scripts. The incorporation of

patterns in design processes can be accomplished in two (complementary) ways so that the patterns

act as assistants or as templates:

- Pattern-based assistant: a context-aware advising mechanism based on the knowledge

formulated in the patterns. It is possible to imagine this type of assistant similar to, for

example, the animated Microsoft Office Assistant which becomes visible whenever the

system that detects the user could benefit from its advice.

- Pattern-based template or building block: a ready-made skeleton based on the knowledge

formulated in a pattern that can be refined (and probably assembled) to create full-fledged

scripts (cf. subsection 5.2.1). This idea may remind readers, for example, of Web page


117

templates (e.g. for Adobe Dreamweaver or Microsoft Front Page) that enable the easy

production of Web pages by separating their presentation from the content.

Of course, not every pattern that can be used for designing CSCL scripts is suitable of being

represented using a computer-interpretable notation so that it is a template or building block for

creating LD scripts (“an incomplete collaborative UoL or UoL chunk”). Depending on the nature of

the different types of scripting patterns they can be implemented as assistant or as templates (or

building blocks). We discuss this aspect for the different types of patterns considered in our model

for CSCL scripting PL:

- CL flow patterns. When the degree of specialization of the pattern is such that its solution

offers a learning flow (e.g. JIGSAW, PYRAMID, TPS, BRAINSTORMING, SIMULATION and

TAPPS of Appendix A), this pattern is suitable of being provided as a template. On the

contrary, if the pattern offers abstract advice, for example, to enrich the learning flow (cf.

ENRICHING THE LEARNING PROCESS PATTERN of Appendix A), it may be implemented

as an assistant.

- Activity patterns. In this case the problem is quite similar to the collaborative learning flow

level. That is, the generality degree of the patterns DISCUSSION GROUPS (Pattern 2.2 of

Appendix A) or THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5)

does not invite to formalize them as templates but to implement them as advisors. On the

other hand, the specialization of DISCUSSION GROUPS which may be also formulated in

patterns such as the ones that may result from capturing the essence of the two scripts for

argumentative knowledge construction proposed in (Weinberger et al., 2005) could be

represented computationally and implemented as building blocks (according to the create-

by-reuse framework proposed in section 5.2).

- Resource patterns. An approach for the formalization of patterns at the resource level

concerning learning objects has been already proposed in (Jones, 2004). Jones implements

the patterns using XML representations and proposes to incorporate them into authoring

tools for learning objects. There is not any proposal (to the best of our knowledge) to

formalize patterns for tools (also at the resource level). However, it is not clear to which

extent such formalization (beyond the aspects directly related to their “configuration”, cf.

section 4.5.2 of Chapter Four) is necessary. At least as far as the design of scripts (we are not

considering the design of tools) is concerned, the designer just needs to select the tools that

will support the activities. In this line, semantic search of tools using ontologies is being

researched by Vega-Gorgojo et al. (2006). Therefore, patterns at this level (e.g.

MANAGEMENT OF ON-LINE QUESTIONNAIRES, Pattern 3.2) can be implemented as

advisors that can act as mediators between the search system and the user.


118

- Roles and common CL mechanisms patterns. Again, depending on the degree of

generality-specialization of the patterns they can be represented as reusable building blocks

or as assistants. CONTROLLED GROUP FORMATION (Pattern 4.3) is so general that it

should be implemented as an assistant. In contrast, if we formulate a pattern suggesting a

particular group formation policy for assembling heterogeneous group under certain

conditions of a problem and a context, the pattern could be offered as a building block.

As indicated in the introduction of this chapter, the patterns at the CL flow level (CLFPs) are of

special interest in the design of macro-scripts and they are therefore the type of patterns considered

in the actual design process that is proposed in this dissertation. In particular, we also focus the

problem on the CLFPs whose solutions are suitable of being offered as templates, i.e. from Pattern

1.1 to Pattern 1.6 of Appendix A. Of course extending our proposal with CLFP-based assistants and

the other types of CSCL scripting patterns deserves further work which is outside of the scope of

this dissertation. Nevertheless, the orientations presented in this dissertation represent a starting

point towards this ambitious but interesting goal.

Having analyzed the global problem, we focus in next subsection on the implementation of

CLFPs as LD templates. Due to the nature of this type of patterns (the solution is a “learning flow”),

they can be computationally represented using LD (cf. Chapter Two).

5.3.2 Implementing CLFPs as refinable LD templates

The approach of providing CLFPs as script templates could be accomplished with any EML that

enables the computational representation of the scripts. As analyzed in Chapter Four, expressing the

requirements of macro-scripts (and therefore of CLFPs) using the LD interoperable specification is

feasible. The results of Chapter Four as far as the main characteristics of CLFPs are concerned can

be summarized as follows. LD enables the design of processes that include several roles (LD

element), each of which can be played by several persons. A collaborative learning experience can

be described by associating multiple persons and/or multiple roles to the same learning activity.

Furthermore, LD enables their activities to be specified in coordinated learning flows.

On the other hand, as continually discussed along this dissertation, creating successful designs

of scripts and, particularly, LD represented scripts from scratch is a complex task from both

pedagogical (cf. Chapter Two and Chapter Three) and technical (cf. Chapter Four) points of view.

Since CLFPs formulate group practices in structuring macro-scripts that can be applied to any

discipline (or content) and can be computationally represented with LD, they can be provided as

templates (i.e. partly completed UoLs) that needs to be refined into ready-to-run UoLs (scripts). The

resulting UoLs are potentially effective since the LDs (packed in the UoLs) are based on CLFPs.

That is, the roles, the activities and the learning flow of a CLFP-based LD are determined in the


119

particularization of the CLFP; and the CLFP-based UoL consists of the previous CLFP-based LD

and a set of particular resources that depend on a concrete learning situation.

5.3.2.1 From CLFPs to UoLs The schema of the process that can be followed in order to achieve a UoL based on a CLFP is

illustrated in Figure 5.3. This is the rough approach from the LD notation perspective that is

considered in the design process proposed in section 5.4. The approach considers three stages. The

first stage is the computational representation of a CLFP using LD, that is to say, the edition of an

LD-compliant XML document that describes the CLFP. This XML document is the core of the LD

template. Since an LD document that formalizes a CLFP is an incomplete LD, the second stage

involves the particularization of the preceding document, so as to detail all the elements of a

complete LD. When the actual resources that are to be used during the enactment of the CLFP-

based LD are determined and packaged or referenced within a content package (IMS, 2004a), a

CLFP-based UoL is achieved. Next subsection illustrates this schema by means of an example.

Figure 5.3 From a CLFP to a UoL: scheme of the process for obtaining CLFP-based UoLs

5.3.2.2 Example: UoL based on a JIGSAW LD template The context of this example is the same Computer Architecture course described in Chapter

Three (subsection 3.4.1). The whole course is defined as a project, which in turn is divided in three

subprojects, whose objective is the design and evaluation of computer systems. To enable distinct

perspectives of the main subjects within the classroom, five clients are defined, giving answer to

different market sectors and system requirements. Each pair of students is assigned one out of the

five clients for the whole course.

The scenario considered here concerns the first subproject, in which students get to know the

client, model the customer’s presumed computational load by mixing standard benchmarks, test real


120

machines using the benchmarks, and finally make a recommendation to their client. This subproject

pursues that students learn how to use benchmarks, and get a quantitative impression on a few real

machines (with different CPUs, memories, etc.) as well as they develop skills related to interpreting

and selecting information, arguing, and taking compromise solutions are promoted. The learning

process proposed to the students is briefly outlined next.

For the first activity, students should study customer needs while the educator should in turn

play the role of the client in order to clarify customer needs. Then, students model the

computational load of the customer. In the following activity, students distribute four groups of

different real machines among them, so that each student benchmarks a group of machines.

Subsequently, students benchmark those machines that have been assigned to them, collecting the

results and studying the documentation of the benchmarks and the machines.

Next, JIGSAW pattern (Pattern 1.1) is applied. Students who have benchmarked the same

machine debate the suitability of such machine for their customer according to benchmark results.

This activity will be supported by a (synchronous) chat tool. Finally, students have to debate the

results with other members of their group that have benchmarked different machines. As a result,

each group should generate a technical report presenting and arguing the best solution for their

customer.

To create a UoL representing this scripted CL situation so that it can be interpreted by LMSs

(Bote-Lorenzo et al., 2004), a three-stage process described in the previous subsection can be

applied as illustrated in Table 5.1. Column 1 of this table illustrates the LD representation of the

JIGSAW CLFP. The learning flow this CLFP suggests is included in a play of the LD method: one

act of the play is devoted to the individual study of a subproblem, the discussion of the subproblem

by expert groups takes place in the following act, and a third act is devoted to the Jigsaw group

debate, in which participants are supposed to solve the whole problem.

Column 2 of Table 5.1 represents the teacher customization of the LD description of JIGSAW

for the depicted example: a particular JIGSAW-based LD. This LD specifies that, in this case, the

problem involves deciding which is the best machine for a specific customer and the subproblem

entails testing a group of the available real machines. Besides, it also states that expert groups

debate synchronously.

Once the teacher has determined the binding of this LD with actual resources (tools or

documents), the UoL is achieved. Column 3 of Table 5.1 shows how two specific resources are

referenced within the CLFP-based LD. One of them is a document that includes the guidelines

students have to follow during the discussion. The other one is a concrete chat tool that enables this

debate.


121

The CLFP suggests an individual study of a subproblem, this subproblem is discussed in group of experts, then a Jigsaw group meet to solve the problem

Particular resources used in a particular performance of the example scenario

Again, Table 5.1 reveals that directly using the LD notation to refine the templates may be not

easy for most teachers. We revise the conclusions of this subsection towards a user-friendly CLFP-

based design process for the creation of LD scripts next.

Table 5.1 Excerpt showing the process for obtaining the JIGSAW-based UoL. A sample definition of an activity and an environment can be found under <learning-activity> and <environment> tags respectively.

The description of learning flow is shown under <method> tag

1. LD representation of JIGSAW CLFP

2. A JIGSAW-based LD (The teacher customizes the previous

description of the CLFP for the example situation)

3. A JIGSAW-based UoL (The binding of the previous LD for the

example situation with concrete resources)

<learning-design identifier="CLFP-Jigsaw" uri="" level="A"> <components> … <learning-activity identifier="LA-discuss-experts"> <environment-ref ref="E-discuss-experts"/> <activity-description> <item identifierref=""/> </activity-description> </learning-activity> … <environment identifier="E-discuss-experts"> </environment> … </components> <method> <play identifier="PLAY-CLFP-Jigsaw"> … <act> <role-part identifier="RP-individual-work"> <role-ref ref="R-expertLearner"/> <learning-activity-ref ref="LA-subproblem-study"/> </role-part> <role-part identifier="RP-expert-groups"> <role-ref ref="R-expertLearner"/> <learning-activity-ref ref="LA-discuss-experts"/> </role-part> <role-part identifier="RP-jigsaw-groups"> <role-ref ref="R-jigsawLearner"/> <learning-activity-ref ref="LA-problem-resolution"/> </role-part> </play> </method> </learning-design>

<learning-design identifier="CLFP-Jigsaw" uri="" level="A"> <components> … <learning-activity identifier="LA-discuss-experts"> <environment-ref ref="E-discuss-experts"/> <activity-description> <item identifierref=""/> </activity-description> </learning-activity> … <environment identifier="E-discuss-experts"> <service identifier="S-discuss-experts"> <conference conference-type=”syncronous”> <participant role-ref=”R-expertLearner”/> <item identifierref=””/> </conference> </service> </environment> … </components> <method> <play identifier="PLAY-CLFP-Jigsaw"> … <act> <role-part identifier="RP-individual-work"> <role-ref ref="R-expertLearner"/> <learning-activity-ref ref="LA-benchmarking"/> </role-part> <role-part identifier="RP-expert-groups"> <role-ref ref="R-expertLearner"/> <learning-activity-ref ref="LA-discuss-experts"/> </role-part> <role-part identifier="RP-jigsaw-groups"> <role-ref ref="R-jigsawLearner"/> <learning-activity-ref ref="LA-choose-machine"/> </role-part> </play> </method> </learning-design>

<imscp:manifest …> <imscp:organizations> <imsld:learning-design identifier="CLFP-Jigsaw" uri="" level="A"> <imsld:components> … <imsld:learning-activity identifier="LA-discuss-experts"> <imsld:environment-ref ref="E-discuss-experts"/> <imsld:activity-description> <imsld:item identifierref="RES-discuss-guideless"/> </imsld:activity-description> </imsld:learning-activity> … <imsld:environment identifier="E-discuss-experts"> <imsld:service identifier="S-discuss-experts"> <imsld:conference conference-type=”syncronous”> <imsld:participant role-ref=”R-expertLearner”/> <imsld:item identifierref=”RES-GSIC-chat”/> </imsld:conference> </imsld:service> </imsld:environment> … </imsld:components> <imsld:method> <imsld:play identifier="PLAY-CLFP-Jigsaw"> … <imsld:act> <imsld:role-part identifier="RP-individual-work"> <imsld:role-ref ref="R-expertLearner"/> <imsld:learning-activity-ref ref="LA-benchmarking"/> </imsld:role-part> <imsld:role-part identifier="RP-expert-groups"> <imsld:role-ref ref="R-expertLearner"/> <imsld:learning-activity-ref ref="LA-discuss-experts"/> </imsld:role-part> <imsld:role-part identifier="RP-jigsaw-groups"> <imsld:role-ref ref="R-jigsawLearner"/> <imsld:learning-activity-ref ref="LA-choose-machine"/> </imsld:role-part> </imsld:play> </imsld:method> </imsld:learning-design> </imscp:organizations> <imscp:resources> <imscp:resource identifier=”RES-discuss-guidelines”/>  <imscp:resource identifier=”RES-GSIC-chat”/>  … </imscp:resources> </imscp:manifest>

5.3.2.3 Discussion: towards a user-friendly CLFP-based design process CLFPs and LD have similar purposes in the sense that both aim at describing learning

processes. However, CLFPs represent good practices in natural language and LD is a computational

language that does not help users to conceptualize the pedagogical features of effective units.

Computationally representing CLFPs as LD templates provides the following advantages:

- Software tools can automatically process CLFPs. This facilitates the incorporation of design

techniques in CSCL systems.

- CLFPs can be easily reused and refined into particular CSCL scripts represented with LD.

- The structure of learning situations is dissociated from the learning resources (content,

tools), among other details such as the description of the activities and the particularities of

the roles. Therefore, resources can be reused within different scenarios structures (LDs or

In the example, which requires a synchronous chat, the subproblem is to test a group of the available real machines using benchmarks. In this case the resolution of the problem is to decide which the best machine is for a specific customer


122

scripts). In the same way, LDs or CLFPs can be reused when particularized with different

resources that depend on the concrete learning situation.

- Since the LD specification provides a standard language, the reuse and integration of CLFPs

or CLFP-based LDs in different systems are facilitated.

On the other hand, when formalizing a CLFP, it is possible to achieve one or many templates

that reflect such pattern. For example, there are two possible ways to model groups using LD. The

types of groups indicated by the patterns can be represented with the LD role and the attribute

create-new set to “allowed” (cf. Chapter Four), so that the actual numbers of groups (occurrences of

the type of role) are determined at instantiation time (after refining the pattern-based template but

before running the resulting script). An alternative approach is to determine the number of groups at

authoring time. In this case, authoring tools must incorporate a mechanism that automatically

creates and includes in the template a new role for each group (of a type of group) together with the

rest of the necessary LD elements related to the activities to be accomplished for this type of group.

Similarly, some specific characteristics of each CLFP require specialized attention, for example

PYRAMID (Pattern 1.2) can be applied using as many levels of the Pyramid as necessary. This

needs to be also supported by authoring tools implementing the design processes based on CLFPs in

such a way that its use is intuitive.

In this sense, we also propose the use the diagrams of CLFP solutions (cf. Appendix A) as

visual representations of refinable LD templates with the aim of adding intuitiveness (Waters et al.,

2004) and familiarity (Harrer et al., 2006) to the LD notation system. This intuitive intelligibility

results from the visual similarity to the mental images and ideas that the potential users have or

build regarding the patterns. The particularities of our CLFP-based design process proposal are

detailed in next section.

5.4 A CLFP-based design process for the generation of LD scripts

This section is devoted to propose a pattern-based design process for the generation of LD

scripts whose main goals are:

- Achieve a satisfactory trade-off between the reuse of the CLFPs (keeping the intrinsic

constraints that potentially lead to learning outcomes) and the creation of meaningful scripts

contextualized according to the needs of a particular situation.

- Enable that a conceptualization of the expected interactions (at the macro level) is made

explicit in advance, what requires a focus on CSCL critical elements.

- Make easier for teachers without advance knowledge of LD but with interest in applying

scripted CL by means of LMSs (or similar learning environments) the creation of LD-

compliant scripts.


123

Therefore, we start by clarifying the scope of the design process within the outline of the global

system that enables the implementation of LD.

5.4.1 Scope: support the creation of LDs

The Best Practice and Implementation Guide included in the LD specification (IMS, 2003a)

details the different conceptual components of a system implementing LD. Figure 5.4 summarizes

these modules, emphasizing the authoring problem and illustrating what functions are covered by

our design process. Namely, it is not devoted to the enactment problem: instantiating LDs, binding

participants to roles, interpreting an LD, etc. However, being implemented in an authoring tool, it

aims at supporting the creation of LDs. As aforementioned, a UoL is a content package (IMS,

2004a) including an LD and a set of physical resources (content and tools) or their location. The

idea is that the resources that contain learning objectives, prerequisites, descriptions of activities

and information about roles are also created during the design process. The creation of other types

of resources (content or tools supporting the activities) is outside the scope of the design process.

Authoring

IMS LD authoring environment

Production

Instantiating IMS LD

Delivery

Executing/interpreting IMD LD

Creation of IMS LD documents

Creation / selection of resources

Text of learning objectives, prerequisites, description of activities and roles’ information

Other types of content or tools needed to support the activities, etc. (pictures, web pages, videos, conceptual map editor, services like e-mail, asynchronous or synchronous groupware, etc.)

Validation and publication of the IMS LD document Population of a Learning Design instance (creation of a run or community of users), assigning actual users to the instance of the LD…

Actual live interpretation of the Learning Design (it depends on the technical architecture)…

Figure 5.4 General modules of a system implementing LD

Designer’s Guide, which is also included in LD specification (IMS, 2003a), proposes the basic

procedures for creating a UoL. (Sloep et al., 2005) details these procedures according mainly to

three phases:

- The first phase involves the analysis of a specific educational problem, whose result is a

narrative description of the educational situation.

- Then, the narrative is translated into a UML (Unified Modelling Language) activity diagram

(Arlow & Neustadt, 2001) in the design phase.


124

- The diagram is the basis for the XML instance IMS LD compliant document. In the forth

phase the resources are developed (if necessary) and added to the design. Thus, a UoL is

created.

These basic design procedures are useful depending on the type of audience that creates the

UoLs. That is, different institutions, types of users (learning designer, teachers) or communities of

practice may require different types of design processes. These processes are specially valuable if

they provide a methodology for the analysis phase and enable teachers to understand and edit the

UoLs (Griffiths et al., 2005). The use of CLFP-based templates provides a structure for the first

stage of analysis and creation of LD scripts. However, it is not realistic to consider that scripts are

always structured as indicated by a unique CLFP. Like other types of patterns, CLFPs can be used

collectively in order to define richer CL flows (cf. Chapter Three).

5.4.2 Combinations and concatenations of CLFP-based templates

According to Chapter Three, CLFPs can complete and complement each other generating new

structures of scripts. That is to say, CLFP-based templates can be combined in such a way that a

phase of the learning flow is refined (completed) by replacing the phase with another template. Or

they can be concatenated so that some phases of a script are structured according to one CLFP and

other (separated but consecutive) according to another CLFP (that can be eventually the same). In

this case the flow suggested by the former pattern is complemented with the latter CLFP (the limits

of the whole resulting from this concatenation of patterns are determined by the two patterns).

Figure 5.5 illustrates these two ways of forming CLFP hierarchies. The “Expert Group” phase

of JIGSAW (Pattern 1.1) can be structured following PYRAMID (Pattern 1.2).The result is a

combination of two CLFPs. In contrast, the JIGSAW can be concatenated with PYRAMID by

preceding (as it is shown in Figure 5.5) it in the learning flow of the script.

Now, how does this approach of combining and concatenating CLFPs fit within the create-by-

reuse framework?


125

+

(a) (b)

The Expert Group phase of the JIGSAW is structured according to PYRAMID

The first set of phases of a script

are structured according to

JIGSAW while the second set of phases

are structured according to

PYRAMID

Figure 5.5 CLFPs hierarchies (a) combined CLFPs: jigsaw is completed with PYRAMID; (b) concatenated CLFPs: jigsaw is complemented with pyramid

5.4.3 The design process in the create-by-reuse framework

We define template in section 5.2 as a partly completed UoL. The level of granularity and

degrees of completeness of templates is varied. Figure 5.6 shows that a template which represents a

CLFP is more incomplete than the template that results from particularizing the pattern into an LD

(actual description of activities, group-size limits, etc. but still without the resources that are needed

in order to achieve a ready-to-run UoL).

Consequently refinement steps are necessary to create a complete UoL. These refinement steps

coincide with the stages depicted in subsection 5.3.2 (cf. Figure 5.3 and Table 5.1). The first

refinement step produces an LD, while the second results in a UoL, ready to be played in an LD

engine. This can be seen as a pure “horizontal” design process with refinement steps.



Incomplete

CLFP-based template

UoL LD refining the template


UoLs of different

granularities)

Templates (Partly


Building blocks/

Compontents(Partly


UoL chunks (A UoL


granularities)

Complete

Refinement

Figure 5.6 CLFP-based design process for the creation of LD scripts within the create-by-reuse framework:

refinement process


126

Besides the pure refinement process, the potential of the CLFP-based design process is

“mixed”. Combining or concatenating CLFP-based templates involve their assemblage. Figure 5.7

illustrates it. When two templates are concatenated, they are actually assembled forming a coarser

template, which in turn needs to be refined. On the other hand, a combination of CLFPs, which is

also an assemblage of reusable design solutions, results in a refinement of the pattern that is

completed. Both cases imply mixed design processes, having the second case a “refinement by

assembling” step.

(a) (b)



Incomplete

CLFP-based template

UoL LD refining the concatenationExemplars

(Ready-to-run UoLs of different

granularities)

Templates (Partly


Building blocks/

Compontents(Partly


UoL chunks (A UoL


granularities)

Complete

Concatenation of two CLFP-

based templates

Assembly

Refinement



Incomplete

CLFP-based template

UoL

LD refining the combination


UoLs of different

granularities)

Templates (Partly


Building blocks/

Compontents(Partly


UoL chunks (A UoL


granularities)

Complete

Combination of two CLFP-

based templates

Refinement by assembling

Refinement

Figure 5.7 CLFP-based design process for the creation of LD scripts within the create-by-reuse framework:

mixed process (a) combination of CLFPs, (b) concatenation of CLFPs

To this point several important elements of our design process for creating LD scripts are

presented: it uses the “template” as the type of reusable design solution; the pedagogically-based

formulations behind the templates are CLFPs; it enables processes of type “refinement” and

“mixed” (assembly + refinement, refinement by assembling + refinement). Next section tackles

how CLFP-based templates are selected and refined so that it enables teachers to focus on CSCL

critical elements.

5.4.4 Focus on CSCL critical elements: selecting the templates and authoring the scripts

One of the main goals of the proposed design process is to consider the CSCL critical elements

that affect interaction, namely: learning objectives, task type, level of pre-structuring, group size

and computer support. In this sense, our design process is a particularization of the framework

introduced in (Strijbos et al., 2004), which proposes a process-oriented methodology for the design


127

of CSCL settings in view of these critical elements. The methodology implies that a

conceptualization of the expected interaction is made explicit in advance and consists of six steps:

1. Determine which type of learning objectives should be specified.

2. Determine the expected interactions according to the specified objectives. It is related to the

co-ordination of activities and the types of interaction promoted by the different types of

activities (e.g. discussion).

3. Select task-types with respect to the learning objective and expected interaction. For

example, if students have to solve a complex and ambiguous problem with no clear

solution.

4. Determine how much structure is necessary to accomplish the learning objectives, expected

interactions and task-types (e.g. privileged roles within an activity).

5. Determine which group size is best suited with respect to learning objective, expected

interaction, task type and level of pre-structuring.

6. Determine how computer support is best used to sustain learning and expected interaction:

face-to-face or computer mediated (synchronous or asynchronous).

Figure 5.8 shows in detail the design process that we propose. It is not strictly sequential. It

aims at providing guidance but without directing the user through a rigid wizard-style set of steps.

The design process considers two general phases, namely selecting a CLFP-based template and

authoring the CLFP-based script.

The different design tasks included in the design process can be easily mapped to the steps

indicated in the methodology proposed by Strijbos. Step 1 regarding learning objectives is partially

performed in task a and completed in task c. Steps 2, 3 and 4 correspond to a large extent to the

selection of a CLFP (mainly task a). Note that tasks a and b are repeated if the collaborative

learning flow is structured according to a hierarchy of CLFPs (task d). Tasks e and h embody also

step 4 as far as the structure of the interaction processes within activities is concerned. The

description of an activity and the tool that supports it can represent a certain level of activity pre-

structuring (e. g. a discussion activity supported by a simple chat vs. a chat with a structure dialogue

interface that allows different roles). Task e clearly refers to step 5 (group-size). While determining

the computer support (step 6) is accomplished in tasks f, g and h.


128

Selecting a CLFP-based template

a. Choose a CLFP depending on: The learning objectives proposed by the CLFP, the type of problem or task the CLFP is more suited to be applied and the complexity of the CLFP in terms of the collaborative learning experience needed.

b. Read the help about the chosen CLFP: Understand the learning flow structure (CLFP) on which the UoL will be based.

Authoring a CLFP-based UoL (script)

Identification and formulation of CL structuring techniques as patterns (CLPFs) and formalization using IMS-LD

f. Create or select resources (content and tools)

c. Determine the title, learning objectives and prerequisites of the LD

d. Specify the collaborative learning flow: The learning flow of the selected CLFP can be enriched by combining or concatenating CLFPs. Depending on the CLFP some aspects should be determined (e. g. levels of the Pyramid CLFP). Additional activities can be also inserted.

e. Define the description of activities, activity completion, the information about roles (including groups), group-size limits.

g. Determine and configure the resources needed to support the activities

h. Associate resources to activities

i. Package the LD into a UoL

Figure 5.8 CLFP-based design process for the creation of LD scripts: selecting the templates and authoring

the scripts

5.4.4.1 Selecting a CLFP-based template With the aim of facilitating the choice of CLFPs, the selection has been planned considering the

following premises:

- Potential Collage users may not explicitly know the collaborative techniques formulated in

the CLFPs.

- Users may not be familiar with pedagogical jargon. In this context it is more appropriate to

indicate the meaning of the psychological term. E.g.: positive interdependence means that

team members need each other to achieve a common goal.

- Teachers should be able to select a CLFP, so that the LD they create is adequate for their

educational situation. Moreover, they should find CLFPs addressing their needs even if they

do not know exactly the learning outcomes they want to promote.


129

Several approaches can be used to support the selection of the CLFP-based templates, from the

use of metadata to the use of ontologies (cf. subsection 5.2.3). As an initial approach, we propose

the use of the following metadata elements (see task a in Figure 5.8), whose related information is

captured in the patterns (mainly problem and forces fields, cf. Appendix A):

- Name of the CLFP on which the LD template is based.

- Learning objectives that the CLFP elicits. They are related to the gain of conceptual

knowledge, on one hand, and to the gain of meta-cognitive strategies. However, these

objectives have been formulated in a simplified way (so teachers may understand them

better) and classified in two types: attitudinal and procedural objectives. Attitudinal

objectives are related to motivational and emotional competencies, while procedural

objectives refer to the acquisition of skills. An example of attitudinal objective is “to

promote tolerance and respect” (BRAINSTORMING, Pattern 1.4). “To promote analytical

reasoning skills” (TAPPS, Pattern 1.6) is an example of a procedural objective. The fact that

a CLFP can be selected according to objectives fulfils the two first steps of the methodology

proposed by Strijbos et al. (2004).

- Types of problems that are best served with the CLFP. It is equivalent to the selection of

task type proposed in step 3 of Strijbos’ methodology. For instance, the task type of JIGSAW

(Pattern 1.1) is “complex problem that can be easily divided into sections or independent

sub-problems”.

- Complexity or risk in terms of collaborative learning experience needed. Depending on the

conditions in which the CLFP is to be applied or the experience in collaborative learning of

teachers and learners, some CLFPs are recommended above others; e.g. Jigsaw CLFP is

complex and is probably more appropriate for experienced participants (NISE, 1997).

When performing the selection of CLFPs, teachers should make use of these characteristics in a

priority order that depend on their situation. To further support the selection of the templates,

teachers can read extended information about the patterns (see task b in Figure 5.8). This

information includes:

- Overview: apart from the learning objectives, the type of problem and the complexity of the

CLFP, it contains the context in which the CLFP can be applied. It also explains the

collaborative learning flow proposed by the pattern.

- Diagram: a diagrammatic representation of the CLFP solution. The same visualization is

used in the authoring process as the graphical representation of the CLFP-based template.

- Use guidelines: indications and recommendations for particularization / customization,

instantiation and execution (or authoring, production and delivery according to Figure 5.4).

- Example: a sketch of a particular LD based on the CLFP.


130

With this information about CLFPs, teachers can be quite sure about the usefulness of a CLFP

for their particular needs. In other words, they can understand the CLFP before reusing it and,

consequently, they can be quite confident of the adequacy of the created UoL before running it in a

real situation.

5.4.4.2 Authoring a CLFP-based script Authoring a CLFP-based script is actually a process of particularizing and adapting a CLFP-

based template according to the requirements of a particular learning situation. This includes tasks

from c to i, as it is exposed in Figure 5.8. After determining the title, learning objectives (besides

the objectives of the selected template) and prerequisites of the LD, the user should specify the

collaborative learning flow. This task involves configuring some aspects that depends on the

selected CLFP. That is, the user has to decide several aspects of the activity flow or the way in

which the number of groups has been modelled (to be determined at design, using a mechanism that

creates the roles, or at instantiation time, setting create-new argument to “allowed”, cf. subsection

5.3.2.3). A good example is PYRAMID (Pattern 1.2): it specifies the organization of activities (in a

series of levels), and how participants will form groups and interact in each level, but the number of

levels is not fixed. To create an LD based on this CLFP, users must first determine how many levels

they want. Moreover, in this task the user decides if the flow of activities is enriched by combining

or concatenating CLFP-based templates (cf. subsection 5.4.2). In this case, a selection of a new

CLFP needs to be accomplished again. The result is a hierarchical structure of CLFPs presented as a

single template which is graphically represented according to the diagrams visualizing the solutions

of the patterns. Moreover, additional activities can be inserted, as a “didactic envelope”

(Dillenbourg et al., 2007) of the script (pre- or post- activities) or within a CLFP phase (typically

maintaining the group structure of the phase, they may be also individual or collective activities that

involve the whole class).

In this sense, we propose to hide several LD elements that are difficult to understand without

knowing the specification: activity-structure, method, play, act and role-part. These elements can

be substituted by the graphical representation, which symbolizes these concepts (directly related

with the learning flow). For example, the visual representation of JIGSAW (cf. for instance Figure

5.5) denotes that the suggested learning flow has three phases (modelled as three acts within a play

within the method). Each phase includes two role-parts one for the teacher (represented as a grey

circle), which indicates the activities or the activity structures of the teacher in this phase, and other

for the students, which specifies the activities or the activity structures assigned to the student in

this phase. Similarly, the environment LD element can be automatically generated (without the

need that the teacher knows its existence) when the user determines, configures and references the

resources that will be available in each activity (cf. task g and h of Figure 5.8).


131

Though more elaborated types of resources can be used to contain the learning objectives,

prerequisites, descriptions of activities and information about roles (cf. task e), we argue that a

simple text description whose edition is facilitated by the own authoring tool implementing our

proposed design process would provide fluency to the design.

All in all, we argue that this design process represents a trade off between generality and

unrestricted design options vs. good reuse and particularization of CLFPs (and hierarchies of

CLFPs) as well as easy authoring of collaborative LDs.

5.4.4.3 Discussion This design process represents an innovation to the phases recommended by LD specification in

creating a UoL. Selection of CLFPs supports the analysis phase in which a CSCL situation is

planned. It is necessary for teachers to know the CLFP-based templates that are available to plan a

feasible design. On the other hand, the need for understanding those learning flows promotes the

application of CL good practices, i.e. reuse of CLFPs in their own educational situations. The

design phase is highly simplified mainly thanks to the use of specific high-level collaborative

learning structures (CLFPs) instead of raw LD elements. Moreover, the templates are graphically

represented using the diagrams of the solutions captured by the CLFPs. That is, the UML diagram is

not necessary (each CLFP has an intuitive diagram that represents the learning flow) and the XML

code can be automatically generated. Furthermore, available information about each CLFP allows

teachers to understand and easily edit collaborative UoLs.

The proposed design process has been implemented in an authoring tool, named Collage, what

proves the feasibility of the approach and enables its evaluation.

5.4.5 Collage authoring tool: implementing the proposed design process

Collage (COLlaborative LeArning desiGn Editor) is an authoring tool that implements the

proposed CLFP-based design process for the creation of scripts computationally represented with

LD (the authoring tool, user manual and documented examples are available at

http://gsic.tel.uva.es/collage). According to the LD authoring tools framework (cf. subsection 2.4.1

of Chapter Two) devised by Griffiths et al. (2005), Collage is a high-level LD editor whose specific

purpose is CSCL (cf. Figure 5.9). It is LD level A (IMS, 2003b) compliant (Villasclaras-Fernández,

2005) and it is currently being extended towards level B compliance.


132

Specific purpose tools

General purpose tools

Distant form specification

Close to specification

LD Editor

COLLAGE

Figure 5.9 Two dimensions of LD tool design (Griffiths et al., 2005)

Collage is developed on top of the Reload Learning Design Editor (LDE), which is an Open

Source, general purpose, close-to-specification LD editor written in Java (University of Bolton,

2004; Wilson, 2005; Griffiths et al., 2005; Milligan et al., 2005). Reload LDE is considered the first

and currently “de facto” reference implementation of an LD editor intended primarily for

developers and early adopters to explore and think about the implications of the specification for

creating UoLs.

5.4.5.1 CLFP-based LD templates as plug-ins on top of Reload LD Editor Reload draws on a plug-in framework whose architecture is shown in Figure 5.10. Plug-ins can

offer different authoring capabilities (providing controllers and views that fit into the Presentation

Layer framework) in such a way that the validity and integrity of the created LDs is ensured

(accessing instance data model though the Learning Design Model Layer) (Wilson, 2005). This fact

enables the developers of authoring tools for different types of users or pedagogies to focus on the

user interface and avoid handling all the underplaying processes involved in manipulating LD

structures (Griffiths et al., 2005).

Collage is developed as a collection of Reload LDE 2.0 plug-ins. This version of Reload is

XML Schema driven: the final XML manifest for the UoL is created directly by a generic engine

using the specification schema to provide the rules whose elements can be added and deleted. This

is facilitated by reading, parsing and modelling the schema as a Document Object Model (DOM).

The current version of Reload LDE (which was not available when Collage was developed) adopts

the Eclipse Rich Client Platform (RCP) architecture and moves on from the Schema driven

approach. In the new version, resource file dependencies are managed separately and the LD

manifest is created only for export of the final UoL (Milligan et al., 2005).


133

The Editor Presentation Layer is a framework that allows plug-ins to install new items and views within the Editor user interface

The Learning Design Model Layer manages the internal representation of the Learning Design instance, and controls the modification of the

instance by Plug-Ins

<< framework >> Editor Presentation Layer

<< framework >> Learning Design Model Layer

Plug-In

provide controllers and views

access instance model

Figure 5.10 Plug-in framework for an LD editor (Wilson, 2005)

As mentioned above, some CLFPs impose the need of offering specific configurable

functionalities in the corresponding LD templates. These functionalities refer to configuring aspects

that depends on the patterns. Moreover, the graphical representation of each pattern solution, and

thus of each template, is unique. Therefore, Collage includes at least a plug-in associated to each

CLFP (commonalities are implemented in separate plug-ins that are associated to several patterns

(Villasclaras-Fernández, 2005)). Each plug-in is in charge of the graphical representation and the

specific functionalities of each CLFP.

To handle the specification of CLFPs hierarchies (cf. subsection 5.4.2), Collage makes use of

additional information stored in the Collage UoL package. This information is an XML document

that includes the structure of the learning flow in terms of CLFP hierarchies. The current version of

Collage enables combinations of CLFP-based templates. In this case, the plug-ins are not devoted to

the creation of CLFP hierarchies (this is responsibility of the editor). However, the CLFP-based

templates can indicate constraints regarding specific connections of patterns (e.g. it does not have

sense to replace a phase of TAPPS with another CLFP).

Next subsections detail the implementation of design process general phases, selection and

authoring, in Collage.


134

5.4.5.2 Selecting a CLFP-based templates in Collage Collage provides a repository with a pool of CLFPs. The available CLFPs at the moment are

JIGSAW, PYRAMID, SIMULATION, BRAINSTORMING, TPS and TAPPS (cf. Appendix A), but

more CLFPs can be added. A section utility is implemented in Collage according to the guidelines

indicated in subsection 5.4.4.1.

Figure 5.11 shows the interface of the CLFP selection utility, which allows the user to choose a

CLFP directly or select one or several characteristics of CLFPs. The list of CLFPs displayed in the

interface shows only the CLFPs that comply with the selected characteristics. These characteristics

may or may not univocally identify a CLFP and they are retrieved from CLFPs’ metadata.

Further information about each CLFP can be read by clicking on the title of a CLFP in the list

of the selection interface. This information is displayed in a window that provides a navigation tree

including four hyperlinks that show types of information indicated in subsection 5.4.4.1: namely

overview (cf. Figure 5.12 (a)), diagram (cf. Figure 5.12 (b)), use guidelines (cf. Figure 5.12 (c)) and

example (cf. Figure 5.12 (d)).

The more significant functionalities of Collage that facilitate authoring of a CLFP-based LD are

explained next.

Figure 5.11 Collage selection interface


135

(a)

(b)

(c)

(d)

Figure 5.12 Help information about the JIGSAW CLFP: (a) overview, (b) diagram, (c) use guidelines, (d) example

5.4.5.3 Authoring a CLFP-based UoL in Collage Collage also conforms to the authoring phase of the proposed design process (cf. subsection

5.4.4.2). Figure 5.13 (a) shows the Collage authoring interface regarding task c of Figure 5.8, where

the title, the objectives and prerequisites of the LD are defined. They can be directly written in the

text boxes provided by Collage.


136

(a)

(b)

(c)

(d)

(e) Figure 5.13 Collage authoring interface: (a) general tab, (b) resources tab, (c) collaborative learning flow tab, (d) form for the description of activities, association of resources to activities, (e) form for the description of

roles (groups) and group-size limits

Specifying the collaborative learning flow (task d) is facilitated with the interface shown in

Figure 5.13 (c). In this example the JIGAW-based template does not have any configurable aspect as

it has been modelled. However, the flow of activities indicated by the template can be enriched by

replacing phases of this CLFP with another CLFP-based template (that can be eventually the same).

In this way, the current version of Collage enables combinations of CLFPs. Although the actual


137

number of CLFPs implemented in Collage is not large, there is no theoretical limit on the

combinations of CLFPs that can be described.

The solution adopted in this first version of Collage for combining two CLFPs in a single

template is a simple approach that can be further elaborated in future versions. What can be

replaced by the whole learning flow of another CLFP-based template is a phase of the base template

that corresponds to an LD act (it is not possible to replace a single activity of an act). Besides, the

roles of the CLFP that completes the base template are included as sub-roles of the role

corresponding to the replaced phase.

As the JIGAW-based template, the templates based on BRAINSTORMING and TPS patterns do

not include configurable aspects. In contrast, those representing PYRAMID, SIMULATION and

TAPPS require that the plug-ins that implement them incorporate configuring utilities. Figure 5.14

shows how the Collage plug-in corresponding to PYRAMID includes a functionality that enables the

configuration of number of levels of the Pyramid.

(a) (b) Figure 5.14 Configuring the flow of the PYRAMID-based template in Collage: (a) determining the number of

levels in the Pyramid, (b) resulting template based on PYRAMID of 6 levels

Similarly, Figure 5.15 illustrates the specific configuring requirement of the SIMULATION-

based template. In this case the CLFP has been modelled in such a way that the designer must

specify at design time the number of groups (and characters participating in the simulation).

Therefore, the plug-in for this template implements a utility that enables the user to indicate it. It is

worth mentioning that this approach could have been also employed in the formalization of

JIGSAW as an LD template, instead of relying on the option of specifying the number of groups at

instantiation time. Both solutions represent two possibilities with advantages and limitations as

deeply discussed in Chapter Four. For example, the approach adopted for the JIGSAW template is

more flexible in the sense that it is not necessary to know in advance the concrete number of

students (and thus groups) that will attend in the actual enactment of the script. On the contrary,


138

defining at design time the concrete groups enables resource distribution (associating a different

environment to the activities to be performed by each group) without making use of external tools

in charge of accomplishing such distribution depending on the occurrence of the group (role

specified in the design) (cf. section 4.5 of Chapter Four for more details).

(a) (b)

Figure 5.15 Configuring the flow of the SIMULATION-based template in Collage: (a) determining the number and name of characters (roles) and simulation (small) groups that will participate in the Simulation, (b)

resulting template based on simulation with 2 types of characters and 2 simulation groups

TAPPS CLFP (Pattern 1.6) suggest to structure the learning flow so that students are paired and

given a series of problems in such a way that they switch the roles Problem Solver and Listener (cf.

subsection 4.4.2 of Chapter Four). However, it does not indicate the number of problems that

should be pair. This is the configurable aspect of TAPPS-based template and is implemented in

Collage as shown in Figure 5.16.

Figure 5.16 Configuring the flow of the TAPPS-based template in Collage: determining the number of

problems that will be though aloud in pairs


139

Apart from the configurable elements of each CLFP and the opportunity of combining several

CLFPs, it is not explicitly possible to add or delete phases and activities. Nevertheless, Collage

allows specifying an activity as not visible, which ensures that it will be ignored during the

execution of the UoL. This can be done in the form (cf. Figure 5.13 (d)) that also enables the

description of activities and the association of the resources that will support the activities (design

tasks e and h). The determination and configuration of the external resources should be previously

accomplished (task g) by means of the Collage interface shown in Figure 5.13 (b). The creation and

search of the resources is outside the scope of Collage (task f) and should be accomplished with

complementary tools. On the other hand, information about the roles/groups including the group-

size limits (minimum and maximum number of persons that can be associated to this role/group),

which is also part to the task e of the design process, can be incorporated in the script through the

interface shown in Figure 5.13 (e). The forms are accessible by clicking on the graphical

representation of each CLFP phase. Collage also includes the functionality of showing a table with

a summary of the created script. It allows users to check whether they forget to particularize any

element of the template. Finally, packaging the CLFP-based LD into a UoL (task i) can be done

using the utility that Reload provides for its own LD editor (University of Bolton, 2004).

5.4.5.4 Discussion Since Collage has been implemented as a new editor in Reload, the tool identifies whether a

UoL has been created by Collage or by another Reload editor (in this case LDE), and opens the

UoL using the appropriate tool. However, the LDs created using Collage can be eventually opened

by, a priori, any LD compliant editor. (Note that high-level or specialized editors, such as Collage

or ASK LDT (Sampson et al., 2005), may need additional information about their representation in

the authoring tool, etc.). This point leads the discussion to one of the limitations of Collage: it

cannot be used as a viewer for any UoL. Other types of authoring tools should be employed to

accomplish this goal and to change low-level elements of the LDs created by Collage.

Although Collage can be used by learning designers, the design process that it implements is

specifically designed to be used by teachers. We support the idea that teachers should be able to

intervene actively in the design process, especially if they do not have the support of specific

instructional designers. A massive support of ICT in Education or specifically of LD requires the

participation of teachers, as real practitioners who know the reality of their context and could

possibly assume the adoption of good practices (such as those reflected in CLFPs).

The initial adopted approach for the selection of CLFPs is simple. A more valuable approach

could be, for example, the use of ontologies. Another limitation regards the addition of new CLFPs.

If a new CLFP-based template is to be included in Collage, the plug-ins related to the graphical

interface for editing the collaborative learning flow must be implemented. Although there is no


140

limit to the possible combinations of CLFPs that can be created, concatenations of CLFPs (adopting

separate sequenced CLFPs) and inserting additional activities are not allowed yet. At this point, it is

necessary to state that our LD editor approach is similar to LAMS (LAMS, 2006) in that it reuses

predefined “modules” created with lower-level tools. However, Collage does not reuse at the

granularity level of activities yet. It reuses the whole learning flow implied in collaborative learning

best practices. There is definitely value in both approaches as they are complementary.

On the other hand, Collage supports only level A of the specification, which is sufficient to

formalize the learning flows suggested by CLFPs. However, in order to enable the creation of more

sophisticated and flexible collaborative UoLs, we are exploring the use of level B and C. Their

support to computationally representing CSCL script requirements is important and necessary in

many situations, as it is analyzed thoroughly in Chapter Four. We argue that the incorporation of

these level B and C elements in Collage should be accomplished by adding reusable chunks and

building blocks that represent specific learning design solutions (based on patterns or not) which

requires these LD elements.

As aimed by the design process, Collage represents a trade off between generality and

unrestricted design options vs. good reuse and particularization of CLFPs as well as an easy editing

of collaborative LDs. First, a simple intuitive graphical representation of each CLFP is provided.

Second, users do not need to be aware of the existence and function of particular LD elements

which are difficult to understand without knowing the specification. These statements are evaluated

in Chapter Six. The particular scripts involved in the evaluated case studies are detailed in next

section, which also illustrates the CLFP-based design process for the creation of scripts

implemented in Collage.

5.5 Creating LD scripts using Collage

Several different CSCL scripts are involved as show examples or as UoLs that are created by

the participants in the experiences comprising the evaluation of the proposed design process (cf.

Chapter Six). These scripts are introduced in this section (the scripts and some snapshots showing

their execution are also available in the CD-ROM). In the description we illustrate how Collage is

used according to the design process. Particularly, the CTM2 script is exposed thoroughly given

that it is the most relevant script in the evaluation experiences. This script is originally proposed by

one of the teachers participating in one of the experiences considering an authentic learning

scenario that he realizes in his own course (at the University of Valladolid, Spain). It is also the

example proposed to other teachers that use Collage. Besides, the CTM2 script is enacted with

students in an authentic situation.


141

5.5.1 CTM2 (Network Management)

The context of this script is a Network Management course (whose Spanish acronym is CTM2),

whose objectives include knowing the different versions of the SNMP (Simple Network

Management Protocol) protocol and familiarizing with the use of scientific and technical literature.

Consequently, the teacher decides that the students should understand the contents of an important

and well-known paper in the field of Network Management (Stallings, 1998), as well as identify its

most relevant ideas. A non-collaborative pedagogical approach might simply consist of providing

the students with the paper, asking them to individually read it and reflect on its content, and finally

request them to write a list of the main identified ideas of the paper. But the teacher also wants the

students to develop competencies related to group work that they will need in their professional

future, as well as to improve the understanding of the paper contents. Therefore, he decides to

create a collaborative pedagogical design, in the form of a computer-interpretable collaboration

script, which takes into account both types of requirements (concepts and competencies).

Table 5.2 Refining the template resulting of the combination of CLFPs towards the ready-to-run script (students’ activities)

Learning Flow Phase Group characteristics

Activity description and supporting resources

Time frame

individual phase of Jigsaw

Each “jigsaw group” has at least 3 people. (Plan: 4 groups of 3 members.)

- Read the introduction and one of the three sections of the paper available in Synergeia

“expert” phase of Jigsaw

Each “expert group” has at least one member of each “jigsaw group”.

- Discuss (using the chat) your part of the paper with the classmates that have read the same part in order to understand it well.

Pyra

mid

leve

l 1

“jigsaw” phase of Jigsaw (See Pyramid level 1:

individual phase of Jigsaw)

- Explain to the rest of the members of your “jigsaw group” your part of the paper. - The group has to agree on which are the 10 main ideas of the whole paper and 2 questions. Use the tool for questionnaires (Quest).

Firs

t fac

e-to

-fac

e se

ssio

n of

two

hour

s D

ista

nt

activ

ity

Pyra

mid

leve

l 2

Each group of the second Pyramid level comprises from 6 to 8 students. (Plan: 2 groups in this level of the pyramid.)

- Read other group’s results of the previous phase at home. Their results are available in a shared repository (Synergeia). Critically comment their answers (using Synergeia). - Discuss face-to-face with another “jigsaw group” and jointly agree on the 8 main ideas of the paper and 2 questions (use Quest).

“think” phase of TPS

In this pyramid level there is only one group: the whole class.

Not visible (there is no task in this phase)

“pair” phase of TPS

Each group of the second level of the pyramid is one of the two members of the “Pair”.

- A speaker of each group exposes their conclusions to the other group.

Pyra

mid

leve

l 3

“share” phase of TPS

(See Pyramid level 3: “think” phase of TPS).

- The teacher mediates a discussion aiming at reaching consensus.

Seco

nd fa

ce-to

-fac

e se

ssio

n of

tw

o ho

urs

With that purpose, Collage assists the teacher in the creation of potentially effective scripts,

using a proposed design process based on CLFPs. As it is described above, the first steps of the


142

design process is to select and combine, if required, the LD templates based on the CLFPs that

Collage provides. After reading information about the patterns, exploring examples, as well as

checking the educational benefits and types of problems supported by each pattern, the teacher

decides which CLFPs can be used and how they can be combined in order to structure the flow of

activities. The first column of Table 5.2, Figure 5.17 and the left frame of both Figure 5.18 and

Figure 5.19 show the result of these steps.

Figure 5.17 Planning the learning flow: combining PYRAMID, JIGSAW and TPS in form of LD templates using a graphical notation in which the visual representation of each template corresponds to the diagram of

its related pattern’s solution

With the aim of fostering discussion and reaching agreement, the teacher selects PYRAMID,

which promotes positive interdependence (i.e. the feeling that the students need each other to

succeed, cf. Appendix A). The number of levels of the template can be configured with Collage (cf.

Figure 5.18). In each level of the Pyramid the groups of the previous level join in larger groups in

order to generate agreed “solutions”. In this case the “solution” consists of successively proposing

which are the most important ideas of the paper and what questions they would like to be answered.

Since the teacher is also interested in focusing the students’ attention on particular topics of the

article, he plans a plenary discussion mediated by him and structured according to the TPS (cf.

Figure 5.19) for the last level of the Pyramid.


143

Figure 5.18 Graphical refinement of the template resulting of the combination of CLFPs using Collage (I)

Figure 5.19 Graphical refinement of the template resulting of the combination of CLFPs using Collage (II)

Moreover, the teacher chooses to design the first pyramid level according to the strategy

collected in JIGSAW-based template. The selection of this pattern is motivated by the fact that the

paper can be divided in different parts, because the article includes several sections devoted to

different versions of SNMP. An additional motivation for its use is that the JIGSAW strategy


144

reduces students’ workloads but at the same time fosters that they contribute with their fair share

(individual accountability). This pattern (cf. Appendix A) suggests that each participant in a

(Jigsaw) group first studies a particular sub-problem (a section of the paper). Then, the members of

the different groups that have worked on the same sub-problem meet in an “experts group” to

exchange ideas. Finally, each participant contributes with his expertise in the original group in order

to tackle the global problem (i.e. the whole paper).

After selecting and combining CLFPs, the next step in the design process implies refining the

resulting LD template so as to obtain a computer-interpretable script. This step includes

particularizing the description of the learning activities (e.g., reading a part of the paper, discussing

on questions/ideas, etc.), providing information about roles and groups, establishing group-size

limits, and determining and configuring the resources (tools and content) needed to support each

activity. These tasks are accomplished using forms, as it is shown in Figure 5.18, where the first

activity of the second Pyramid level is described, and Figure 5.19, where the Pair phase is refined

by including the description of the activity and the groups. Table 5.2 summarizes the result of this

refining step and details the planned time frame. The refinement also includes indicating the tools

that support each activity. In this script the tools are: Synergeia system (ITCOLE, 2005), which

mainly provides a shared web-based workspace in which documents and ideas can be shared; a chat

tool developed by members of the GSIC/EMIC group (GSIC/EMIC, 1994), and the Quest web-

based questionnaire tool also developed by the same group (Gómez et al., 2002).

5.5.2 NNTT

The “NNTT script” is highly inspired in a real experience that takes places within the course on

“the Use of ICT in Education” at the Faculty of Education of the University of Valladolid.

Moreover, this experience is one of the case studies included in the TELL project (TELL, 2005b). It

is a blended situation, where normal face-to-face activities are interleaved with technology-

supported (distant and face-to-face) activities. The experience also makes use of the Synergeia

system (ITCOLE, 2005). The script guides the students in the revision of three topics in order to

produce a deeper understanding of them. Table 5.3 describes this script, which is based on a

combination of the Jigsaw and the Pyramid CLFPs.

Figure 5.20 illustrates how the learning flow of the example can be edited using Collage. After

selecting the CLFP base of the flow of activities, i.e. JIGSAW, the “Expert Group” phase is replaced

with a two-level PYRAMID-based template. This is indicated with the circled “1”, “2” and “3” of

Figure 5.20. “4” points to the whole structure of the activity flow. This tree also provides access to

any CLFP included in the hierarchical structure, thus the activities of each CLFP can be further

particularized. The tasks of describing activities, roles, associating resources to activities, etc. are

accomplished using a form analogous to the example shown in Figure 5.20 (these steps are detailed


145

in the worksheet included in the Collage user manual, which is available at

http://gsic.tel.uva.es/collage).

Table 5.3 Refining the template based on a combination of a JIGSAW and a PYRAMID towards the ready-to-run script

Learning Flow Phase

Group characteristics Task and supporting resources

Individual phase of Jigsaw

Initial groups: pair of students (at least 2 students). There must be the same number (approximately) of pairs working on each of the three topics.

- Work on one particular topic of the subject "Use of ICT Resources in Education". The resources needed to perform the activity are available in Synergeia. Each pair should create a conceptual map regarding their topic. Employ a template and the conceptual map tool of Synergeia, and upload the resulting document to Synergeia.

Pyramid level 1

Half of the pairs that have worked on the same topic form a group at this first Pyramid level.

Compare their conceptual maps. (Note that the conceptual maps are all available in Synergeia). You can use a chat. Create a draft document according to a provided template, and upload the document to Synergeia. Compare and discuss the draft documents generated in the previous phase (you can use a chat).

“Exp

ert”

pha

se o

f Jig

saw

Pyramid level 2 All pairs with the same topic. Create an agreed report according to the same

previous template, and upload the document to Synergeia. Discuss using the reports created previously, which are in Synergeia (you can use a chat). “Jigsaw” phase of

Jigsaw Three pairs (or four if necessary) with different topics form a “Jigsaw Group” Create a global final report according to what

has been discussed (a template is provided).

Figure 5.20 Authoring the NNTT script with Collage

5.5.3 Pyramid-based paper discussion

The context of the example could be a course about “technical specifications for interoperable

learning technology”. The script specifies a learning situation in which the participants read a paper

titled “The role of teachers in editing and authoring Units of Learning using IMS Learning Design”


146

(Griffiths et al., 2005) and (collaboratively) discuss about the contents of the paper. They have to

achieve gradual consensus about good solutions to address the identified challenges regarding the

way in which teachers can author UoLs using LD. The example is designed for one synchronous

session of 2 hours. However, we do not know the number of students that will attend this session at

design time. The script is structured according to the PYRAMID CLFP and is particularized as

indicated in Table 5.4.

Table 5.4 Refining the template based on the PYRAMID towards the ready-to-run UoL Learning Flow

Phase Group characteristics Task and supporting resources

Pyramid level 1 (Individual activity)

Read the paper to grasp the main ideas. A synchronous conference service is available so that each student can ask questions to the teacher.

Discuss your opinions about the paper. Pyramid level 2

Each group of the second Pyramid level comprises from 3 to 6 students. (Depending on the participants who attend the session, the script will be instantiated with a concrete number of groups, which in turn will be populated with (from 3 to 6) students.)

Extract common conclusions about the paper.

Each group presents their conclusions achieved in the previous phase.

Pyramid level 3 All the groups of the second Pyramid level form a single group at this level.

Finally, agree on common conclusions about the paper (and try to achieve consensus about good solutions to address the challenges regarding the way in which teachers can author Units of Learning using LD).

5.5.4 Job interview simulation

The objectives of the “Job interview simulation” script, which is collaborative and blended, are

related to the development of the social skills required in job interviews. The script is based on the

SIMULATION (Pattern 1.5 of Appendix A). This script describes a situation in which two students

play the role of “interviewers” and other two students play the role of “candidates”. One of the

interviewers and one of the candidates train a simulation of an interview for a “developer vacancy”.

At the same time, the other interviewer and candidate train a simulation of a job interview for a

“salesman vacancy”. Previously, each participant reads general information about job interviews

and prepares their role according to their assigned interview. After each pair trains their simulation,

they perform it, so that the other pair can observe the role play. Finally, the teacher moderates a

debate about the social skills required in this type of situations.

5.5.5 STA (Advanced Telematic Systems)

The STA script is applied in a graduate course on research methodologies at the University of

Valladolid. In the STA script, the students should try to propose a research question on a complex

multidisciplinary field that involves several keywords. In order to achieve this goal, JIGSAW is


147

employed, where students in the expert group study and propose research questions related to some

of the keywords. Then, students in the “Jigsaw groups” try to merge the research questions.

5.6 Discussion: towards more general approaches

Two of the strengths of our approach consist in the use of graphical representations that are

intuitive and the promotion of interoperability since the resulting scripts are LD compliant.

However, these strengths at the same time are limitations of the approach. First, the use of graphical

representations that strongly depends on the CLFPs implies that adding new CLFP-based templates

or other types of (pattern-based) reusable solutions is a demanding task. Second, this task is even

more challenging if the reusable elements are not represented with LD. This section discusses

potential solutions towards a more general pattern-based design process for the creation of scripts.

5.6.1 Using more general visual representations

There are currently only six patterns incorporated in Collage. However, since they can be

creatively combined in order to create new richer learning flows, it offers some degree of flexibility.

As categorized in (Botturi et al., 2006), Collage provides generative patterns which can then

communicate with a more finalist language (LD) using the graphical representations of the patterns’

solutions. However, one of the limitations of Collage is that the addition of new templates based on

other patterns is laborious: each template has its own representation and includes pattern-specific

functions. To overcome these drawbacks, it would be interesting to study the possibility of using

visual languages for learning design, such as E2ML (Botturi, 2006) or the others approaches

presented in (Botturi & Stubbs, in press), to graphically represent patterns’ solutions. This may not

only facilitate the addition of new patterns (using the same graphical notation) to Collage, but it

may also afford more flexible editing possibilities of the learning flow (e.g., using drag-and-drop

elements of the visual language that can be added or removed from the templates).

Nevertheless, this topic deserves devoted research that investigates the trade offs between

intuitiveness and generality of the different graphical approaches.

5.6.2 Assembling learning design solutions formalized with different languages

The create-by-reuse framework proposed in section 5.2 envisages an interesting challenge

which may be considered in future versions of the proposed design process: the involvement of

learning design solutions formalized with different languages (e.g. the formalisms used in LAMS

and IMS QTI for questionnaires) so that they can be assembled in order to generate an LD-

compliant UoL (or eventually another type of UoLs using a different formalism). Therefore, the

problem that emerging design processes should address is not trivial. Not only do we need to


148

assemble and refine learning design solutions at different level of granularity and completeness but

we also need to transform formalizations (Hernández-Leo et al., 2006a; Dodero et al., 2007). These

ideas are illustrated with the following ad-hoc design process example, which is represented in

Figure 5.21.

n

n

Collage Templates

IMSLD/LAMSmapping rules

IMSLD/QTI mapping rules

t1

QTI Learning Objects

LAMS Activities

LAMS building blocks

QTI building blocks

IMS LD templates

qti1 qti2 qti3

a1

a2

qti1qti2qti3

t2

a1

a2

qti1qti2qti3

t3

Refine by assembling

(mixed)

Assemble

Refine

Reusable exemplar

(UoL) t2

Assemble

Assemble

Refine

a1

a2

chunk Figure 5.21 Example design process in which various learning design solutions are integrated into

refinement, assembly and mixed processes, according to the create-by-reuse framework

The process starts by choosing Collage templates to select the PYRAMID-based LD template

(t1), which consists of two incomplete activities (an individual and a collective activity). Then it

proceeds to the selection of three QTI items, which are assembled forming a questionnaire. The

template is refined into t2 by assembling the questionnaire: the individual activity will consist in

answering a questionnaire. In addition, two LAMS activities (which include the supporting tools)

are assembled and subsequently refined with the necessary text that particularizes the activity. a1

encourages the students to share resources and a2 provides a forum for discussing. To particularize

for example a2 the title, the instructions and the topics of the forum must be typed. The resulting

chunk is assembled with t2 as additional activities according to the rules used to map LAMS

activities into the coarser grained LD template. The outcome is the template t3, which still needs to

be refined in order to be ready-to-run. Once the activities of the template t2 are set up by adding the

necessary text (the task of the collective activity, the grades related to each question of the

questionnaire, etc.), a complete exemplar is achieved. This exemplar can be delivered as a UoL or,

according to the designer’s criteria, be reviewed and modified. The complete process is graphically

depicted in Figure 5.22 according to the create-by-reuse framework.


149



Incomplete


UoLs of different

granularities)

Templates (Partly


Building blocks/

Compontents (Partly


UoL chunks (A UoL


granularities)

Complete

t1 t2

A1

M2

a1+a2

qti1+qti1 +qti1

qtij ai

t3

M1

R R

A2

Figure 5.22 Snapshot of the example design process that integrates assembly, refinement processes and

mixed processes, in accordance with the create-by-reuse framework

In the figure, point t1 is the entry LD template that represents a coarse-grain LD abstraction (e.g.

a CLFP) that is used as a starting point for transformations. Since t1 is an incomplete LD template, it

is situated above the horizontal axis. At the same time, selected learning objects and activities are

composed by means of assembly transformations on the vertical dimension (A). The addition of

item qti1 does not increase the granularity on M1 (mixed process: assemble by refining) step, since

it is used to fill in a gap on the t1 template, so that t2 is generated. t3 results from the assemblage of t2

and the chunk consisting of two already refined and assembled building blocks (a1 and a2). That

entails increasing the coarseness with respect to t2 as it can be seen in the Figure 5.22. In this

example, we omitted the modification processes of the framework, which are orthogonal to R and A

and not explicitly represented in the two-dimensional figure. However, to envisage modifications,

the input and output can be depicted sharing the same projection on the R-A plain.

5.7 Conclusion

LD promotes interoperability and reuse of teaching and learning processes, including those

described in CSCL scripts. However, the specification is a set of dense technical documents,

intended for a technical audience and does not enforce design processes that support the creation of

pedagogically sound designs (cf. Chapter Four). In this sense, approaches to facilitate the creation

new UoLs based on pre-existing learning design solutions are emerging. The target audience of


150

these approaches are teachers who use concepts and structures more relevant to their community of

practice than the terms of the LD specification. This chapter introduces a comparison framework for

conceptually analyzing and classifying reusable learning design solutions and processes that drive

the creation of ready-to-run UoLs. The framework provides a comprehensible representation of

such processes and units of reuse over two dimensions, namely granularity and completeness.

The design process proposed in this chapter employs CLFP-based templates as the units of

reuse for the creation of scripts. In this case the target audience is therefore teachers that are

interested in practicing scripted CSCL. Instead of collecting CSCL scripting patterns (cf. Chapter

Three) in repositories, these patterns can be explicitly incorporated in design processes embedded in

authoring tools as assistants (advising mechanisms) or as refinable templates and building blocks

(partially completed designs or design chunks). The CLFPs which provide structures of CL flows in

their solution can be offered as LD templates. These templates, which are partly completed re-

usable designs, includes the description of the flow of activities and associated groups (or roles) that

potentially elicits expected interactions leading to certain well-known CL benefits. The diagrams of

the CLFP solutions are used as the graphical representations of the templates, which include

configurable aspects that depend on the pattern. Though more general visual notations are possible,

we argue that these particular representations increase intuitiveness and foster the understanding of

the CLFPs.

The templates can be assembled forming new templates representing CLFP hierarchies. Of

course, these templates need to be refined with the details that particularize them into full-fledged

scripts according to the needs of actual educational situations. In this way, our approach aims at

reducing the complexity of the learning design process by hiding the terms of LD, as well as at

guaranteeing potentially effective results, since the process is based on the reuse of good practices

in CL. Consequently, it fosters the reuse of the patterns as a way of communicating CL expertise to

other, eventually novice, practitioners. The proposal offers a trade off between generality and

unrestricted design options vs. adequate reuse and particularization of CLFPs (and hierarchies) and

an easy edition of collaborative LDs.

Two main phases are considered in the design process: selecting a CLFP-based template and

authoring a CLFP-based UoL. The selection of templates is mainly supported by using metadata

whose elements include the learning objectives that the CLFP elicits, the type of problems (task-

type) that the CLFP best serves, and the complexity or risks in terms of CL experience needed to

put in practice a script based on the pattern. Further information about the CLFPs, including an

example, should be available to enable teachers understand a CLFP before reusing it. When

teachers select a CLFP-based template, they automatically determine the level of structure that is

necessary to accomplish a set of learning objectives, to elicit certain expected interactions and to

undertake specific task-types.


151

In the authoring phase of the design process, teachers refine the templates and indicate further

objectives (beyond those depending on the CLFP) that are typically (but not necessarily) related to

the subject matter or content to which the CLFP is applied. They can also specify the title of the

script and prerequisites, if necessary. In this phase specific configurable aspects of the templates

need to be determined. Besides, the learning flow can be enriched by combining or concatenating

CLFPs into templates of CLFP hierarchies. If applicable, the selection phase is repeated. Defining

the description of the activities and determining the resources needed to support them implies

determining the structure of the interaction process within activities. This is related to determining

the computer support that is used to sustain learning and expected interaction (face-to-face or

computer mediated). Moreover, determining the group-sizes and providing further information of

groups can be also realized in this phase.

The CLFP-based design process for the generation of LD scripts is implemented in Collage

authoring tool. Collage is developed as a collection of plug-ins on top of the Reload LDE. The

plug-ins implement the graphical representation as well as the interactive functionalities that are

specific to each CLFP. The design process, as implemented in Collage, has been thoroughly

illustrated with several scripts drawn from real practice, which shows the feasibility and usefulness

of the whole approach. Next chapter is devoted to the evaluation of the proposed design process in

several case studies mainly with teachers, who use Collage to create CLFP-based scripts, and

students, who experience these scripts.


152

CHAPTER SIX

EVALUATING THE DESIGN PROCESS

WITH A MULTICASE STUDY

This chapter undertakes the evaluation of the proposed CLFP-based design process for creating CSCL macro-scripts computationally represented with LD. The evaluation is accomplished by means of a multicase study that comprises three different case studies. They aim at assessing the same contributions but from different perspectives. The first case study is devoted to workshops where the target audience uses the design process implemented in Collage authoring tool. A second case study implies the design of a scenario proposed by a third-party using our approach. The last case study analyzes an authentic educational situation where students follow a script created according to the CLFP-based design process.

Partial conclusions of the second case study are published in (Hernández-Leo et al., 2006b) and the last experience with students in (Hernández-Leo et al., in press).

6.1 Introduction

The multidisciplinary engineering-oriented research methodology adopted in this dissertation

(cf. Chapter One) highlights the importance of the evaluation phase to demonstrate the

trustworthiness of the proposals. The problem domain in which this dissertation is framed, demands

that the practice presented in this chapter evolves together with the research presented in the

previous chapters. In this sense, although the results of the evaluation phase are presented at the end

of the dissertation, it is necessary to have in mind when reading this chapter that the partial results

of this phase feed previous phases.

The evaluation phase consists of several experiences organized as case studies (Hernández-Leo

et al., 2006; Lundgren-Cayrol et al., 2006). “Case study” is one of the software engineering

validating models identified by Zelkowitz et al. (1998). That review paper groups different models

of technology validation into three broad categories: observational, historical, and controlled. Case

studies belong to the so-called observational methods, which collect relevant data as a project


154

develops. Despite the weaknesses that are typically associated with this type of validating models

(e.g. it is not always possible to generalize), this method is the most adequate for the problem

domain of this dissertation. Historical methods such as literature research or surveys cannot be used

in this phase since there is not enough existing experience yet. The need of involving human

subjects in the experiences makes difficult to use a controlled method, which requires multiple

instances of an observation in order to provide statistical validity of the results. Because of the

enormous cost of replication (in our context, the possibilities of organizing experiences with

teachers and students), the controlled experiments are often limited to few replications, which

seriously increases the risk related to the validity of the results. Moreover, “authentic” experiences

with real users cannot be rigorously replicated due to their non-trivial situational-dependence

characteristics. Furthermore, we are more interested in the appropriateness of our contributions,

evaluating the features that they provide in relation with the objectives target in this dissertation (cf.

section 1.2), than in their measurable effects (Kitchenham, 1996).

In the case studies, we employ adaptations of a mixed evaluation method (Martínez-Monés et

al., 2006) combining quantitative and qualitative data gathering techniques (Goubil-Gambrel,

1992). Quantitative data are considered useful for showing trends. In addition, qualitative results are

used to confirm or reject those trends as well as to understand them and identify emergent features

in the particular representative situation (Denzin & Lincoln, 2005).

According to Stake (1995), there are many purposes for case study analysis: from the most

theoretical to the most practical. The purpose of an “instrumental” case study is to go beyond the

case. In contrast, a case study is “intrinsic” when the main interest is in the case itself. In this

dissertation, we need to assess the features of the proposed design process (cf. Chapter Five) with

different types of audiences (mainly teachers and students) and different types of functioning

(applying the design process to create a script and experiencing the resulting script). Since we aim

at studying different manifestations (types of functioning) of the same “proposal under evaluation”,

what Stake calls “quintain” (what we truly seek to understand), we organize the experiences as a

multicase study (Stake, 2005). The central interest of a multicase study is in the quintain; therefore

the interest in the cases is primarily instrumental.

In this way, we use the multicase study method to systematize the evaluation of the different

experiences accomplished with diverse types of participants. It is necessary to point out however

that we are not “orthodox” using multicase study analysis as the inclusive research methodology of

this dissertation, what is typical in pedagogical dissertations (see for example (Jorrín-Abellán,

2006)). In summary, we employ an instrumental multicase study as an approach to structure the

evaluation phase of our multidisciplinary engineering-oriented methodology. The approach includes

the application of a mixed evaluation method for data gathering.

EVALUATING THE DESIGN PROCESS WITH A MULTICASE STUDY

155

Particularly, the multicase study applied in this dissertation comprises three different case

studies, each of which involves a different functioning. The final goal is to aggregate and

comparatively analyze the case findings (the conclusions of the data analysis and interpretation

regarding a case study) in order to conclude cross-case assertions concerning the evaluation of the

design process (the quintain). The main action that enables the evaluation of the design process

refers to the functioning of creating CSCL scripts (in form of UoLs which are LD compliant)

following the proposed design process, which is implemented in Collage. Therefore, we carry out

four experiences with slightly different audience (mainly university teachers) that comprise the first

case study (called “Collage workshops”). Most of the scripts created in these experiences are

proposed by the author. In this sense, the functioning of the second case study is solving a scenario

proposed by a third-party. This scenario is also tackled by other researchers proposing related

approaches (some of them employing different script computational representations), what allows

us to compare them with our proposal. However, in order to test if the LD-represented scripts

created using the design process based on CLFPs are meaningful, it is necessary to put them in

practice in an authentic situation with students. This functioning is the focus of the third case study.

Therefore, the rest of the chapter is structured as follows. Section 2.2 formulates the multicase

study. Then, we describe and present the findings of each case separately (sections 6.3, 6.4 and 6.5).

In section 6.6 we relate the case studies including a cross-case analysis with emphasis on the

binding ideas that built the assertions regarding the global evaluation of the main proposals of this

dissertation. Finally, section 6.7 includes a discussion preceding the last chapter of conclusions

(Chapter Seven).

6.2 Formulation of the multicase study

The starting point of multiple case study is the quintain (Stake, 2005). As it has been anticipated

in the introductory section (2.1), the quintain is our ultimate evaluation goal: what we seek to

understand. It is the umbrella for the experiences involved in the evaluation. These experiences are

organized in case studies according to their functioning or different manifestations of the quintain.

The collection of the case studies comprises the multicase study.

6.2.1 The quintain

In this dissertation the quintain is its main contribution: the pattern-based design process for

CSCL macro-scripts computationally represented with IMS LD. Besides, the global research

questions derived from the quintain (what is called “themes” by Stake (2005)) are:

Theme 1: High-level generation of contextualized CSCL scripts reusing CLFPs and focusing on

CSCL critical elements


156

Theme 2: IMS LD computational representation of CLFP-based scripts

These themes are directly related to the objectives of this dissertation (cf. Chapter One) and can

be formulated as questions:

Theme 1: Does the proposed design process facilitate the high-level generation of

contextualized CSCL scripts reusing CLFPs and focusing on CSCL critical elements?

Theme 2: Is IMS LD suitable for a computational representation of CLFP-based scripts?

Moreover, additional themes may emerge as a result of the cross-case analysis. This analysis is

a comparative aggregation of the findings obtained in the different case studies. The outcomes of

the analysis are cross-case assertions related to the themes.

6.2.2 Overview of the cases forming the multicase study

A case needs to be studied in its own context. On the other hand, in a multicase study, a single

case is of interest because it belongs to a concrete collection of cases: each case illuminates a

different functioning in which the quintain operates (Stake, 2005).

The cases are selected considering the following criteria, which are discussed in the section 2.2

of (Stake, 2005):

- Relevant to the quintain: the planning of the cases should be consistent with what can help to

understand the quintain.

- Diversity across situations to examine different functioning (probably representing different

audiences) in which the quintain is manifested, i.e. diversity in the relationships with the

quintain.

- Good opportunities to learn: it may represent a trade-off between how typical and how

accessible the cases are, i.e. representativeness vs. potential for learning.

- Embraceable: we need to reach an integrated, holistic qualitative comprehension of the

cases, so that the multicase should not be too large and diverse.

Considering these criteria, we organize the multicase with the three cases shown in Figure 6.1.

Besides, the different experiences involved in the cases are summarized in Table 6.1. As we

anticipated in the introduction, the experiences with the same functioning compose a case study.


157

ISSUE:

Can we use CSCL scripts created with Collage in real situations?

ISSUE:

Can we use Collage for creating a script representing a scenario proposed by a third-party?

ISSUE:

Does the design process implemented in Collage facilitate the reuse of CLFPs in the creation of particularized LD-represented CSCL scripts in a way that allows teachers to focus on CSCL critical elements?

QUINTAIN:

The proposed pattern-based design process for CSCL macro-scripts computationally represented with IMS LD

Research

environment Relevant

research

Relevant

research Research

environment

Mini-Cases - GSIC/EMIC members

- UNFOLD members

Relevant

Research

Techniques and data - Web-based questionnaires - Observations - Discussion Group

Activities - Hands-on sessions - session1 (UVA) - session2 (UCA)

Sites - Laboratory at the University of Valladolid - Laboratory at the University of Cádiz

Research

environment

Mixed method Interviewees - Teachers from the University of Valladolid, Spain - Teachers from the University of Cádiz, Spain

CASE STUDY A: COLLAGE WORKSHOPS

Activities - Two face-to-face sessions and a distance session in between

Sites - Laboratory at the University of Valladolid - Students’ homes Mixed method

Function Putting into practice a CSCL script

CASE STUDY C: NETWORK MANAGEMENT

Techniques and data - Web-based questionnaires - Observations - Discussion Group - Event Log files - Student outcomes

Function Solving a third-party scenario

Sites - Rooms at ICALT conference (Kerkrade)

Activities (- Pre-work) -Presenting -Discussing

Documents and data -Achieved CSCL script - Papers and video-recorded discussion

CASE STUDY B: PLANET GAME

Function Creating CSCL scripts based on CLFPs

using Collage

Figure 6.1 Graphic representation of the multicase study

Table 6.1 Summary of the experiences involved in the multicase study

CSCL script Functions (Global activity

accomplished in the experiences)

Name of the experience and dates

Interviewees (type of audience)

“Proposed” (previously tested

in lab. experiments)

“Free” (ideated by the

audience or a third-party)

Deployment Materials Evaluation data

gathering

GSIC/EMIC (19 May 2005)

Five members of the GSIC/EMIC research group: CSCL practitioners without deep knowledge on LD

NNTT CTM2’, STA - Free deployment (with assistants)

Collage user manual and worksheet (NNTT). Collage already installed

- Final questionnaire - Created scripts

UNFOLD (13 October 2005)

Seven (official and invited) members of the UNFOLD project: educational technologists (some of them experts on LD)

Paper-discussion (NNTT, show example)

-

-Familiarization (presentation), -guided creation of a CSCL script

Worksheet (Paper-discussion). CD containing Collage installers, user manual and the resource needed in the script

- Final questionnaire

UVA (3 March 2006)

Five teachers of the University of Valladolid (Spain) with interest in ICT and CL and without knowledge on LD

(Job-interview, show example) CTM2 (a slightly modified version of CTM2’)

(various, the audience had some time to create their own scripts)

-Familiarization (presentation), -guided creation of a CSCL script - free deployment (optional, with assistants)

Collage user manual and worksheet (CTM2). CD containing Collage installers and the resources needed in the script

- Initial (before familiarization) and final questionnaire - Observations - Discussion group

Creating CSCL scripts based on CLFPs using Collage (CASE STUDY A: COLLAGE WORKSHOPS)

UCA (10 March 2006)

14 teachers of the University of Cádiz (Spain) that use virtual campuses without deep knowledge on LD

(Job-interview, show example) CTM2

(various, the audience had some time to create their own scripts)

- (Previous sessions about LD, by Daniel Burgos) -familiarization (presentation), -guided creation of a CSCL script, - free deployment (optional)

Collage user manual and worksheet (CTM2). Internet access to Collage installers and the resources needed in the script and further information

- Initial (before familiarization) and final questionnaire - Discussion group

Solving a third-party scenario (and comparing our proposal with other approaches) (CASE STUDY B: PLANET GAME)

ICALT (5-6 July 2006)

Nine participants in a workshop and a panel at ICALT conference: researchers proposing related approaches

- L3-astronomy (or Planet Game)

- We propose a solution for the scenario using our proposal beforehand. - Comparison of the different approaches: presentation and discussion

Description of the scenario provided by a third-party

- Achieved CSCL script with our proposal - Papers and recorded discussion

Putting into practice a CSCL script created with Collage in a real educational situation (CASE STUDY C: NETWORK MANAGEMENT)

CTM2 (29 March – 5 April 2006)

12 university students and the teacher (engineering education)

(Paper-discussion’, show example) CTM2

-

-Familiarization (a session for presenting the script, the tools and training with another script: Paper-discussion’) - Experiencing the script (a week)

Gridcole system integrating a chat, Synergeia and Quest and interpreting the script. A document explaining the roles students play in each moment

- Initial and final questionnaires - Observations - Discussion group - Log files - Students’ outcomes

The graphical representation (Figure 6.1) of the three cases is an adaptation of the worksheet 1

included in (Stake, 2005). This graphic model has been shown to be a valid framework in other

studies (Jorrín-Abellán et al., 2006; Jorrín-Abellán, 2006). The smaller semicircles around the main

circles constitute the context of the case. The context greatly helps us to understand the case within

a complex reality that conditions its realization. Particularly, the context of the multicase study that

is applied to evaluate the proposed design process is:

- The relevant research around the topics of the cases is studied in Chapter Two.

- The research environment in which this dissertation is carried out does not only conditioned

the case studies but also facilitates its fulfilment. In the evaluation phase the concerned

research environment includes the GSIC/EMIC research group, the UNFOLD project and

the ICALT conference (they are already pointed out in Chapter One).

We might also consider as part of the context the characteristic of the audience (interviewees)

and their motivation for participating in the experiences (cf. Table 6.1). With the aim of simplifying

the description of the cases, we consider these aspects as part of the cases themselves. Therefore,

this is part of the information related to the main topics of the cases that is represented inside the

thick lined circle. This information includes the functioning of the case, the sites in which it took

place and the activities that were analyzed during the experiences.

The lower side of the circles show components of the set of techniques applied to data

gathering, analysis and interpretation processes. This mixed method is introduced in next section

(6.2.3). On the other hand, one of the cases (Case Study A: Collage workshops) shows more

elements in this side of the circle. These elements are represented in order to illustrate that several

experiences (cf. Table 6.1) are involved in this case. Teachers from two different universities are

interviewed and two embedded cases (mini-cases) are used to illuminate some aspects of the case

under study. Mini-cases could be considered single case studies, but we decide not to focus on

them. In contrast, we use them with the aim of illuminating some relevant aspect of the other

(larger) case. In these two mini-cases we interview educational technologists and CSCL researchers.

In the lower side of the diagram a rectangle can be found. It contains the issue of special

concern or importance regarding each case. An issue has associated topics and more concrete

information questions that shape the “conceptual structure” needed for designing and interpreting

the study. The issue is related to the functioning of the case, reflecting its main purpose. In this

multicase we consider three different types of functioning and therefore three case studies with

associated issues:

A) The main functioning that should be analyzed to evaluate the proposed design process is

“creating CSCL scripts based on CLFPs using Collage”. This is the main functioning

since the associated case study (Case study A: Collage workshops) allows us to


160

understand to a large extent a relevant issue concerning one of the objectives of this

dissertation: does the design process implemented in Collage facilitate the reuse of CLFPs

in the creation of particularized LD-represented scripts, in a way that allows teachers to

focus on CSCL critical elements? Due to the relevance of this case study, it involves two

different experiences with university teachers (from the University of Valladolid and from

the University of Cádiz, both in Spain) consisting of four-hour workshops (or “hands-on”

sessions) in which the participants create, as indicated in the design process implemented

in Collage, concrete CSCL scripts (proposed by us). The case study is enriched with two

mini-cases accomplished as (shorter) workshops in which educational technologists

(members of the UNFOLD project) and CSCL practitioners (members of the GSIC/EMIC

group) participate. Some participants of the experiences forming this case study have the

opportunity to create their own scripts. However, this fact is not deeply analyzed in this

case study (as it is analyzed in case study B). A more rigorous experience regarding the

creation of a script that is not proposed by us would provide further conclusions related to

the quintain.

B) A good chance to analyze the functioning “solving a third party scenario” entails the

participation in a workshop integrated in the ICALT 2006 conference (Vignollet et al.,

2006). The organizers of the workshop propose a challenge consisting on implementing a

scenario (a “Planet Game”) that they anticipate. The participants apply their approaches to

solve the scenario before the workshop. In this way, we create and execute a CSCL script

reflecting the scenario. During the workshop the different solutions are presented and

compared. Hence, the issue that this case study (Case study B: Planet game) illuminates

is: can we use Collage for creating a script representing a scenario proposed by a third

party?

C) The last but not least functioning is “putting into practice a CSCL script”. It is crucial

to study this functioning in order to prove that the scripts created using the proposed

design process are meaningful and can be satisfactorily used in authentic situations, i.e. to

study the issue: can we use CSCL scripts created with Collage in real situations? The

experience analyzed in the case study devoted to this functioning (Case study C: Network

Management) is a one-week blended learning situation part of a university (engineering

education) course at the University of Valladolid. The script is interpreted using Gridcole

system (Bote-Lorenzo et al., 2004).

In conclusion, the selected cases are relevant to the quintain and provide opportunities to

examine different functioning (having different relationship with the quintain). Moreover, they

represent a trade-off between accessible chances with a good potential for learning and

representative experiences. The dimension of the multicase study is embraceable: the experiences


161

are not too large and the quantity of data gathered according to the following mixed method enables

an analysis that can be accomplished by one person in a reasonable time.

6.2.3 Mixed method

Since we use a multicase study to organize the evaluation, the analysis process highlights a

qualitative perspective. That is to say, we aim at understanding each case taking into account its

context. However, the method employed to collecting data is mixed in the sense that it integrates

quantitative and qualitative data gathering techniques (Martínez-Monés et al., 2003). Quantitative

data allow detection of general trends, while data obtained through qualitative techniques allows the

evaluator to understand these trends better by introducing contextual issues and considering

participants’ perspective. Figure 6.1 and Table 6.1 show the techniques and data sources considered

in each case: qualitative sources such as open questionnaires (useful for obtaining quotations),

observations and discussion groups, and quantitative sources such as closed questions (to obtain

measurement data) and automatically generated event log files. The techniques used in the case

studies differ due to the circumstances related to time or accessibility constraints. However,

according to (Stake, 2005) this is not a problem since the cross-case analysis is qualitative and does

not only compared quantitative measurable effects.

The mixed method considers three phases:

1. Definition of a scheme of categories: the initial categories are proposed according to the

evaluation objectives and, therefore, to the conceptual structure of each case (particularly to

the concrete information questions derived from the topics structuring the issue under

evaluation). However, new categories emerge from the study, through the specialization of

the existing categories or the addition of new ones. This fact provokes that the conceptual

structure of the case studies (especially the information questions) evolve along the

analysis.

2. Data gathering: collecting quantitative and qualitative data using different techniques

before, during and after the experience. The use of the techniques is designed according to

the previously defined categories of study.

3. Analysis and data interpretation: the quantitative information is aggregated and pre-

processed using simple descriptive statistical analysis, which is used only to support and

complement the analysis of the qualitative data. The qualitative data is accumulated and

structured into the initial and emergent categories, facilitating an organized interpretation of

the arguments.

A critical concern when interpreting the data is related to the trustworthiness of the conclusions

or findings. The mixed method relies on the process of gaining assurance of the interpretations

called “triangulation”. Triangulation has to do with redundancy (Guba, 1981; Guba, 1985). It is the


162

comparative analysis and critical review of evidence proceeding from different sources (and, when

applicable, from several experiences with different participants). Each finding needs to have several

confirmations, supported by the data gathered and the context, indicating that key conclusions are

not being overlooked or misinterpreted (Stake, 2005).

It is also important to comment, that although the multicase study has been completed mainly

by a single researcher (the author of this document), other researchers (of the GSIC/EMIC group)

have helped in reviewing the design of the study, the careful preparation of the materials and

deployment of the experiences (cf. Table 6.1), as well as the thoroughness of interpretation. This

“member checking” (Guba, 1985; Stake, 2005) is a crucial technique that contributes to the revision

and improved formulation of the conclusions. However, the complexity of multicase study demands

all the experience to be managed by one person.

The details of each case in addition to their separated analysis and data interpretation are

developed in the next three sections (6.3 for “Collage workshops case study”, 6.4 for “Planet game

case study” and 6.5 for “Network management case study”).

6.3 Case study A: Collage workshops

This section, devoted to the “Collage workshops case study”, is divided into three subsections.

The description of the case study is introduced first. This description includes sites, deployment,

materials, data gathering, interviewees as well as embedded mini-cases (cf. Figure 6.1 and Table

6.1). This is followed by the explanation of the conceptual structure of the case study, which guides

the analysis to the case findings discussed in the last subsection.

6.3.1 Description of the case study

Since the “Collage workshops” is the most critical case study involved in the multicase

(concerning the objectives of this dissertation), it comprises four different experiences as shown in

Table 6.1. Each of the first two experiences, namely GSIC/EMIC and UNFOLD (chronological

order) form a mini-case study. In contrast, UVA (University of Valladolid) and UCA (University of

Cádiz) experiences are the actual focus of the “Collage workshops case study”.

This organization is motivated by the following reason: the audience of the proposal under

evaluation (the pattern-based design process implemented in Collage) includes teachers (mostly at

the university level) interested (but not necessarily experts) in CL and in applying ICT to support

their educational practice. Actually, the majority of the interviewees who participate in the UVA

and UCA experiences are university teachers with a profile that conforms to the target audience.

However, the interviewees of the GSIC and the UNFOLD experiences (mainly educational


163

technologists and CSCL practitioners) are less representative in this sense, but can provide valuable

information to illuminate some of the aspects of the case study.

6.3.1.1 Sites, deployment and materials The sites where the UCA and UVA experiences take place are laboratories at the University of

Cádiz and at the University of Valladolid, respectively. In the UCA experience some of the

participants use the PCs available in the laboratory, while others employ their own laptops. The

participants in the UVA experience use their own laptops. In both experiences, most of the

machines use the MS Windows operating systems (different versions) while one of the laptops runs

Linux.

The deployment and materials used in the experiences are the same. Both consist in 4-hour

wokshops including an intermediate break. Firstly, we make a presentation in order to familiarize

the participants with the context in which Collage is situated, its purpose and the main ideas on

which it is based (including the description of the design process). In fact, the familiarization

activity begins with the illustration of a script created with Collage (“Job interview simulation”, cf.

subsection 5.5.4) and run using CopperCore (Martens et al., 2005), so that the participants visualize

a possible result of what can be produced using Collage. It is important to mention that the

interviewees of the UCA experience achieve a higher level of familiarization since they participate

in several previous sessions about the LD specification (the workshop is part of a larger seminar of

two days).

After the familiarization activity, the interviewees install Collage and create a proposed script

(“CTM2”, cf. subsection 5.5.1) according to the steps guided by the tool and commented by us (two

workshop organizers). The participants have access to the installers of Collage (in a CD and/or on-

line) and to a user manual, including a worksheet that indicates how to complete the CTM2 script.

However, this document is hardly used during the workshops probably because of the (substituting)

presence of the workshop organizers. Besides, the resource needed to create the ready-to-run script

and a summary of the script (in a table) are also available to support the authoring activity (these

materials are available in the attached CD-ROM). Since the creation of the proposed script is quite

repetitive and long (the example is a combination of three CLFPs), we ask the participants if they

prefer to finish the example or have some time to partially create a script that they ideate. In both

experiences, the interviewees choose the second option.

At the end, we validate a completed script (in fact, we prepared the example beforehand in a

kind of laboratory experiment in which we validated and run the script using CopperCore). We also

“open” the UoL using the Reload LDE (University of Bolton, 2004) in order to show the

participants not only that the created script is truly LD compliant but also the fact that many

difficult-to-understand LD elements are transparently generated by Collage.


164

6.3.1.2 Data gathering In this case study we use three different types of data gathering techniques according to the

adopted mixed method (cf. subsection 6.2.3). The techniques represent sources providing data of

quantitative and qualitative nature. The use of the selected techniques is planned in three phases:

before the deployment itself, during the deployment and at the end (after the deployment). The data

sources are (all the data, designed questionnaires, etc. are available in the CD-ROM):

- Questionnaires: two web-based questionnaires (created and published using the Quest tool

(Gómez et al., 2002)). The first questionnaire is completed before the experience, even

before the familiarization phase. This questionnaire is intended to provide information about

the participants’ previous experience, profile, motivations and expectations. The second

questionnaire is answered at the end of the workshops. In this second questionnaire the

participants are asked to value quantitatively (closed questions) and qualitatively (open

questions) different aspects related to the topics of the case study. Therefore, the

questionnaires are designed according to the conceptual structure of the case.

- Observations: in the UVA experience two observers note down the interactions, attitudes

and incidents that occurred during the workshop. After the workshop each observer

elaborates a report exposing this information. One of these observers is also present at the

UCA experience. According to him the observations regarding the UCA experience are not

very significant probably because the participants are more familiarized with the context,

purpose, etc. (the UCA workshop is part of a larger seminar), in such a way that their

reactions or attitudes are not sufficiently relevant to be mentioned. This is the reason why,

we do not have observations reports of the UCA experience.

- Discussion groups: with the aim of gathering detailer information about the aspects of

interest, a researcher (member of the GSIC/EMIC group but not the author of this document)

guides a discussion group at the end of the workshops (due to organization and time

limitations, the discussion group is accomplished after the final questionnaire in the UCA

experience, but before the final questionnaire in the UVA experience). The debate is

recorded and transcribed into a document in order to facilitate its qualitative analysis.

6.3.1.3 Interviewees: UCA university teachers 18 teachers of the University of Cádiz (Spain) participate in the UCA experience. 14 of them

stay during the whole workshop, completing the final questionnaire and contributing in the

discussion group.

The UCA demands a Collage workshop, as part of a larger seminar on “Collaborative learning

and the IMS LD specification”. Their motivation is twofold. On the one hand, the university is

already using a “Virtual Campus” in which some courses are blended, combining face-to-face

activities with distant activities mediated by the “Virtual Campus”. The teachers in charge of the


165

blended courses (some of them are the participants of the UCA experience) are interested in

developing collaborative learning experiences taking advantage of the ICT and educational

strategies. In fact, they have felt this need as a result of their previous experiences using the “Virtual

Campus”. On the other hand, the university is currently considering the guidelines of the future

EEES (Spanish acronym of the European Higher-Education Space), which requires a change in the

teaching-learning methodologies. Since the Collage workshop is last session of the global seminar,

the interviewees already have a general idea of LD when they start using Collage.

The disciplines of the participating teachers are varied: Medicine, Physics, Statistics,

Economics, Nursing, Pedagogy, Chemistry, History, etc. Their experience in terms of years

working as teachers is also diverse, ranging from four to 35 years.

6.3.1.4 Interviewees: UVA university teachers Five teachers of the University of Valladolid (Spain) participate in the whole UVA experience,

completing the final questionnaire and contributing in the discussion group. Other three teachers

attend part of the workshop, answering the initial questionnaire and being part of the noted

observations.

The majority of the participants are involved in projects, funded by the university, regarding the

accomplishment of educational innovations in their own courses. The innovations implicate active

learning methodologies and intend to be a step forward towards the EEES. The Collage workshop is

offered to them as an activity related to these projects.

In this experience, there are two clearly differentiated groups in terms of their disciplines. Four

of the initial participants work at the School of Computer Engineering, teaching Mathematics and

Computer Science. In contrast, the other four interviewees are at the Faculty of Education, teaching

diverse courses related to the Didactics of the Social Science, Foreign Languages or Physical

Education. Their experience in terms of years working as teachers is varied, ranging from two to 15

years.

6.3.1.5 UNFOLD mini-case study UNFOLD is a European project that supports the adoption of open e-Learning standards

(especially LD). The experience formulated as the “UNFOLD mini-case study” arises as an

invitation of UNFOLD to organize a (2-hour) workshop on Collage (http://dspace.ou.nl/handle/

1820/466) in the fifth UNFOLD Communities of Practices meeting (http://www.unfold-

project.net/project/cops) in Glasgow. Nine UNFOLD project (official and invited) members

participate in the workshop and seven of them complete a questionnaire distributed at the end of the

session (the questionnaire and the answers, as well as the material used in the workshop are

available in the CD-ROM). At the beginning of the workshop we briefly present some of the ideas


166

founding Collage. Then, the participants install Collage in their laptops and partially create a script

based on the Pyramid CLFP (“Paper-discussion”, cf. subsection 5.5.3) following the steps guided

by the tool and commented by us (two workshop organizers). The participants have access to the

installers of Collage (in a CD and/or on-line), a user manual, a worksheet and the resource to be

packaged in the ready-to-run script. They do not create the completed script because they prefer to

employ some time to see how we generate a richer example based on a combination of several

CLFPs using Collage (cf. “NNTT” in subsection 5.5.2). Satisfying a request of the participants, we

also validate our examples successfully using CopperCore in order to check if the scripts are LD

compliant. The participants consider themselves: (three) e-Learning specialists, (two) educational

technologists, (one) researchers and (one) instructional designers.

6.3.1.6 GSIC/EMIC mini-case study The “GSIC/EMIC mini-case study” is carried out as the first set of experiences using Collage in

order to provide a preliminary evaluation of the tool. These experiences are two 2-hour workshops

in which members of the GSIC/EMIC research group (to which the author of this document also

belongs). They are also teachers at the University of Valladolid. However, their knowledge of LD is

minor and these experiences form their first contact with Collage. Both experiences take place in

the laboratory of the GSIC/EMIC group during less than two hours. The last minutes are reserved to

fill a web-based questionnaire, and to send us the created scripts for their assessment.

In the first experience, three teachers involved in the course “the Use of ICT in Education”

(whose acronym in Spanish is NNTT) at the Faculty of Education, create a script strongly inspired

in their own course (cf. subsection 5.5.2). Two support persons are available for any question they

have. During the first 15 minutes, the teachers are informed about the task and are provided with a

user manual. Then, they create the example, which is described in a worksheet included in the

manual.

Two Telecommunication Engineering teachers are involved in the second experience, in which

they create scripts reflecting existing educational situations that they have already performed in

their classes. A Collage user manual and a support person are available; however they do not have

any worksheet indicating how to complete the particular scripts. One of the teachers designs a script

reflecting his graduate course of “Advanced Telematic Systems” (whose Spanish acronym is STA).

The other teacher creates a script representing an experience accomplished in his course on

“Network Management" (whose Spanish acronym is CTM2). Significantly, this script is used as the

basis of the “Network Management case study”, which in previous academic years has been carried

out without ICT support. It is important to highlight that those scripts reflect real practices that are

modelled with Collage a posteriori. These and the previously referred scripts are described in

section 5.5.


167

6.3.2 Conceptual structure

The conceptual structure of this case study includes the issue, representing the general research

question of the case, the topics on which we focus the study and the particularization of the topics

into more concrete information questions that guides the data analysis. Some of these information

questions are defined beforehand, a few of them evolve and some new questions emerge as a result

of the data analysis and interpretation. We recapitulate the issue of interest in this case study: does

the design process implemented in Collage facilitate the reuse of CLFPs in the creation of

particularized LD-represented CSCL scripts in a way that allows teachers to focus on CSCL

critical elements? Therefore, we propose the following topics that influence the issue and that

guide its study:

- We claim that our contributions facilitate the reuse of CLFPs, which can be assembled and

refined, when teachers create their own computer-interpretable scripts, instead of starting

from scratch. Therefore, one of the centres of our approach is the proposal of a pattern-

based design process in which CLFPs are integrated in authoring tools as visual LD

templates that can be selected and particularized according to the needs of a concrete

situation.

- Moreover, we state that the proposed design process enables that a conceptualization of the

expected interactions (at the macro level) is made explicit in advance. According to Strijbos

(Strijbos et al., 2004) such a process should focus on CSCL critical elements that affect

interaction, namely: learning objectives, task type, level of pre-structuring, group size and

computer support.

- The design process under evaluation has been implemented in the Collage authoring tool.

While utility is defined as the set of features incorporated by the tool, usability is concerned

with the satisfaction with which users can accomplish the set of design tasks (Kirschner et

al., 2004). Furthermore, since the utility of the process cannot be separated from the

functionality and usability of the tool (Grudin, 1992), we are interested in the use of

Collage.

- Finally, the design process, and thus Collage, is intended for a specific audience: (mostly

university) teachers interested (but not necessarily experts) in collaborative learning and the

use of ICT. Therefore, an important topic of study refers also to the characteristics of the

audience.

The data analysis and interpretation, accomplished in order to reach case findings, is done

around the following information questions that narrow down the topics:

Topic 1: Pattern-based design process:

I. Is the selection of the CLFP-based LD templates and their representation useful and

satisfactory?


168

II. Does the design process achieve a satisfactory trade-off between the reuse of the CLFPs

and the creation of scripts contextualized according to the situational needs?

Topic 2: Focus on CSCL critical elements

III. Does the design process implemented in Collage help to determine the learning

objectives, task-type and expected interaction that will be developed?

IV. Does the design process help to understand and determine the structure regarding the

flow of activities and the hierarchy of groups?

V. Does the design process support also the definition of group-size, resource distribution,

computer support and the structure within activities?

Topic 3: Use of Collage

VI. Can the teachers use Collage successfully?

VII. How can Collage be improved?

Topic 4: Potential audience characteristics

VIII. Which are the characteristics and motivations of the potential audience of Collage?

6.3.3 Case findings

This section introduces the findings which result from the analysis of the data gathered in the

experiences comprising the case study (cf. subsection 6.3.1). This qualitative analysis is supported

by the Nud*IST tool (SQR, 1997). The findings represent the conclusions about the issue

established in the “Collage workshops case study”.

The data gathered from different sources and experiences are triangulated (as it is explained in

subsection 6.2.3) in order to support the credibility of the partial results sustaining the findings,

especially when they are critical or controversial. The integration of the data is done using an

evolving scheme of analysis categories (the final list of categories is available within the “Nudist

project” available in the CD), which are founded on the information questions.

In this way, the partial results, together with their supporting arguments, are listed in a series of

tables collected in Appendix C. Each table corresponds to an information question. In order to

understand from which source the supporting arguments proceed, the tables include the “coding of

the data source” that is generated by Nud*IST (the coding also allows the reader to check the

“original” data (mainly in Spanish), which are available in the CD-ROM).

The findings, and sustaining arguments, are discussed through the following subsections. They

are organized also in line with the conceptual structure of the case, i.e. sequentially answering the

information questions. With the aim of providing a reader-friendly presentation of the findings, the

subsections only reproduce a selection of the most representative supporting arguments, labelled

with the simplified coding shown in Table 6.2. The reader may consult the Appendix C for further

information.


169

Table 6.2 Labels used in the text to quote the data sources of the “Collage workshops case study” (A specific label to quote the scripts created the GSIC/EMIC experience is not included in the table)

Experiences Data source

UCA UVA UNFOLD GSIC/EMIC

Questionnaires [UCA-quest-initial] [UCA-quest-final]

[UVA-quest-initial] [UVA-quest-final]

[GSIC/EMIC]

Observations - [UVA-observer-1] [UVA-observer-2]

[UNFOLD] -

Discussion group [UCA-discussion] [UVA-discussion] - -

6.3.3.1 Finding I: on the selection and representation of CLFP-based LD templates The first stage of the pattern-based design process is the “selection phase”. As the previous

chapter describes in detail, this phase enables the selection of patterns according to the learning

objectives, type of tasks and complexity (in terms of the recommended collaborative learning

experience) associated to the CLFPs. In this phase, the teacher may also read provided “help

information” about the patterns. This information includes an overview, a diagram, use guidelines

and an example.

According to the interviewees participating in the case study, the “selection phase” is critical

and promotes the understanding of the patterns. As one of them mentions “… a minimum formation

on patterns is necessary, for which Collage is helpful [UVA-discussion].” In fact, many

interviewees provide arguments that agree with this declaration of a UCA teacher “Collage

systematizes the selection of patterns [UCA-quest-final].” However, their opinions also indicate that

the suggestion of patterns (as it is implemented in Collage) in terms of their matching with the

educational benefits should be made more flexible. In this sense, two ideas are proposed. A UVA

participant prompts “… there can be patterns that do no strongly serve an objective but that do not

constrain its achievement. Then, these patterns should not be removed from the recommended list

[UVA-discussion].” The characteristics used in the selection of patterns are also reproduced in the

overview of their “help information”. Even so, the interviewees suggest “… I would design the

selection of patterns the other way around. That is, if I select a pattern, then the types of objectives,

types of problems, etc. that can be done with this pattern will be automatically shown [UCA-

discussion].”

On the other hand, there are enough arguments manifesting the significance of the CLFPs in

which the templates provided by Collage are based. Not only is this assertion supported by the

literature on the (best or good) collaborative practices that the patterns formulate, but it is also

corroborated by the interviewees. Qualitative opinions and quantitative ratings provided in the final

questionnaires sustain it. For example, five (out of seven) interviewees select in a closed question of

the [UNFOLD] questionnaire that “the CLFPs are significant, they are relevant examples of CL

techniques” (the other two choose “I do not know any CL technique, but it seems to me that these

are adequate”). Of course, some interviewees indicate that there are other examples of well-known


170

CL strategies: Collage only incorporates six CLFPs and there are more CSCL scripting patterns at

the learning flow and other levels of granularity (cf. Chapter Three). Particularly, a participant

misses patterns for assessment such as a technique that the interviewee usually applies “… I miss a

pattern that I employ in my classes, which is randomly selecting a member of a group to explain

what the group has done [UVA-quest-final]”.

There is not a single possible computational representation of a CLFP as an LD template (cf.

Chapter Four). Besides, the templates can be presented using different graphical visualizations. In

general, the interviewees agree on that most of their ideas about the CL strategies collected in the

CLFPs coincide with what is presented in Collage. The related quantitative ratings are in line with

qualitative opinions such as this: “I think that the patterns are perfectly transferred to the user

workspace, reproducing the needed roles and activities for their execution [GSIC/EMIC].” The

visualizations as well as the interactive possibilities are valued as especially useful. However, the

combination of arguments (cf. Table C.1) also points us to the conclusion that new (drag and drop)

elements, which enable to slightly modify the modelled learning flows, would provide more

flexibility in the design.

6.3.3.2 Finding II: on the trade-off between reuse of patterns and design options The “authoring phase” is the second stage of the pattern-based design process, in which the

teachers reuse the CLFPs by refining them into ready-to-run scripts according to the characteristics

of their particular situations. They also have the possibility of combining the patterns in order to

create richer learning flows. The process is not strictly sequential. Collage provides guidance but it

does not direct the user through a rigid wizard-style set of steps. The steps can be accomplished

however in the order preferred by the user.

The previous finding already suggests that the selection and representation of the CLFPs foster

their reuse. Additional arguments support that the steps of the “authoring phase” facilitate the reuse

of CLFPs when structuring collaborative learning designs. 12 (out of 14) UCA interviewees and

four (out of five) UVA interviewees agree on that Collage helps to reuse the CL strategies proposed

in the CLFPs (the other interviewees selected N/A). “I think that the tool imposes an elaboration

process that impedes the non-reuse” states a UVA participant in [UVA-quest-final].

Significantly, many support data lead us to the conclusion that the combination of patterns

provides outstanding design flexibility. The utility of the combination of CLFPs is rated (in the

range of 1 (it is not useful) to 5 (it is very useful)) with an average of 4.21 by the UCA participants

and 4.20 by the UVA teachers. As one of them affirms “I think that the combination of patterns

allows a better adaptation of the activity to the problems and methods that we want to develop,

making the activity more complete… [UCA-quest-final].” This interesting result is also shared by


171

the UNFOLD interviewees; “… it would be nice if other patterns could be added! However, as

patterns can be combined, these already offer quite a lot of flexibility [UNFOLD].”

Moreover, the Collage templates, which reflect a CLFP or a combination of CLFPs, can be to a

large extent particularized according to the needs of concrete learning situations. A number of

arguments are in line with these statements: “the same structure may be useful for different courses

/ environments, simply changing the resources, the definition of groups, objectives and activities

[UCA-quest-final]”, “it enables the generation of contextualized learning processes…

[GSIC/EMIC]”

Though it would be interesting, it is difficult to carry out specific experiences in order to test if

modifying a complete example that was indented for a different purpose costs more or less effort

than refining a template. Nevertheless, the collected evidences put forward the idea that the pattern-

based templates are probably more useful in the process of customizing a new situation than ready-

to-run scripts, but complete (or partly complete) examples are also helpful. An interviewee

discusses “No example is directly transferable… I think that it is good to know an example in order

to have something like a “demo”… but, at the end, each teacher has his circumstances and

everything changes [UVA-discussion].” We also argue that a design process integrating pattern-

based templates is more systematic than the procedure of modifying examples, which largely

constrains the creativity of the teachers and implies the risk of adopting (not changing) solutions

that do not fit their contexts. On the other hand, if LD viewers are available, examples representing

previous experience would be very useful in particular communities of practice: “the examples are

useful especially if results are available… like in a community of practice… [UVA-discussion] ”

All in all, the data supports that Collage achieves a satisfactory trade-off between flexibility,

keeping the essence captured in the CLFPs, hiding LD-specific technological details and providing

a clear (but limited) set of design options. In the [UCA-discussion] one of the participants says

explicitly “… Collage is flexible and keeps the essence of the patterns. I think that the most

important aspect of the tool is that it allows us to rationalize our labour without the need of much

technological effort…” In fact, the UVA participants, who do not have any knowledge of LD at all,

declare: “… Collage pushes the user to make decisions that any good professional would determine

[UVA-quest-final]”, “… it saves the teacher a lot of specification workload [UVA-quest-final]”, “…

I do not want more flexibility… if we have to apply something, let’s apply it truly… [UVA-

discussion]”, “I do not find the process too constrained since it enables the combination of

patterns… and I think that it is easier for a novice to have an already structured model…[UVA-

discussion].”

On the other hand, since these design steps are not presented in a rigorous sequence, some of

them may be forgotten. Regarding the fact that the design process implemented in Collage does not


172

enforce a sequence of authoring steps, in the [UVA-discussion] is discussed, “… all the parameters

are at the same level of importance… it is necessary to manipulate all at the same time when we are

making the design decisions”. However, it derives the problem pointed out in [UNFOLD]: “the

number of things to be done and the various places where they have to be put make this tricky for

teachers…” To overcome this difficulty, Collage incorporates a table, which can be consulted

anytime, that indicates what has been already refined and what needs to be completed. This

functionality is appreciated by the UVA interviewees (cf. Table C.2).

6.3.3.3 Finding III: focus on learning objectives and task type Two of the CSCL critical elements that affect interaction and which condition the design of the

other elements (Strijbos et al., 2004) are the learning objectives and the task type. Our inquiry

concludes that the proposed design process helps to determine the learning objectives related to

collaborative learning that will be promoted and to select the task-type that will be solved by the

students. As Finding I indicates, this is supported by the “selection phase”, which on the other hand

should be made more flexible (cf. subsection 6.3.3.1). Quantitatively, the help provided (by the

selection of CLFPs and the associated information) to determine the learning objectives is rated (in

the range of 1 (it does not help) to 5 (it helps a lot)) with an average of 3.86 by the UCA

interviewees and of 3.40 by the UVA participants. Similarly, the help to determine the task type is

rated with an average of 3.43 in the UCA workshop and of 3.20 in the UVA experience. In this

sense, there are also qualitative opinions: “in this way we are not just reflecting on what we are

already doing in our practice… but also developing transversal competences and other things…

[UVA-discussion]”, “it helps to determine the CLFP more suitable to foster particular objectives

[UVA-quest-final].”

In addition, the UCA and UVA interviewees rate with an average higher than 3.60 (same range)

the extent to which the design process helps to determine the expected interaction (discussing,

reasoning…). As one statement, whose source is [UCA-quest-final], summarizes, “it helps to

envisage how the interactions will develop...”

6.3.3.4 Finding IV: focus on the structure regarding the activity flow and group hierarchy Due to the type of patterns that are incorporated into Collage (i.e. CLFPs), its design process is

specially focused on structuring the flow of (collaborative) learning activities and the hierarchy of

groups necessarily associated to the flow. As a pedagogy teacher participating in the [GSIC/EMIC]

experience says “Collage is useful because it helps to think in terms of collaborative learning and

its previous arrangement”.

Finding I anticipates that the “selection phase” and the representation of the CLFPs successfully

facilitate the understanding of the learning flow that they propose. Besides, the gathered data value

as very satisfactory the support that the process provides to determine the structure of the activity


173

flow. One initial hint is the high average with which the UVA participants rate this aspect (4.40 in

the range of 1 to 5, where 5 indicates that Collage helps a lot to determine the activity flow).

Arguments of the UCA interviewees are also consistent with this result: “I think that the most

important aspect of Collage is that it helps to structure and systematize the different types of

activities… Collage has helped me to structure what I already do but I do not organize well [UCA-

discussion]”, “it highlights the inter-relation among activities [UCA-quest-final].”

The CLFP-base templates specify which activities are performed by the different groups, saving

the teacher this task. One of the interviewees states, “I find the tool quite powerful, especially

concerning the association of activities to groups [UCA-quest-final].” Accordingly, the participants

agree on that the design process helps to understand and determine the structure regarding the

hierarchy of groups. “The structure of the (learning) process helps to check the roles that each

group plays in each moment”, affirms a UCA participant in [UCA-quest-final]. Another interviewee

adds “… thanks to the graphical representations, Collage helps to conceptualize the groups…

[UVA-discussion]”

6.3.3.5 Finding V: focus on group size and support within the activities The structure of groups should be complemented with a plan regarding the group size limits,

though the actual association of persons to groups (and the definition of the number of groups) is

not decided until the instantiation of the designed LD script (cf. Chapter Five). The workshop

participants acknowledge that the design process helps to establish the group size. For example, one

of their arguments is, “the information help provides clues and the interface of Collage allows the

introduction of the desired group size limits [UCA-quest-final].” We admit however that the group

size and the elements related to the support within activities are more weakly considered in the

design process implemented in Collage than the elements regarding the macro-level structure

discussed in Finding IV (due to the scope of this dissertation, focused on macro-scripts). In this

sense, the determination of the group size limits can be improved, for instance, with automatic

checkups as an interviewee mentions [UCA-quest-final].

Nevertheless, the participants concede a good rate (higher than 3.60 in a range of 1 to 5) to the

help that Collage provides to determine the distribution of resources among activities as well as to

the computer support of the activities. UCA and UVA participants agree: “the design implies the

previous determination of all these elements and their organization [UCA-quest-final]”, “the design

of a unit of learning using Collage includes all these aspects. Since Collage guides the design

process and the activities of each phase, we are obliged to reflect on each element that shapes the

activity… [UVA-quest-final]” But, as expected, they miss the fact that Collage does not provide

suggestions of recommended content or tools: “Collage pushes to specify resources and tools,

though their characteristics should be known beforehand [UVA-quest-final].”


174

Surprisingly, the UVA interviewees rate very highly the extent to which Collage helps to

structure of each activity (average of 4.80 and a low deviation of 0.40 in the range of 1 to 5). This

rate is understood when analysing their qualitative opinions, such as “this information should be

clear when thinking and describing each activity [UVA-quest-final].” They are actually not

assuming that the described structure will be computationally interpreted by the LMS but simply

presented to the students. Therefore, we can conclude that the design process helps to describe each

activity and its eventual (textually-defined) structure according to the envisaged expected

interaction (cf. Finding III).

6.3.3.6 Finding VI: Collage, easier-to-use CL-specific LD editor The discussion of the previous findings already manifests that the utility of the design process

cannot be separately evaluated from the functionality and usability of the tool that implements it

(Collage). Nonetheless, the use of Collage deserves a differentiated attention.

Most of the participants find Collage user-friendly and intuitive. 12 (out of 19) UCA and UVA

interviewees rate as “easy” (among the possibilities of “easy”, “acceptable” and “difficult”) the use

of Collage, five assess it as “acceptable” and only two rate it as “difficult”. “I am satisfactorily

astonished by the user-friendliness of the tool; I thought that it was going to be more difficult…

[UCA-quest-final]” indicates a UCA participant, who has the opportunity to hear of LD and to see

the Reload LDE tool (University of Bolton, 2004) before the Collage workshop. A UVA

interviewee also mentions “Collage is intuitive, user-friendly and does not provoke an “insecurity”

feeling [UVA-quest-final].” The observations accomplished during the UVA experience are in line

with this conclusion: “… during the experience the participants affirm the user-friendliness of the

tool [UVA-observer-1]”, “... the participants create the design without difficulties… [UVA-

observer-2]”

In effect, the participants are able to create an almost completed example during the workshops.

They do not finish because of time limits, but they have the feeling that they could complete the

examples without problems. As a UCA interviewee declares “I have not finished it completely

because of a lack of time, but it has worked correctly…” Due to the particular characteristics of the

GSIC/EMIC experience (c.f. subsection 6.3.1.6), its participants do complete their scripts. The

scripts are correctly validated by CopperCore. Moreover, the scripts created by the pedagogy

teachers largely describe the learning situation of the proposed example (cf. Table 6.1). However,

they fail to particularize a specific activity, i.e. they do not specify a description or associate

resources to an activity. This also supports the last partial result of Finding IV, which leaded us to

extend Collage with a table indicating the user which activities have not been completed (cf.

subsection 5.4.5.3). The engineering teachers are also satisfied with their experience using Collage.

“I was able to create an IMS LD reflecting a previous CL experience without deep knowledge on


175

IMS LD” states one of them. While the other affirms “I have the feeling of having done something

useful and real.”

Very few problems appear during the workshops. UNFOLD interviewees pinpoint: “… only

minor bugs that you know about…”, “… navigation in the expansion window is confusing, needs

time to familiarize.” UCA and UVA participants demand: “it is not possible to delete a pattern once

it is added… [UVA-quest-final]”, “I would like that the number of the pyramid levels could be

modified [UCA-quest-final].” Additionally, only some interviewees add the importance of applying

the scripts created using Collage with students, as it is done in the “Network Management case

study”, in order to assess whether new problems appear.

On the whole and compared to other LD compliant tools, Collage is easier to use, specific to

collaborative learning, and it is the first editor providing pattern-based templates. All the UNFOLD

and GSIC/EMIC interviewees answer “yes” to a closed question asking if they find Collage useful.

Besides, the UNFOLD participants, most of who have used other LD compliant tool, argue: “at last,

patterns in practice!”, “Collage seems to add some important features which will facilitate that

teachers access LD authoring tools”, “less complicated and more intuitive than Reload on its

own…”, “makes Reload much easier to use for specific purposes .”

6.3.3.7 Finding VII: improvements associated to selection of patterns, further design options and integration with complimentary tools

The preceding findings elucidate directions in which Collage can be improved. This Finding

VII reviews these ideas and formulates new ones.

Finding I illuminates the need of a more flexible selection of patterns. In addition, more CLFPs

should be incorporated as well as other types of patterns, including other (not necessarily pattern-

based) elements, which would provide more flexible design options. The interviewees assert: “… it

may happen that advanced users cannot specify things that they would like [UVA-quest-final]”, “…

when I apply the Jigsaw, I also apply “continuous” evaluation in order to make the students

participate well [UVA-discussion]”, “the incorporation of activities that are not part of a pattern

but that may be complementary…[GSIC/EMIC]” Of course, we agree on that these extensions are

very desirable as long as the trade-off concerning a good reuse of the patterns and an easy edition is

preserved. Further research is needed in this sense. A first step that will increase the design options

is that Collage supports the concatenation of CLFPs, which is considered in the design process (cf.

Chapter Five) but is not implemented in Collage yet. Meanwhile, advanced users (with LD

knowledge) still have the possibility of modifying the Collage scripts with lower-level LD editors

such as Reload.

A table indicating which activities have not been completed was added to Collage as a result of

the need illuminated by Finding II of providing tracking information about what has been already


176

refined. This table is also useful to show a “preliminary view” of the script as a “lesson plan”, as it

is demanded by the participants, “I would suggest that the tool generates a document with the

resulting design… [UCA-discussion]”

Additional improvements regarding the edition possibilities include the automatic calculation of

group size (cf. Finding V) and the option of changing a resource in “one step”. A UNFOLD

interviewees proposes “make a “search and replace” that clones a UoL with different resources.”

This functionality is especially helpful if a resource is associated to various activities. When asking

for suggestions to improve Collage, the support of LD levels B and C is obviously pointed out by

the LD-aware interviewees. We argue that this extension should be done carefully and by means of

“hiding” the complexity of the additional LD constructs in reusable (pedagogically-founded)

elements (e.g. a mechanism that enables teacher to connect students with specific activities, or

groups, according to their previous outcomes).

Another important area of improvement is the integration of Collage with complementary

needed tools (production and delivery systems), which should support flexibility (e.g. changes on

the fly). A user-friendly tool that enables the instantiation of the scripts is required. This tool should

guide the teacher in the creation of the needed number of groups and their population with the

actual participants, according to the group hierarchy specified in the CLFP-based script. As a UVA

interviewee indicates “… in order to have the complete view the assignment of participants to roles

is missing… [UVA-quest-final]” The need of supporting flexibility is largely discussed by the UVA

participants, which are reflecting on how their own practice develops. “Before the fall semester we

plan the objectives, activities and timing of the courses. However, the planning cannot be closed

because many times external circumstances appear (e.g. holidays, illnesses)” declares an

interviewee in [UVA-quest-initial]. Along the [UVA-discussion] we debate this problem: “… if

some students abandon the course in the third session, then I need to re-structure… ”, “… the

structure can be kept if the student number variation is within a range…” The debate includes the

idea that the required changes may depend on the characteristics of the planned script (i.e.

objectives, duration, etc.): “in the TTG course (Hernández-Leo et al., 2006)... we apply a Jigsaw

and in some sessions an expert group does not attend the session… then we just form a new group

comprising two jigsaw groups”, “there are other cases… in which a Jigsaw is applied along the

whole course… then since the objectives are at the long-term the problem is no so important

because at the end they have to contribute...”

Finally, the information reflected in the resulting scripts is also useful for the students, what

decreases the teacher’s workload. Apart from the fact that the scripts can be interpreted by an LMS

that guides the students, the information and the visual representation of Collage scripts may largely

help students to be aware of the learning design beforehand and to follow their own progress. As a

UCA interviewee pinpoints “since the structure is clearly planned, it may also facilitate that the


177

students know it well and can follow it easily… so that the workload of the teacher

decreases…[UCA-discussion]”

6.3.3.8 Finding VIII: audience interested in CL and in the use of ICT Having reached this point, it is necessary to tackle whether the gathered data supports that the

characteristics of the potential audience coincides with the target users of the proposed design

process (and Collage). The previous results, but mainly Finding VI, are consistent with our

objectives. However this finding deepens in this topic.

The motivation of the UCA and UVA interviewees, who decide to participate in the Collage

workshop (cf. subsections 6.3.1.3 and 6.3.1.4), together with their own arguments allows us to

conclude that the audience of Collage should be interested in designing CL processes to be used

with an LMS in face-to-face, distance or blended situations or as lesson plans (without being

interpreted by a player). The changes imposed by the European convergence to the higher

educational system (EEES) largely motivate the teachers to be interested in Collage. Many UCA

participants refer to it with statements like these: “it is going to be necessary to use this type of tools

in our educational system. The EEES will also push to use these tools [UCA-final-quest]”, “Collage

is very well designed for collaborative learning and it seems that we need to get used to it hereafter

[UCA-final-quest].” UVA participants are also committed to reviewing their educational methods

according to the European guidelines, which highlight the importance of CL processes. As one of

them states “I think that the tool can be of great utility in order to apply this kind of techniques, now

that it seems one of the methods to apply in our classes … [UVA-quest-final]” The proposed design

process guides the teachers in the creation of CSCL scripts that offer new educational opportunities.

As an interviewee explains “I currently use the ICT in virtual education with individual learning

and applying collaborative learning just in face-to-face situations… It is a good idea to blend both

aspects, it broadens the possibilities of work and its control [UCA-quest-final].” The participants

insist on the usefulness of Collage, even though the user does not aim at running the scripts in an

LD-compliant player: “… once we have the design, we can reproduce it in Moddle [UCA-

discussion]”, “… the design… can be also applied without computer support [UVA-discussion].”

With regard to the CL experience required to exploit the design process, we can conclude that it

best serves (university) teachers, novice or with experience in CL practice. A UNFOLD interviewee

exclaims “patterns are quite adequate and for non-experts on CL this is really and eye-opener!”

This is corroborated by UCA and UVA statements: “for the novice teachers, a tool that guides

users in the design process of units of learning is essential [UVA-quest-final]”, “it enables the

direct implementation of collaborative learning systems without a great expertise [UCA-quest-

final].” Teachers already practicing CL also appreciate the support of Collage: “the patterns help

me to systematize ideas that I applied before without a theoretical basis… [UCA-quest-final]”, “I


178

have accomplished some CL experiences, but I do not consider myself an expert on this topic…

However, I have used satisfactorily some of the patterns, what leads me to think that the tool would

be useful for persons with knowledge similar to mine [UVA-quest-final].” The experience of the

students that will play the script is also considered by the design process. As an interviewee

discusses “it is not the same to use a Jigsaw than a Pyramid, it depends on the level of the student

compromise, the collaborative learning experience… then the tool also suggests: if you are starting

practicing collaborative… and the students do not seem to be prepared to collaborate, then use the

TPS, which is easier… [UVA-discussion]”

In accordance with the data, the design process provides ideas and fosters the design creativity

of the users, even promoting an increase in their interest in CL and the application of ICT. “I was

already very interested in CL, however the workshop has broadened my perspectives… because I

see that Collage facilitates the design of complex collaborative activities” indicates a participant in

[UVA-quest-final]. A statement provided in the [UVA-discussion] adds “… but the user is in

charge of knowing what he wants to design and how… The tool helps of course, but the user is who

defines…” Besides, a GSIC/EMIC interviewee affirms “the experience was quite dynamic and

offered me new design ideas.”

Findings II and VI illuminates to a large extent the conclusion that we discuss next. The

potential audience of Collage requires some experience and interest in the use of ICT, but it is not

necessary to be expert (LD) technologists. A minimum familiarization with ICT is recommended in

order to use Collage. As one of the UVA observers notes down “the participant with more

difficulties is UVA-participant-1, who is a teacher of the Faculty of Education that has very few

knowledge of Informatics [UVA-observer-2].” A GSIC/EMIC pedagogy teacher argues “Collage is

an interesting tool, though I think that in order to use it a minimum training and some

familiarization with the use of ICT are needed.” However, the experiences comprising the “Collage

workshops case study” prove that the majority of the participants, without being LD experts, can

easily use Collage. This is probably because they usually work with ICT and they are motivated to

apply these technologies in their classes. In fact, the UCA and UVA interviewees think in general

that ICT will play an important role in their future educational practice, what fosters their interest in

using ICT. In the [UCA-initial-quest] two participants anticipate: “I am interested in discovering

tools that I could apply in my virtual courses”, “I would like to have tools that facilitate the design,

implementation and evaluation of collaborative activities.”

Case studies B and C provide further evidences related to some of the findings of this case

study. Section 6.6 aggregates the main conclusions and formulates the global assertions resulting

from the multicase study.


179

6.4 Case study B: Planet game

In the previous case study, the involved scripts are proposed by members of our research group.

In the “Planet game case study” the key added value is that we evaluate the CLFP-based design

process implemented in Collage with regard to the functioning “solving a third party scenario”.

This section elaborates on the description of the case study summarized in Figure 6.1 and Table 6.1.

As in case study A, this is followed by the conceptual structure of the case study. The section also

finishes with the findings that result from the analysis of the case study according to its conceptual

structure.


This case study entails the participation in a workshop integrated in the ICALT 2006 conference

(Vignollet et al., 2006). The workshop is entitled “Comparing educational modelling languages on

a case study”. The motivation of the organizers, as they indicate in the workshop description,

concerns with whether IMS LD (IMS LD is used in this section instead of LD to distinguish it from

related proposals) can be satisfactorily used to model CSCL situations. They state that it is not

always easy to model a CSCL situation using IMS LD and that available examples are still rare.

Since several researchers adopt IMS LD, some of them using different compliant tools, and others

propose alternative approaches, the aim of the workshop is to share and confront the approaches

through a common scenario.

6.4.1.1 Sites, deployment and materials The organizers of the workshop propose a challenge consisting on implementing a scenario (a

“Planet Game”) that they anticipate. The narrative of the scenario is shown in Table 6.3. The

scenario is part of a real learning situation in astronomy. Students have the same problem to solve.

They are divided into two teams. Each team has only a part of the domain’s knowledge and data

required to solve the problem. So, they need to cooperate exchanging information. Finally, each

student has to classify the planets with respect to their distance from the Sun. Students have at their

disposal: two expert interviews, one for each team; a questionnaire given to each student at the end

of the game (to indicate the results, i.e. the association <planet –position relative to the Sun>); one

forum for both teams; and two chat rooms, one for each team.

The deployment of the workshop is organized in three phases. First, the participants apply their

approaches to solve the scenario before the workshop. In this phase, apart from modelling and

implementing the scenario, they are encouraged to analyze the possibilities of their approach

regarding observation (how could the script be observed at runtime?), traces (are traces produced at

run-time?) and re-use/adaptation (how could the script be adapted for a different topic, keeping the

same general structure?). As a result, each participant writes two (brief) papers to be published in


180

the conference proceedings. One of them is a “panel paper” devoted to expose their general

approach. The other one is a “workshop paper” where the application of their approach to solve the

scenario is explained.

Table 6.3 Narrative of the Planet Game scenario proposed to the participants of the workshop (Vignollet et al., 2006)

The learners have to classify the planets with respect to their distance from the Sun (from the nearest one to the most distant). They are split into two teams (Team A and Team B). Clue distribution: Team A knows the planets’ properties (taken from expert interviews): they can deduce the planets’ order, but they don’t know the planets’ names; Team B knows the planets’ names and some properties (taken from other expert interviews), however, many properties are missing. The teams have to cooperate using a forum to exchange information. They must, at the very least, associate the names of the planets to their position relative to the Sun. Each team can use a chat room to enable their members to have a discussion. The teacher has access to the forum, and can intervene in the discussions in it. S/he can also add new clues to either of the expert interviews. The teacher decides when the exchanges are finished. Then, each learner fills in a questionnaire about the planet classification. The winner is the one who gives the right associations : <Planet – Order>. The activity finishes when a winner is nominated

Therefore, the second phase of the deployment of the workshop is a (2 hours and a half) session

in which participants present their approaches and put forward how they tackle the script. The day

after, a further (1 hour and a half) session entails a discussion comparing the approaches on the

different aspects. Since both sessions (a workshop and a panel session) are part of the ICALT 2006

conference, they take place in conference rooms of an abbey in Kerkrade, The Netherlands, where

the conference is held.

6.4.1.2 Data gathering and participants Nine participants proposing different approaches to solve the scenario contribute in the

workshop. Table 6.4 lists these participants along with the language used to specify the script and

the tools employed to author and execute them.

Due to the particular characteristics of this case study, we use three types of data sources (the

papers are published in ICALT 2006 conference proceedings. The video and the Collage UoL are

included in the attached CD-ROM. (The UoL is also available at http://gsic.tel.uva.es/collage/

l3astronomy.html):

- Papers written by the participants regarding their general approach (panel papers) and their

application to the script proposed by the workshop organizers (workshop papers).

- Regarding our approach, the UoL package created with Collage is also available.

Consequently, the conclusions and screenshots of resulting from its design and execution are

also used as supporting data.

- A video that records the session in which the discussion takes place.


181

Table 6.4 Participants in the panel and the workshop under analysis in the “Planet game case study”

Participants Institution Language Authoring

tool Execution

environment

(Hernández-Leo et al., 2006b)

GSIC/EMIC group, University of Valladolid, Spain IMS LD Collage Gridcole

(Tattersall, 2006a) OTEC, Open University of the Netherlands, The Netherlands IMS LD Reload LDE CopperCore

(Paquette & Léonard, 2006)

LICEF-Télé-université, Québec, Canada IMS LD MOT+LD -

(Amorim, Lama, & Sánchez, 2006a)

University of Santiago de Compostela, Spain

IMS LD / F-logic Reload LDE CopperCore

(Dalziel, 2006b) Macquarie University, Sydney, Australia LAMS LD LAMS

(Martel, Vignollet, & Ferraris, 2006)

Syscom lab, University of Savoie, France LDL ModX LDI

(Dufresne, 2006b) University of Montréal, Canada Explor@Graph

(Nodenot & Laforcade, 2006b)

LIUPPA-Bayonne /

LIUM-Laval, France CPM Objecteering and

UML profile -

(David, Lejeune, & Villiot-Leclercq, 2006) CLIPS-IMAG, France Natural

language Paper/UML tools -

As indicated in subsection 6.2.3, the data is critically reviewed and comparatively analyzed in

order to provide trustworthy findings in accordance with the conceptual structure in which the case

is outlined.


Though this case study is highly related to the quintain that we seek to understand in the

evaluation phase of this dissertation, as the other two case studies it has its own conceptual

structure. Its distinctive functioning together with its context makes it special and unique. Defining

the conceptual structure of this case study implies focusing attention on its singularity and

complexity.

The issue that this case study illuminates is: can we use Collage for creating a script

representing a scenario proposed by a third-party? Within the subjects tackled in the workshop

the most important concerning the objectives of this dissertation are: computational representation,

design and re-use. However, other subjects related to the enactment are also relevant since they

indicate whether the created script can be satisfactorily enacted. Therefore, the topics on which we

focus the study are:

- In the “Collage workshops case study” the scripts are proposed by members of our research

group. In this sense, it is interesting and necessary to test the application of our pattern-

based design process for the creation of LD scripts, as it is implemented in Collage, to a

scenario proposed by a third party.


182

- Besides, this case study offers the opportunity to understand the pros and cons of our

approach compared with the related approaches that participate in the workshop. The case

findings related to this topic represent important evidences of our proposal contributions.

The concrete information questions that derive from this topic are:

Topic 1: Application of our approach to a scenario proposed by a third-party

I. To which extent is it possible to design a script proposed by a third party using Collage?

II. Can the script created with Collage be enacted by an actual LMS?

Topic 2: Comparison of our proposal with related approaches

III. What are the pros and cons of our approach compared to other approaches regarding

computational representations?

IV. What are the pros and cons of our approach compared to other approaches regarding

design?

V. What are the pros and cons of our approach compared to other approaches regarding

enactment?

VI. What are the pros and cons of our approach compared to other approaches regarding

observation and trails aspects?

VII. What are the pros and cons of our approach compared to other approaches regarding re-

use/adaptation aspects?

6.4.3 Case findings

Table 6.5 shows the labels of the data sources from which the arguments supporting the case

findings are extracted. The findings are discussed through this subsection which is, once more,

organized consistent with the conceptual structure of the case study.

Table 6.5 Labels used in the text to quote the data sources of the “Planet game case study”

Data source Participants

Panel papers Workshop papers Recorded discussion

IMS LD, Collage, Gridcole (Hernández-Leo et al., 2006c) (Hernández-Leo et al., 2006b)

IMS LD, Reload LDE, CopperCore (Tattersall, 2006b) (Tattersall, 2006a) IMS LD, MOT+LD, Reload Player or

TELOS (Paquette, 2006) (Paquette et al., 2006)

F-logic, Reload Editor, Coppercore (Amorim, Lama, & Sánchez,

2006b) (Amorim et al., 2006a)

LAMS LD, LAMS (Dalziel, 2006a) (Dalziel, 2006b)

LDL, ModX, LDI (Martel, Vignollet, Ferraris, David, & Lejeune, 2006a) (Martel et al., 2006)

Explor@Graph (Dufresne, 2006a) (Dufresne, 2006b)

CPM, Objecteering and UML Profile (Nodenot & Laforcade, 2006a) (Nodenot et al., 2006b)

Paper/UML tools - (David et al., 2006)

[Video]


183

6.4.3.1 Finding I: the main aspects of the script can be designed with Collage It is possible to apply our pattern-based design process for the creation of LD scripts, as it is

implemented in Collage, to designing the script described in Table 6.3 (Hernández-Leo et al.,

2006b). Though the script is not rigorously a JIGSAW-based situation (students do not collaborate to

jointly solve a problem but they “compete to propose individually the solution”), its learning flow

structure is inspired in its essence. That is, the script considers a “Jigsaw group”, which is the whole

class, that is divided into two “Expert groups” representing teams A and team B, each of which

accesses complementary information.

Therefore, the IMS LD template representing JIGSAW CLFP can be selected in Collage and

particularized as illustrated in Figure 6.2 and Figure 6.3.

Figure 6.2 Authoring the script with Collage

The “Individual” phase of JIGSAW is devoted to present the rules of the Planet Game and clue

distribution depending on the team to which each student belongs. In this sense, it should be noticed

that although the expert group phase of the JIGSAW is not strictly considered in this scenario (not

visible), the corresponding expert-group role must exist to differentiate between members of team A

and team B. This is needed for providing the right expert interview (through a shared document

repository) and the specific chat room in the discussion activity of the “Jigsaw Group” phase. We

model both tools using a general definition for group services as discussed in subsection 4.5.2 of

Chapter Four. The particular solution adopted in this script regarding a general way of specifying a

group-service (not necessarily dedicated to conferences) is using the conference service element

of IMS LD and an external binding document that indicates which groups need a different instance

of the service (Bote-Lorenzo, 2005). Therefore, a differentiated instance of the chat and the shared


184

repository will be available only to the members of a particular team (each team is an instance of

the expert-group role, thanks to the use of the created-new attribute of the IMS LD role element in

the JIGSAW-based template implemented in Collage, cf. Chapter Four). Each instance of the group-

service that models the repository storing the interviews will be also available to the teacher, so that

s/he can add new clues.

Figure 6.3 Refining the JIGSAW-based template with the description of the activities and the collaborative

tool supporting the activities

Table 6.6 Summary of the UoL (based on JIGSAW CLFP) created using Collage JIGSAW

CLFP phase

Group/ role

Activity Activity description Resources

Jigsaw group

Individual study

At the end of this game you have to be able to classify the planets according to their distance to the Sun (from the nearest one to the most distant). Extract planets’ properties from the assigned expert interview. (Team A members’ interview contains planets’ order and some properties (without names) and team B’s interview informs about planet’s names and some properties.)

*expert_interview

Individual study

Teacher Activity control You have privileged access to the expert interviews *expert_interview

Expert group

Subproblem

Teacher

Empty! (NOT VISIBLE) Global discussion

Cooperate with the other team using a forum to exchange information. Each team can use a chat to discuss.

*forum *chat *expert_interview

Jigsaw group

Solution proposal

Fill in (individually) a questionnaire about the planet classification.

*questionnaire tool

Global problem

Teacher Activity control

You have access to the forum, and you can participate to discussions. You can also add new clues in any expert interview. You have to nominate a winner according to the questionnaires.

*questionnaire tool *chat *expert_interview


185

An analogous approach is adopted for the forum, which is available to all the participants, and

the questionnaire tool that will be answered in the last activity (solution proposal) of the JIGSAW.

Table 6.6 shows a summary of the resulting UoL as created by Collage.

Concluding, the main aspects of the script can be modelled with Collage. There are only two

details that cannot be rigorously authored with Collage. It is not possible to specify that “the

teacher decides when the exchanges are finished (cf. Table 6.3)” because the IMS LD elements that

enable its computational representation (the teacher makes the activity visible setting a property or

the act is completed only when the teacher finishes her/his activity) are not included in the JIGSAW-

based template. However, it is possible to add the necessary IMS LD constructs to the script using

Reload LDE or another non-constrained IMS LD compliant editor (even a plain XML or text

editor). Similarly, an additional activity to describe that “the game finishes when a winner is

nominated” cannot be added in the current version of Collage. Yet, this possibility is considered in

the proposed design process. Anyway, this can be solved by simply using the forum for the

nomination of the winner or by modifying the script with Reload LDE, for example. Instead of

using a new activity to model this requirement, it is possible to design it as the feedback (viewing a

property value) of the solution proposal activity of the last phase of JIGSAW.

6.4.3.2 Finding II: the script created with Collage can be enacted using Gridcole system Gridcole system (Bote-Lorenzo, 2005), which includes the CopperCore IMS LD engine

(Vogten et al., 2006), is capable of interpreting the UoL created using Collage. Making use of the

UoL and an external binding document that indicates which groups need a different instance of a

service (as mentioned in the previous section), Gridcole provides the required service instance to

users. Therefore, this system guides users through the flow of collaborative learning activities

integrating the tools needed to support them. In this scenario (cf. Table 6.6) the selected

collaborative tools are the GSIC-UVA chat, Synergeia (ITCOLE, 2005) (a shared repository for the

interviews and the forum) and Quest (Gómez et al., 2002) (for the final questionnaires). Figure 6.4

is a screenshot of Gridcole giving access, to a student of team A, to the repository folder that

contains the expert interview assigned to her/his team. The top left frame of the interface indicates

the sequence of activities that should be performed by the user. If the user clicks on the name of the

activity, its description is shown in the right frame. Also, in the bottom left frame students can see

the documents and tools available for the support of the activity (in this shared repository). When

the user selects a web-based tool or a document, the selected resource is provided by the system

using the right-hand frame.


186

Figure 6.4 Enacting the Planet game script created with Collage using Gridcole integrating a shared

repository: clue distribution

Figure 6.5 shows how Gridcole makes available the common forum for exchanging information

with the other team and the particular chat room that students can use to discuss with their team’s

members. In this case the integrated chat is a grid service-based tool which is installed, configured

and launched (as a Java client in the user’s machine) by Gridcole.

Figure 6.5 Enacting the Planet game script created with Collage using Gridcole integrating a discussion

forum and a chat: cooperative phase


187

Similarly, Gridcole provides direct access to the questionnaire published in Quest in such a way

that students can use immediately the tool to indicate the answer of the game (cf. Figure 6.6).

Figure 6.6 Enacting the Planet game script created with Collage using Gridcole integrating a questionnaire

tool: proposing the solution

6.4.3.3 Finding III: on the comparison regarding the computational representation It is straightforward to compare our proposal with related approaches defended in the

workshop since they are applied to the same example. In this sense, the contributions of (Tattersall,

2006a; Paquette et al., 2006; Amorim et al., 2006a) confirm our statement (Hernández-Leo et al.,

2006b) that IMS LD supports the implementation of this script, with the interoperability advantages

that it implies. However, it is worth mentioning that the way of modelling the script using IMS LD

notation diverges. This shows the many possibilities of the specification which is flexible enough to

describe scripts with the same core design but with different details open to author interpretations,

intentions, authoring tool design constraints or features of the available runtime systems.

For example, while Tattersall (2006a), Amorim et al. (2006a) and Hernández-Leo et al. (2006b)

use two IMS LD acts to model the script, Paquette et al. (2006) employ four acts: “Using the IMS-

LD terminology, we have divided the proposed collaborative scenario into four acts (Paquette et al.,

2006).” In this case, the reason of the divergence is author interpretations (e.g. the authors prefer to

separate into different acts the phase of answering the questionnaire and the phase of announcing

the result). Another example refers to the definition of roles. We do not follow the solution adopted


188

by the other participants using IMS LD who define a role for each team at design time, “The

approach adopted makes uses of […] a role for each of the teams, Team A and Team B. (Tattersall,

2006a)”. In contrast, the JIGSAW-based template, used as a basis to create the script with Collage,

undertakes a challenge that is also pointed out by Tattersall: “One interesting challenge with respect

to the approach is to generalize to several teams depending on the cohort size. As the approach

stands, the number of roles is fixed, but a solution which allowed any number of teams would

clearly require a different approach (Tattersall, 2006a).”

In effect, as discussed in Finding I only one role (expert-group) is defined in the UoL created

with Collage and the two occurrences of the role (Team A and Team B) are determined when the

script is instantiated. This is a possibility enabled by the attribute create-new set to “allowed” of

the IMS LD role that incorporates the JIGSAW-based template as implemented in Collage. Fixing

the number of groups at design time would be also possible if this template is implemented as

SIMULATION-based template, which incorporates a mechanism that automatically creates and

includes in the template a new role for each actual group (cf. sections 5.3.2.3 and 5.4.4.2). Actually,

these templates are modelled differently and implemented in Collage to illustrate the two possible

ways of modelling groups that accomplish the same types of activities. Of course each possibility

has pros and cons compared to the other. In the Planet Game script, if the concrete roles are

specified in the UoL, then clue distribution can be modelled as learning objects within

environments that are linked to the concrete activities associated to each role, “The expert

interviews are seen as Learning Objects (Tattersall, 2006a).” The determination of the actual

number of groups at instantiation time provides flexibility and generality, however a service making

available the clues to users depending on their group is necessary. In this sense, we include a

reference to this service during the design (shared folder with the interview in a repository)

generalizing the way of specifying a collaborative tool (cf. Finding I).

On the other hand, we also use an external tool for managing the questionnaires (Hernández-

Leo et al., 2006b). In this sense, the questionnaires are designed separately and they are not

included in the UoL package. Since Collage is currently only IMS LD level A compliant, we cannot

use IMS LD level B properties or IMS QTI specification (IMS, 2006) for creating the

questionnaire, as it is pinpointed by Tattersall (2006a) “each user provides an answer (via an IMS

LD locpers-property) to the ordering and naming question […] the questionnaire could be

implemented as a QTI item”. Though these solutions are probably more convenient in terms of

interoperability, using external tools we take advantage of their special functionalities (e.g. statistics

of the answers, adding comments to the expert interviews) and of the familiarity that users may

already have with the tools.

Only few details cannot be formally expressed using the LD notation itself, such as the

automatic random allocation of participants to groups, which on the other hand it is not required by


189

the scenario. In LAMS “the “Grouping” tool – set to divide students randomly into two groups

(Dalziel, 2006b).” However, as discussed in section 4.5, this is a particular group formation policy

which could be instead supported by complementary specifications and administration tools in

charge of instantiating the UoLs and associating persons to groups.

Several aspects of the script cannot be described in a single LAMS V1.02 “sequence” (Dalziel,

2006b). For example, “while LAMS can provide text instructions about which resource and chat

room are to be used by which group, it does not enforce this – students are able to enter the other

group’s resource/chat.” Nevertheless these aspects are solved with LAMS V2.0 according to

(Dalziel, 2006b). LAMS is “inspired” by IMS LD but is not a reference implementation of the

specification. Therefore it makes used of their own LAMS computational representation that

emphasizes the role of activity tools: “The early development of LAMS was based on the general

concepts of […] IMS LD, but with a particular focus on developing a range of activity tools

(especially collaborative activity tools) (Dalziel, 2006a).” This is in line with our concern that more

(collaborative) tools and description of tools are needed in IMS LD or other eventual related

specifications (cf. Chapter Four).

Another language proposed in the workshop is LDL (Learning Design Language) which

according to their authors “… relies on a powerful meta-model which allows the representation of

various situations, particularly collaborative ones, with few concepts (Martel et al., 2006)”. Though

the number of concepts employed in the LDL meta-model is smaller than in IMS LD, it seems that

LDL is not more expressive than IMS LD in terms of representing the proposed script. Besides, it

remains to be seen whether the concepts of LDL are easier to use than IMS LD terms, valuing also

the trade off that may imply their specific vs. general expressiveness. All in all, we agree with the

comment of Dufresne, whose Explor@Graph (Dufresne, 2006a) system does not use IMS LD but

an embedded proprietary ontology: “I saw the complexity of IMS LD in others… but I saw some

interesting solutions and I think that it is a very powerful tool to express the pedagogy using this

language as long as we have tools which are easy and useful to describe this logic and use it…

[Video]”

The contribution of CPM (Cooperative Problem-Based learning Metamodel) is not that much on

the computational representation aspect but on the design of Problem Based Learning situations

(PBL). CPM models need to be transformed to IMS LD, for example, in order to be computer-

interpretable: “We also use transformation techniques and tools to map our CPM models with more

technical standards (IMS-LD Binding for example) (Nodenot et al., 2006a).” The design aspect is

analyzed in next finding.


190

6.4.3.4 Finding IV: on the comparison regarding design Tattersall uses the basic design procedure recommended in (IMS, 2003a) and Reload LDE to

create the UoL representing the script (Tattersall, 2006a). As he admits in [Video], “Reload and

CopperCore are tools at the notation level […] it is nice to see other approaches here … that makes

easier to use LD…” He also states in (Tattersall, 2006b), “we believe the following trends will likely

emerge: The tooling used for creation of UoLs will likely less directly reflect the concepts in the

specification and will tend more towards those of educational practice. As a result, templates will

likely emerge which can be used by instructional designers as a starting point for modification and

tuning”. Again, our approach is pioneering in this trend: hiding the concepts of IMS LD by

providing a design process that offers templates based on sound educational practice (cf. Finding I).

The authors of LDL (and the associated Learning Design Infrastructure, LDI) also admit the

need of this kind of design processes implemented in authoring tools: “The building phase is not

completely achieved yet in the current version of LDI. Indeed, a user-friendly scenario editor

destined to the teachers is required… (Martel et al., 2006a).”

On the other hand, the orientation of (Amorim et al., 2006b) is not clearly devoted to teachers.

In contrast, they affirm “Our approach is for machines instead for humans, so it is certainly

complementary to different approaches [Video].” Nonetheless, their contribution is important as far

as design is concerned. They incorporate implicit knowledge found on IMS LD in terms of formal

axioms (in an ontology) that can be used for testing the semantic validity of the designed UoLs. As

they illustrate with the Planet Game script, “With our approach, the steps to create and execute the

Learning Design (LD) document are as follows: (1) creation of the LD document with Reload and

represented in XML Schema, (2) translation from the XML document into F-Logic with a Learning

Design translator, (3) consistency validation using ontology axioms (Amorim et al., 2006a).”

The approach of (Paquette et al., 2006) is qualified by Nodenot in [Video] as a tool at the

“technical level”: “I think that MOT+LD solution has also focused on the technical level.” This is

true in the sense that it uses a graphical representation that is directly related to the IMS LD

elements: “Using a graphical representation technique and a modelling tool like MOT+LD,

concepts, procedures and principles are used to describe all IMS LD level-A components as well as

their relationships (Paquette et al., 2006).” However, MOT+LD approach is significant not only

because the graphical notation is easier to use by a certain type of users but also because it

complements IMS LD with an instructional method: “The IMS LD specification provides a

standardized machine-readable representation of a learning design, whereas MISA proposes a

systemic and mostly graphic method to design and implement such learning designs. MISA helps

develop learning designs by specifying four models for knowledge and competency model,

pedagogical strategy, learning material structure and delivery processes (Paquette et al., 2006).”

Therefore, the possibilities of MOT+LD to specify knowledge/competency are rich; however the


191

target users of MOT+LD are learning designers familiar with the specification and its capacity to

represent CSCL scenarios is limited: “It (IMS LD) permits a more developed multi-actor

representation than MISA, but it is poorer for knowledge/competency… (Paquette et al., 2006).”

CPM provides a rich graphical formalism to designers of PBL situations from the initial

requirements phase to the detailed design steps (Nodenot et al., 2006a). However, though it is an

independent language of IMS LD (and other EMLs), UML-based CPM is also intended for users

with advanced technological skills: “CPM language addresses pedagogic engineers […] A

pedagogic engineer activity consists in coordinating all of the actors involved in learning situations

[…] As a pre-requisite, pedagogic engineers have to know UML modelling bases (Nodenot et al.,

2006a).” This approach is similar to the contribution of (David et al., 2006), who also propose to

enrich the graphical formalism of UML activity diagrams as models independent of the script

computational representation: “These views are respectively: an enriched narrative form, an array

form and a graphical form (refereeing to an enriched UML activity diagram) […] The three views

of a scenario could establish a Computer Independent Model (CIM), in the Model Driven

Approach. After having detailed the scenario into these three views, it has then to be implemented

on an e-learning platform, using previously existing meta-models...”

Regarding the use of UML, the authors of MOT+LD claim “We have experimented such a

graphical language to be closer to instructional designers, than software engineering graphical

languages like UML or text-based editors like RELOAD… (Paquette, 2006)” As in Collage (cf.

Finding I), MOT+LD is only IMS LD level A compliant and it is needed to use other editors to add

level B elements to the resulting UoL. Paquette explains this solution as follows, “IMS-LD provides

the basic functionalities to build collaborative scenarios but graphic design tools are essential to

cope with the inherent complexity. Text based hierarchical tools like RELOAD should be used when

the basic level A design has been completed enough and exported to it to add the more technical

elements such as items addresses, metadata, properties and conditions, notifications. This two-step

approach corresponds to different designers’ competencies (Paquette, 2006).” We agree with this

position; however we argue that the problem is that MOT+LD symbols aim at covering all the IMS

LD primitives. In our approach we plan to incorporate IMS LD level B and C constructs in Collage

by means of reusable elements that represent specific pedagogical-founded design solutions (cf.

subsection 5.4.5.4). Over again, our approach anticipates the envisaged trend: “We should keep

improving on IMS-LD (standards are hard to impose) Important tasks are to: simplify the graphic

languages; extend the EML with basic collaborative templates (Paquette, 2006).”

Though without being IMS LD compliant, Explor@Graph also uses graphs for facilitating the

design of scenarios: “The Explor@Graph system is a tool where scenarios are designed as

conceptual graphs […] Designing the scenarios can easily be done top down, first describing the

activities, adding description of resources and then adding links to them in the graphs. […] Not all


192

the needed functionalities are accessible, for example, the integration of evaluation coming from the

teachers is not. But the system is certainly very easy to use to describe activities, concepts, to link

resources and even integrate support (Dufresne, 2006b). As discussed in subsection 5.6.1, a possible

direction towards a more easily extensible approach of our proposal could be researching the use of

more general visual representations (not directly related to the diagrams of pattern solutions), such

as for example the conceptual graphs and their implied trade offs (e.g. facility to include new

templates vs. the intuitiveness of the visualizations).

As we indicate in [Video] “we are interested in extending Collage with other reusable elements

that add more design possibilities.” Collage implements a design process that fosters the reuse of

patterns capturing successful CL flow structures. However, it is deeply discussed in section 5.2 that

other types of reusable elements, such as LAMS activity tools are possible. To design the script

with LAMS, “The teacher opens LAMS Authoring to create a linear sequence using, in order: (a)

the “Grouping” tool – set to divide students randomly into two groups; (b) a pair of “Share

Resources” tools named according to the groups, and incorporating the relevant expert interviews.

[…] (d) a “Forum” tool available to all students, including some prompt questions to encourage

discussion between the groups. […] (e) a questionnaire/quiz in which each student tries to identify

the planet name/order, and then receives a score and feedback on his/her answers (Dalziel,

2006b).” The very easy way of assembling LAMS activity tools is complementary to the refinable

Collage templates. LAMS activity tools could be assembled in Collage templates to refine its

activities or as new ones that enlarge the learning flow (see the example discussed in subsection

5.6.2).

Adding different types of reusable elements and constrained connecting rules between them to

the design process behind Collage would provide more design options without endangering the

principles of the reused element. However, not only are design constraints what limit flexibility at

design time, the available enacting system can also influence this effect as advanced in Finding III.

This idea is also pointed out by Dalziel in [Video], “…quite different approaches of modelling the

scenario […] limited to the features of the available tools,” and discussed next.

6.4.3.5 Finding V: on the comparison regarding enactment There are two different perspectives in the approaches participating in the workshop that

include execution environments (cf. Table 6.4). Explor@Graph (Dufresne, 2006b) and LAMS

(Dalziel, 2006b) are integrated systems for authoring and execution, while the approaches of

(Tattersall, 2006a; Amorim et al., 2006a; Martel et al., 2006; Hernández-Leo et al., 2006b) employ

different systems for design and enactment (CopperCore, LDI and Gridcole). Besides, Gridcole and

current developments around CopperCore advocate the integration of external tools according to

service-oriented technologies.


193

The first conclusion in this sense is that the differences in the enacted scripts of the different

approaches are quite influenced by the available tools. For example, in Explor@Graph “… not all

the needed functionalities are accessible, for example, the integration of evaluation coming from

the teachers is not (Dufresne, 2006b).” This is also manifested by (Tattersall, 2006a), which

indicates that service integration (according to a Service-oriented Architecture) into Coppercore-

based environments can solve the problem of limited availability of tools: “Further service

integration into Coppercore-based environments has been the topic of recent R&D and a loose

level of integration has been achieved with Moodle. Through this integration, Moodle’s forum

services are used to facilitate the inter-team cooperation, including the teacher participation. At the

time of writing, no chat service has been integrated with the CopperCore Service Integration layer,

although the TENCompetence project will carry out integration of Jabber, the open source instant

messaging service (Tattersall, 2006a).” As elaborated in Finding II, this approach is already

implemented by Gridcole system with especial emphasis in CSCL requirements (integrating CSCL

tools according to user’s group) and the possibility of using tool requiring super-computing

capabilities.

This service-oriented approach is more general (though not very different and probably more

ambitious) than the solution of LAMS V2.0 to tackle this problem. The approach of LAMS consists

on a new architecture based on a “tool contract” that specifies the requirements for activity tools

that can be integrated in LAMS: “LAMS V2.0 contains a new architecture based on a “tools

contract”. The concept of an interface or API between a Learning Design controller and a suite of

activity tools […] Beyond the requirements of a generalized LD tools contract, there may be value

in descriptions of unique features for each type of activity tool, such as Discussion Forums, Chat,

Peer Assessment, etc... ”

On the other hand, an aspect that is not satisfactorily covered by current IMS LD tooling is user-

friendly administrative facilities needed when instantiating UoLs. As Tattersall, we use the

“command line” Clicc functionality of Coppercore to manually associate users to groups (in our

case also to create the occurrence of groups): “Once a Unit of Learning has been exported as a Zip

file, it can be uploaded into a CopperCore based environment […] Using administrative facilities, a

run of the Unit of Learning is created and individuals are manually associated with a role

(Teacher, Team A or Team B) (Tattersall, 2006a)” In contrast, LDI includes such facilities so that

they can be easily used: “This consists in putting at the teacher’s disposal an interface allowing him

to choose the participants, to attribute the roles, and to select the services and contents required by

the scenario…(Martel et al., 2006)”

Moreover, further advances regarding enacting UoLs are envisaged. They are related to the use

of mobile devices and tighter integration of design and enactment systems to increase flexibility at

runtime: “New IMS LD-aware players will emerge, including micro-players allowing learning


194

processes to be coordinated across mobile devices. […] A tighter integration of design-time and

runtime perspectives on IMS LD will occur, so that designs can be critiqued and improved on the

basis of log data (Tattersall, 2006b)”.

6.4.3.6 Finding VI: on the comparison regarding observation and traces LAMS, LDL and Explor@Graph consider important functionalities regarding observation and

traces. This is especially noteworthy in LAMS: “The LAMS Monitoring environment allows

teachers to observe both whole class and individual student progress in real time. Whole class

observation is provided through a visual representation of the sequence. Integrated with the

observation features of the Monitoring environment is the ability to see student “traces” (ie,

records of activity). The teacher can click on any activity for any student to view the student’s

contribution to the activity (Dalziel, 2006b).” LDI also enables that every concept of LDL is

observable, “In the context of our case study, the interesting concepts to be observed are: the

productions, exchanges between teams, exchanges within each team, the progression of the

scenario itself. Trails of the observed concepts are produced (Martel et al., 2006).” Though with

limitations regarding traces, teachers can see the mean progression in graphs of Explor@Graph, “he

(the teacher) can watch the evolution of the discussion, evaluate contributions of each student using

the search, add his evaluation on contributions and add his comments or clues. The forums are

kept, with their evaluations and content (Dufresne, 2006b).”

With regard to the current IMS LD tooling, observation and traces aspects are limited to the

facilities provided by external tools integrated according to the UoL and by the use of the monitor

service element. (Tattersall, 2006a) argues “observational facilities are provided by the use of IMS

LD’s monitor service when linked to specific properties (e.g. responses to questions) for particular

roles. In terms of the way in which observations can be used to modify the activity’s progress,

possibilities can be included in the design to have activities, acts, etc. be completed when a value is

set. This can be as simple as having a flag be raised when a member of a particular role sees fit (as

illustrated in this example), through to more complex conditions in which average scores or

numbers of users completing can trigger further events.” In our approach we discuss the role of the

integrated external tools concerning observation: “The discussions of each chat room are

automatically stored in different files. Synergeia also provides log-files collecting students’ actions.

The teacher can use this information for regulation purposes: participating during the execution of

the UoL according to what she observes. The UoL may have been enriched by including LD

properties for directly providing the log-files to the teacher. These log-files can be also used to

evaluate the learning process a posteriori: e.g. how students negotiate with their peers (Hernández-

Leo et al., 2006b).” Of course, we advocate that providing further facilities which enable observing

students’ progress in real time are important for the adoption of IMS LD systems. This is even more


195

crucial in CSCL situations where being aware of the progress of group partners may affect the

effectiveness of the learning process.

6.4.3.7 Finding VII: on the comparison regarding re-use/adaptation Most of the approaches participating in the workshop declare that the resulting script can be

easily re-used by changing the content associated to the activities: “Though this UoL is planned for

a study of the solar system, it can be reused for other subjects by changing document titles and

associating different item locations (Paquette et al., 2006)”, “The pedagogical modelled situation

can be re-used in other contexts: only the contents have to be changed (Martel et al., 2006).”

Our contribution is however that we are already reusing a general structure (JIGSAW CLFP).

Besides, since the JIGSAW–based template implemented in Collage specifies the groups in a

general manner (the actual groups need to be created at instantiation time), the possibilities of re-

using the script increase: “Moreover, this UoL can be easily adapted for a different topic or

learning situation. E.g. forming more teams or several (Jigsaw) groups, i.e. mixing different

members of team A and team B for cooperating in different forums. That would allow (for example)

to study several negotiation strategies. In both cases it is only necessary to create more instances of

the corresponding roles (Hernández-Leo et al., 2006b).” A further possibility is making use of the

“Jigsaw Groups” described in the template to adapt the scenario to a situation that requires several

groups mixing members of different teams for cooperating in diverse forums (new “instances of the

role “Jigsaw Group”). The idea of using the structure of the script as a template is also pointed out

in (Tattersall, 2006a), “… the Unit of Learning can easily be turned into a template by modifying

the resources to address a different topic […] In essence the Unit of Learning could be used for

many different areas.”

Moreover, as mentioned in Finding IV other levels of reuse are possible, such as the approach

of assembling activity tools followed by LAMS, which can be easily adapted by including

additional activity tools, “the LAMS Authoring environment provides the potential for adaptation of

the task described here. New activities can be easily added, such as additional resources to be

provided after the forum, or different team-based tasks based on alternative student

groups…(Dalziel, 2006b)”

6.5 Case study C: Network Management

The “Network Management case study” provides an authentic non-trivial educational

experience that puts into practice a CSCL script created following the proposals of this dissertation.

The script enacted in this case study is originally proposed by one of the teachers implicated in the

GSIC/EMIC mini-case study, which is embedded in the “Collage workshops case study”. The

teacher represents in the script a learning situation that he has realized without computer support in


196

his Network Management course for several years. In fact, the same teacher participates in the

experience under analysis in this “Network Management case study”, putting into practice their own

aforementioned script. This script is already described in subsection 5.5.1. It is based on a

combination of three CLFPs (JIGSAW, PYRAMID and TPS) and is authored with Collage.

This section starts describing the case study, which includes the educational context and

interviewees, the deployment, materials and the data gathering. This is followed by the explanations

of the conceptual structure of the case study, which guides the analysis to the case findings

discussed in the last subsection.


Table 6.1 summarizes the description of the “Network Management case study”, which

analyzes an experience involving a teaching/learning process. In contrast with the previous case

studies, this case study considers students (and their teacher) as the interviewees. The students

experience a script created with Collage as a part of one of the courses in which they are enrolled.

6.5.1.1 Educational context and interviewees The experience is accomplished in an eligible undergraduate course on Network Management

technologies with 12 students. The course takes place in the spring semester of the fifth and last

year of the studies for achieving the degree of “Telecommunications Engineering” at the University

of Valladolid, Spain. The course contents include an overall view of the concepts and purpose of

Network Management as well as a description of the main standard technologies in the field. The

whole course spans a 15-week long semester and involves 30 lecture hours (one 2-hour session per

week) and 30 laboratory hours (also one 2-hour session per week).

Unfortunately, the engineering studies to which the course belongs are characterized by a strong

competitive environment in which collaboration and cooperation are wrongly perceived as

equivalent: assignments are divided into separate tasks that group members complete individually,

without joint activities that induce socio-cognitive processes (characteristic of genuine collaborative

learning (Dillenbourg, 1999b)). Furthermore, the curriculum does not explicitly encourage the

acquisition of skills needed for the professional future of the students. Learning objectives related to

those skills are just promoted in a limited set of courses, under the personal teachers’ initiative. In

fact, there are two previous courses in the curriculum, one dealing with Computer Networks

(Hernández-Leo et al., 2006) and another related to Computer Architecture (Martínez-Monés et al.,

2005) that introduce collaborative pedagogical approaches but with very limited technological

support. This case study goes a step further by augmenting the role of technology and, particularly,

by enacting a script created with Collage.


197

6.5.1.2 Sites, deployment and materials The case study covers face-to-face activities during two 2-hour sessions (one week) devoted to

the presentation of SNMP (Simple Network Management Protocol) Network Management standard

(Stallings, 1998). Both sessions take place at the course laboratory having 25 Intel Pentium IV-class

machines running Linux 2.6.8. Additionally, the case study involves a distant learning activity

during which students employed their own personal telematic resources (they did not have access to

laboratory premises). For that activity the students just need an Internet connection and a web

browser. More details on the design of the learning process are exposed in the description of the

script executed in the experience (cf. subsection 5.5.1).

Table 6.1 summarizes the deployment and materials employed in this case. We use the Gridcole

system (Bote-Lorenzo, 2005) to enact the script. This script is actually a UoL computationally

represented according to the LD specification (cf. Chapters Four and Five). The computational

representation uses an extension to the LD service element so that collaborative tools can be

generally specified (cf. subsection 6.4.3.1). Thanks to the LD extension Gridcole is able to provide

different instances of the same collaborative tool to the members of diverse groups at the same time.

Therefore, Gridcole integrates a chat, Synergeia and Quest as indicated in the script.

The deployment of this case study had two phases: familiarization or initial training and the

actual enactment of the script. The familiarization phase takes place in a session previous to the

experience itself. Nevertheless, only 10 of the expected 12 students attend this session. At the

beginning of the session the students answered an initial questionnaire (using Quest) about their

previous experience, which part of the data gathered for evaluation (cf. next subsection). Then, the

teacher presents the plan of the activities for the next two sessions (and the asynchronous activity in

between) and the tools that they are going to employ to complete these activities. The students have

also the opportunity to informally use Gridcole, Synergeia using an adaptation of the script

introduced in subsection 5.5.3. It is a Pyramid-based script for collaboratively discussing about a

paper, also created with Collage. Besides, this training session is useful to change the initial

passwords of the different tools. The teacher writes an e-mail to the (two) students that do not attend

this session, explaining what have been done in this phase and asking them to answer the on-line

questionnaire. Only one of them reads the e-mail and answers the questionnaire on time.

Before enacting the script, we need to instantiate the UoL so that a particular run for our

situation is produced. In this step the required occurrences of the different types of groups are

created as well as the assignment of participants to the occurrences (cf. section 4.4.2). The hierarchy

of group types as created by Collage according to the combined CLFP-based templates is complex

(cf. subsection 5.4.5.3): the roles corresponding to the first and third level of the Pyramid are

substituted with the roles of the Jigsaw and TPS CLFPs (cf. Figure 6.7). For the instantiation of the

roles (or groups) we use clicc tool provided by CopperCore, the LD engine of Gridcole. We utilize


198

this tool for the case study; however it is clear that it is not intuitive and user-friendly for most of

the teachers as discussed in Finding V of Case Study B. Figure 6.7 also shows the results of this

step. The new occurrences are labelled with new and a number between brackets. The number

indicates the order in which the instances are created (when a new instance of a role is created, new

instances for all available sub-roles or that role are created as well).

In addition, Figure 6.7 indicates the assignment of participants (i.e. its identifiers) to groups. If

we assign a participant to an initial role, this participant is automatically bound to the roles that have

this initial role as a sub-role. Therefore, we just assign each participant to a Jigsaw group (four

groups available, each of which is named with a colour) and consequently to an Expert group (each

member of the same Jigsaw group belongs to a different Expert group). On the other hand, we need

to bind all the participants to the TPS-pair role, since the whole class will form a single pair (each

member of the pair is a Pyramid level 2 group). However, it is possible to observe a problem with

the Pyramid level 2 groups: as a result of their assignment with the Jigsaw and Expert groups,

several participants will have available both occurrences of Pyramid level 2 group. We solve this

problem easily thanks to the configuration of Gridcole services using the extension for their general

specification. However, this drawback is to be solved in new versions of Collage: when a template

based of JIGSAW CLFP replaces a phase of another pattern, the roles defined in the JIGSAW are

included at the same level of the role related to the replaced phase (and not as its sub-roles). ç

Figure 6.7 Hierarchy of group types as created by Collage according to the combined CLFP-based templates (a) and its instantiation with the needed occurrences and assigned participants (b)

According to this distribution of participants, the lab is organized as shown in Figure 6.8.

student1

student2

student3

student4

student5

student6 student7 student8

student9

student10 student11

student12

Pryramid_3 Create-

new=”allowed”

TPS-student Create-

new=”allowed”

Pyramid_1 Create-

new=”allowed”

Pryramid_2 Create-

new=”allowed”

TPS-pair Create-

new=”allowed”

Jigsaw-group Create-

new=”allowed”

Expert-group Create-

new=”allowed”

Pryramid_3

TPS-student

TPS-pair

Pyramid_1

Pryramid_2

Jigsaw-group

YELLOW

Expert-group

PART-1

Pyramid_1 (new 1)

Pryramid_2(new 1)

Jigsaw-group(new 1) BLUE

Expert-group (new 1) PART-2

Jigsaw-group(new 3)

RED

Jigsaw-group(new 4) GREEN

Expert-group(new 2) PART-3

ALL

student1

student2

student3

student4 student5

student6

student7

student8

student9

student10

student11

Student12

(a) (b)


199

Part of the paper (expert groups)

1 2 3

Red

Blue

Green

Yellow





Jigsaw groups

Figure 6.8 Lab distribution according to the different groups

This figure is provided to the students in a document that also includes Figure 6.9, which

explains the roles they are to play in each moment (the original version of this document (in

Spanish) and a number of photos and screenshots are available in the attached CD-ROM).

Figure 6.9 Extract of the document provided to the students

Finally, the script is enacted using Gridcole. A couple of photos regarding this experience can

be observed in Figure 6.10 and Figure 6.11. During the session we collect different types of data

useful to evaluate the experience as it is explained next.


200

Jigsaw group

Expert group

Figure 6.10 Global view of different moments of the experience, in which the students used the different integrated tools

Figure 6.11 A student in the Expert group phase

6.5.1.3 Data gathering Five different types of data gathering techniques are used in this case study. Following an

adaptation of the mixed method described in subsection 6.2.3, the techniques represent data sources

providing data of quantitative and qualitative nature. As in the “Collage workshops case study” the

use of the selected techniques is planned in three phases: before the deployment itself, during the

deployment and at the end (after the deployment). The different types of data sources in the


201

“Network Management case study” are (all the data, the designed questionnaires, observations

template, etc. are available in the CD-ROM):

- Questionnaires: two web-based questionnaires for the students (created and published using

Quest). The first questionnaire is completed before the experience, even before the

familiarization phase. This questionnaire is intended to provide information about their

previous experience in order to value the educational innovation of the case and detect

possible biases or preconceived opinions. The second questionnaire is answered after the

process. In this questionnaire the students are asked to value quantitatively and qualitatively

(open questions) the different aspect of the experience. In addition, and adapted final

questionnaire about the whole process is completed by the teacher. The questionnaires are

designed so that they cover the research questions considered in the (first version of the)

conceptual structure of this case study.

- Observations: using an observation template, the researcher attends the face-to-face sessions

and notes down the interactions, attitudes and incidents that occurred during these sessions.

After each session the observer elaborates a report exposing this information.

- Discussion groups: with the aim of gathering detailer information about the aspects included

in the questionnaires, the researcher (without the present of the teacher) meets all the

participants a week after the experience. She interviews them in a more relaxed, free

environment. The debate is recorded and transcribed into a document in order to facilitate its

qualitative analysis.

- Log files: the shared repository Synergeia generates event logs informing whether the

student read the documents and comments them (and when). It is especially useful to

analyze the distant asynchronous activity (second session at home). The chat employed by

the Expert groups also registers the conversations. (The students are informed about it

beforehand.) These logs are helpful to study the development of this activity.

- Students’ outcomes: the partial results of the students during the process (outcomes of

Jigsaw groups and second Pyramid level groups) are collected.

Due to the complexity of this case study (compared with the previous ones) in terms of the

number of data sources considered, a diagram of the adapted mixed evaluation method used in this

case study is provided in Figure 6.12. As indicated in this figure the categories used in the

comparative analysis is directly related to the conceptual structure of the case study that is

introduced next.


202

Before the enactment

During the enactment

After the enactment

Categories of analysis (related to the conceptual structure of the case study)

Qualitative data

Questionnaires

Partial results of the students during process (Outcomes of Jigsaw groups and second Pyramid level groups)

Comparative analysis of the categories in the aspects collected by the different techniques and phases

Report with the evaluation results

Quantitative data

Observations Discussion group Outcomes Logs

Direct observations during the

first

and third face-to-face

sessions

A debate of 30 minutes (without the

teacher)

Initial questionnaire about previous

experience

Final questionnaire regarding the whole

experience (Another for the teacher)

Chat and Synergeia log files

(Familiarization)

Figure 6.12 Applied mixed evaluation method. Adapted from (Martínez-Monés, 2003)


As in the previous case studies, the conceptual structure of this experience include the issue,

representing the general research question, the topics on which we focus the study and the

particularization of the topics into more concrete information questions that guides the data

analysis. Some of these information questions are defined beforehand, some evolve and some new

questions emerge as a result of the data analysis and interpretation. We recapitulate the issue related

to this case study: can we use CSCL scripts created with Collage in real situations? The

following topics, which influence the issue, are proposed to guide the study:

- Subsection 5.5.1 describes how the teacher creates the pedagogical design of the case study

in the form of a computer-interpretable script using Collage. Along the design process, he

selects and combines the CLFPs that best serve the type of task to be solved and that best

elicit the desired objectives related to collaborative learning. Then, the teacher particularizes

the activities offered in the CLFPs according to the requirements of the learning situation.

Therefore, an important topic refers to the meaningfulness of the CSCL script created with

Collage. That is to say, whether the script created with Collage actually reflects the teacher’s

design intentions.

- Accordingly, the enactment of the CSCL script using Gridcole is also of high interest.

Figure 6.10 and Figure 6.11 show that students interact with the system. This topic focuses

the analysis on how the students follow the Collage script using Gridcole along the blended

learning experience.


203

- On the other hand, we are interested in seeking information regarding the implications of our

approach for new research opportunities that affect education. Hence, the third topic

considered in the conceptual structure of this case study analyzes in which sense it

represents an educational innovation with respect to previous students’ experiences.

Besides, the evolved information questions of that originate form these topics are:

Topic 1: Meaningfulness of the CSCL script created with Collage:

I. Is the CSCL script contextualized to the actual learning situation?

II. Does the CSCL script guide the learning process coordinating the students at the activity

level according to the CLFPs on which is based?

III. Does the CSCL script foster the desired objectives related to collaborative learning?

Topic 2: Enactment of the CSCL script using Gridcole

IV. Can the students follow successfully the CSCL script using the Gridcole system?

V. How can the enactment of the CSCL script be improved?

Topic 3: Educational innovation with respect to previous students’ experiences

VI. Does the enactment of the CSCL script enhance students’ previous experience in terms

of structuring collaboration and use of supporting technology?

6.5.3 Case findings

Throughout this section we discuss in detail the findings representing the conclusions about the

issue established in this case study. As mentioned above, the analysis is organized according to the

information questions forming its conceptual structure. As in Case Study A, the qualitative analysis

is supported by the Nud*IST tool (SQR, 1997).

The data gathered from different sources are triangulated (as it is explained in subsection 6.2.3)

in order to support the trustworthiness of the partial results sustaining the findings. The integration

of the data is done using an evolving scheme of analysis categories (the final list of categories is

available within the “Nudist project” available in the CD-ROM). In this way, the partial results,

together with their supporting arguments, are listed in a series of tables collected in Appendix D.

Each table corresponds to an information question. In order to understand from which source the

supporting arguments proceed, the tables include the “coding of the data source” that is generated

by Nud*IST (the coding also allows the reader to check the “original” data (in Spanish), which are

available in the CD-ROM). With the aim of providing a reader-friendly presentation of the findings,

this section only reproduce a selection of the most representative supporting arguments, labelled

with the simplified coding shown in Table 6.7. The reader may consult the Appendix D for further

information.


204

Table 6.7 Labels used in the text to quote the data sources of the “Network management case study”

Data source Labels

Questionnaires [Quest-initial] [Quest-final]

[Quest-teacher]

Observations [Observation-session-1] [Observation-session-2]

Discussion group [Discussion]

Logs

[Repository-log] [Chat-expert-group-1] [Chat-expert-group-2] [Chat-expert-group-3]

Student outcomes [Outcomes-jigsaw] [Outcomes-2level-pyramid]

6.5.3.1 Finding I: the script is adequate for the task and objectives of the situation Regarding the meaningfulness of the CSCL script created with Collage according to

requirements of the actual learning situation, a first quantitative indication appears in a closed

question of [Quest-final] and [Quest-teacher]. All students (12 out of 12) and the teacher value as

positive (among the available possibilities: negative, acceptable or positive) the structure of the

experience regarding its utility to reach the objectives directly related with the content of the course.

Besides, qualitative arguments confirm this result. In the [Quest-final] a student affirms “… the

structure is very positive to understand the different SNMP alternatives. I have acquired a good

knowledge of SNMP without a great effort.” Others state “… it is positive to break down the paper

so that we have collaborated to extract the fundamental concepts of SNMP. Moreover, we have

worked with a technical article, what we have never done before.”

Moreover, the structure of the experience is considered adequate for the type of task undertaken

in the experience. In this sense, students surprisingly think that the system is developed specifically

for this task: “This type of structure is merely useful to work with the document we have read (or

another). But if we need to work on another type of problem then everything may be changed, and

the system is no longer useful [Discussion]”, “… I think it is a good practice… But it is probably

not useful in other courses [Quest-final].”

Quantitative data show a tendency that is also supported by qualitative arguments: the task is

easy considering the relative duration of the experience to the rest of the course (cf. Table D.1). On

the other hand, the time planned for the experience is sufficient to accomplish the activities which

result motivating. Three selected opinions coming from different sources that are in line with these

conclusions are: “… we might advance more slowly than in traditional classes but I think that more

ideas get fixed in our minds [Quest-final]”, “… the deployment might be excessive for reading an

article. But it should be considered that the experience seems to have increased the motivation

[Quest-teacher]” and “… it motivates more than just reading the article. In this case, many of us


205

would have either not read it, or read it too quickly, or not understood it, or not extracted so many

ideas [Discussion].” However, further work is necessary to provide more support regarding the

preliminary stages of enacting this type of computer-supported experiences, “The drawback of this

type of experiences is the time required to prepare all the details. The tools should provide more

support [Quest-teacher].”

6.5.3.2 Finding II: the script guides the students according to the CLFPs The script actually structures the teaching / learning process by coordinating the flow of

activities according to the combined CLFPs. A partial result underlying this conclusion (cf. Table

D.2) points out the fact that the experience proceeds to a large extent as it is designed in the script.

The teacher considers that the development of the experience reflects exactly what he intended to

design in the script using Collage. However, because of time limitations the teacher decides to skip

a phase of the script (it is not a problem of the design but of the actual circumstances that appears at

runtime). This is noted in [Observation-session-1], “The teacher skipped the “Pair” phase of the

TPS (because of time limitations). He proceeded directly to the discussion…” as well as declared by

the teacher in [Quest-teacher], “The planned design has been followed well... (except of the final

TPS, but it was because I propitiated it).”

As designed, the script helps resource distribution. There are many evidences supporting this

result, which can be also seen in the screenshots of the Gridcole running the script (cf. Figure 6.10,

Figure 6.11 and other screenshots and pictures available in the CD). [Repository-log] shows that the

members assigned to a particular expert role only access to their part of the article. Similarly, the

members of the successively larger groups only read the result of the group which they are to join in

the following activity. Comments of students also manifest this conclusion, for example: “We were

distributed in groups and each member of the group read a part of the article [Quest-final].”

Moreover, the evaluation data point out that the guidance in the sequence of activities, as

indicated by the script, is effective and efficient. As a student pinpoints “The use of a script

facilitates an efficient achievement of the objectives. Besides it allows the teachers to follow the

progress of students [Quest-final].” Interestingly, students do not find the script too coercive

because they can collaborate freely within the activities. In fact, groups differ in the way they

collaborate within activities as it can be appraised comparing [Chat-expert-group-1], [Chat-expert-

group-2] and [Chat-expert-group-3] (group-1 starts discussing doubts, group-2 identifying the main

ideas and group-3 commenting paragraph by paragraph). The observations transcribed in

[Observation-session-1] and [Observation-session-2] also illuminate this result. In addition, one of

the students states “It is better if the coordination within activities is up to us, because each of us

works in his own way. For example we proceeded differently to student1’s group … [Discussion]”


206

It is possible to affirm that the selected tools support the realization of the activities, though the

use of the chat in the face-to-face activity is not perceived as effective. Students agree according to

their opinions in [Quest-final]: “Coordinating using a chat is difficult in a brief period of time”,

“Interacting using the chat has been the most difficult think but we actually understood each other

and organized ourselves quite good”. In the rage of 1 (not adequate) to 6 (very adequate) students

rate the adequateness of the tools to support the activities with the average of 5.33 (in the case of

Synergeia), 4.66 (Quest) and 4.25 (chat tool).

Needs of flexibility emerges in the experience. The fact commented above referring to the need

of skipping a phase of the script is an example of it. Though the circumstances requiring flexibility

that emerge during the experience are managed successfully, Finding V discusses other demanding

circumstances that require further support.

6.5.3.3 Finding III: the script promotes the objectives related to collaborative learning Supporting data allow us to conclude that the script fosters the desired objectives related to

collaborative learning. The students rate (in a range of 1 (very negative) to 6 (very positive)) with

an average of 5.17 (deviation of 0.37) the collaboration with their classmates. The log files

generated by Synergeia [Repository-log], the recorded chat conversations [Chat-expert-group-1],

[Chat-expert-group-2], [Chat-expert-group-3], and student outcomes [Outcomes-jigsaw],

[Outcomes-2level-pyramid] show that all students participate actively. Besides, there are evidences

indicating that the script truly promotes positive interdependence and individual accountability,

which encourage students to reflect on the concepts and to practice the desired competencies. As

two student mentions “… it demands an active participation and responsibility, because we have to

explain our part to the other members of the group… [Discussion]”, “It has been necessary an

effort of synthesis and explanation of the accomplished work to the other members of the group in

the way of reaching consensus [Quest-final].” Though motivation due to the eligible characteristic

of the course may be considered an important aspect of successful collaboration, it is worth noticing

that scripts based on the same patterns and applied to non-eligible courses (also in engineering

education) result in similar positive effects (cf. (Martínez-Monés et al., 2005; Hernández-Leo et al.,

2006) and subsection 3.4.3).

The different data sources also indicate that students reach discussion and agreement. For

example, [Outcomes-jigsaw] and [Outcomes-2level-pyramid] show that all groups, in the different

phases of the script, formulate the maximum number of ideas (10 or 8 depending on the phase) and

(2) questions. Most students indicate in their opinions (cf. Table D.3) the same ideas pointed out in

this statement of one of them: “Everybody participated in the discussions what improves the

sharing of ideas (it helps the shy members to express themselves). A classmate explains what the

others have not read, and this promotes learning. It is possible to ask doubts and everybody may


207

answer, not only the teacher. The experience also helps to reflect in what is being doing [Quest-

final].”

6.5.3.4 Finding IV: the students follow the script using Gridcole Regarding the enactment of the script using Gridcole, we can state that students follow the

script successfully using the Gridcole system in the face-to-face as well as in the distant activities

(cf. Table D.4). The [Repository-log] shows that, after the first face-to-face session, 11 students

read and post comments (accessing through Gridcole) to the outcomes of the other members of the

Jigsaw group that they are joining in the following session. A student points out “The tools worked

correctly and the activities have been accomplished without problems [Quest-final].”

Moreover, not only do students find the system very useful supporting collaboration and

indicating what to do and which tools to use in each activity, but they also consider that having

access to the system from home provides flexibility. Some students affirm “… it is very helpful that

the system indicates the objectives of each phase and provides direct access to the tools needed in

each activity.” In fact, they rate (in the range of 1 (not useful) to 6 (very useful)) with an average of

5.75 (deviation of 0.43) the usefulness of the system to accomplish the task step by step. The

teacher also rates with 6 the usefulness of the system and indicates “Without the integrating system,

the students would have needed to devote more attention to the tools [Quest-teacher].” In addition, a

student states “I did not attend the first 2-hour session and I managed to find out what the rest of the

classmates did during this session from home. Otherwise, I would have been lost in the following

session [Discussion].” Furthermore, students opine that the familiarization session helps but it is not

crucial for them since they have advanced technological skills. One of the comments in this sense

is, “The most difficult aspect was learning how to use the tools… However, with the explanations of

the familiarization session is sufficient, and the system facilitates everything… [Quest-final]”

6.5.3.5 Finding V: improvements associated to flexibility, awareness and authentication Finding II and the previous finding also point out the needs of flexibility that appear in this type

of collaborative strategies. For example, in this case study the teacher skips the “Pair” phase of the

TPS CLFP because of time limitations [Observation-session-1], what is allowed by the script and

the system. However, if three students assigned to the same expert group do not attend the first

session, it would be necessary to change the composition of the experts groups, so that at least two

(out of four) students participate in the corresponding collaborative activity. Doing so with the

current version of the system is not easy, but it is one of the major issues considered in the short-

term future work.

Other concerns include enhancing the intuitiveness of the interface and the addition of

awareness and authentication utilities. Observations present evidences in this sense: “The teacher

reminds again that students should change the role [Observation-session-1]”, “It was necessary to


208

remind students the structure of roles/groups and the flow of activities [Observation-session-2].” In

fact, students miss the incorporation of the information provided in a document (cf. Figure 6.9) as

part of the system. As they states in [Quest-final], “I miss an activity index indicating which role

should be selected in each moment, instead of distributing it in a document…”, “I do not understand

the issue of the groups in the system. I would be easier if the system obviates the change of roles

(and makes it automatically)…”

6.5.3.6 Finding VI: new opportunities for structuring collaboration using ICT Compared to their previous experiences, students value very positively the learning scenario in

terms of structuring collaboration and use of supporting technology. Significantly, when students

are asked to compare how different is this experience compared to their previous ones, nine of them

answer that “they have found quite a lot of differences” and the other three indicate that “they have

found a lot of differences” (cf. Table D.6). One of the main reasons supporting these opinions seems

to be related to the provision of a collaboration strategy in contrast to the encouragement of totally

free collaboration or mere cooperation. There are some indications suggesting that the students

suffer frustrating non-scripted collaboration experiences and non-effective cooperation practices

along their engineering education. A student regrets “I slightly prefer working individually, because

in groups we have always divided the task in order to finish earlier and one of the members has

often worked more than the rest… [Quest-initial]” Another student states “… for example, ‘Name of

a course’ also promotes collaborative activities. However, the activities are a disaster because they

are not well organized. In this experience, there is a structure, a plan and collaboration is really

fostered. In contrast, in ‘Name of the course’ there is not any indication and nobody works, what

does not lead to learning outcomes [Quest-final].” Besides, other students add “… sometimes the

lack of efficiency makes us to waste time and hate working in groups. The realization of guided (or

partly guided) activities and the use of new systems like Gridcole increase the efficiency and our

interest in the activities [Quest-final].” Further indications show the usefulness of the approach to

increase the probability of reaching learning outcomes: “I have felt more guided because I knew

shat to do in each moment [Quest-final]”, “With this type of methods we learn more, work harder

and better and we take more advantage of the time [Quest final].”

Furthermore, students agree that this experience promotes the development of competencies

useful for students’ future processional life. Students argue “… we, as engineering students, are

rather individualist, but in our professional future we will work collaboratively. Thus, this has been

a good training experience and there should be more like this along our education [Quest-final].”

This conclusion is also supported by the results of a previous course on Computer Architecture

(CA) that students have also experienced (Martínez-Monés et al., 2005): “Actually we have not

practiced collaborative learning during the university studies, except for the CA course, in which

collaboration has been very useful [Quest-initial].”


209

6.6 Cross-case analysis

Previous sections emphasize the distinctive strength of each case, noting its particular context

and its functioning, which represents a different perspective or manifestation of the global

contribution of this dissertation (the quintain, the proposed pattern-based design process for CSCL

scripts computationally represented with LD). This section undertakes the cross-case analysis. Its

purpose is not to revise the common relationships across cases, but to understand the commonality

and differences across manifestations of the quintain (Stake, 2005).

The objective is to make assertions about the quintain and specifically to several themes that

derives from this quintain. As advanced in subsection 6.2.1, though some themes are proposed at

the beginning of the research, additional themes may emerge as a result of the cross-case analysis.

In this way, the themes in this multicase study are:

- Original theme 1: Does the proposed design process facilitate the high-level generation of

contextualized CSCL scripts reusing CLFPs and focusing on CSCL critical elements?

- Original theme 2: Is LD suitable for computationally representing CLFP-based scripts?

- Additional theme 3: What predictions can we make about future developments around LD

and CSCL scripts?

The assertions are the findings about the quintain. Each assertion should have a common focus

(the theme) contributing towards understanding the quintain. The assertions result somehow as a

combination of the most important findings from each case. Thus, the assertions are understood

better because of the particular activity of each case. According to (Stake, 2005, pp. 41), this

section, devoted to the cross-case, is however expected to be shorter than the sum of the cases and

tends to oversimplification given the data load. Anyway, the evidence that leads to the assertions

needs to be indicated (through the case findings). In contrast to the cases, the quintain appears less a

coordinated system and more a mosaic.

Therefore, case findings merge across cases around a framework of proposed multicase themes.

Table 6.8, Table 6.9 and Table 6.10 illustrate the approach adopted for formulating assertions,

which emphasizes the various situations and findings of the cases. It consists of rating each case

finding as to its importance for understanding the quintain through a particular theme. As indicated

in section 3.5 of (Stake, 2005), we use a three-point scale in which a high mark means that for a

particular theme, the case findings are of high importance (H = high importance; M = middling

importance; L = low importance). When a finding is of significant relevance considering a

particular theme, the tables include a summary of the finding in the corresponding cell. The

procedure accomplished is moving from case A finding I to case C finding VI. For each finding, we

describe its utility and prominence for its contribution to understanding each theme (until all the

cells of Table 6.8, Table 6.9 and Table 6.10 have an entry).


210

Table 6.8 Partial matrix for generating theme-based assertions from case findings, case A (adapted from the worksheet 5A of (Stake, 2005)) H = high importance; M = middling importance; L = low importance. A high

mark means that for this theme, the case findings are of high importance.

Case A:

Original theme 1 High-level generation of

contextualized CSCL scripts reusing CLFPs and focusing on


Original theme 2 IMS LD

computational representation of

CLFP-based scripts

Additional theme 3 Predictions about future

developments around IMS LD and CSCL scripts

Finding I: on the selection and representation of CLFP-based LD templates

H (The “selection phase” is critical and promotes the understanding of the patterns)

M (Their ideas about the CL strategies collected in the CLFPs coincide with

what is presented in Collage)

H (Other examples of well-known CL strategies; patterns

for assessment)

Finding II: on the trade-off between reuse of patterns and design options

H (The steps of the “authoring phase” facilitate the reuse of CLFPs when

structuring collaborative learning designs; the combination of patterns provides

outstanding design flexibility; particularized according to the needs of concrete learning situations; satisfactory trade-off between flexibility, keeping the

essence captured in the CLFPs, hiding LD-specific technological details and providing a clear (but limited) set of design options)

L

H (Pattern-based templates are probably more useful in the

process of customizing a new situation than ready-to-run

scripts, but complete (or partly complete) examples are also

helpful)

Finding III: focus on learning objectives and task type

H (The proposed design process helps to determine the learning objectives related to

collaborative learning that will be promoted and to select the task-type that will be solved by the students; the design process helps to determine the expected

interaction)

L L

Finding IV: focus on the structure regarding the activity flow and group hierarchy

H (The support that the process provides to determine the structure of the activity flow

is very satisfactory; the design process helps to understand and determine the

structure regarding the hierarchy of groups)

H (Determine the activity structure and the hierarchy

of groups) L

Finding V: focus on group size and support within the activities

H (The design process helps to establish the group size; determine the distribution of resources among activities as well as to the computer support; the design process helps

to describe each activity and its eventual (textually-defined) structure according to

the envisaged expected interaction)

H (Establish the group size; determine the

distribution of resources among activities as well as to the computer support;

the design process helps to describe each activity and

its eventual (textually-defined) structure)

M (The determination of the group size limits can be

improved, for instance, with automatic checkups)

H (Collage does not provide suggestions of recommended

content or tools)

Finding VI: Collage, easier-to-use CL-specific LD editor

H (Most of the participants find Collage user-friendly and intuitive; the participants

are able to create an almost completed example during the workshops; Collage is

easier to use, specific to collaborative learning, and it is the first editor providing

pattern-based templates)

M (The participants are able to create an almost

completed example during the workshops)

L

Finding VII: improvements associated to selection of patterns, further design options and integration with complimentary tools

L L

H (Enhance the selection of patterns, changing a resource in “one step”; support of LD levels

B and C, the integration of Collage with complementary needed tools, which should

support flexibility; the information reflected in the

resulting scripts is also useful for the students)

Finding VIII: audience interested in CL and in the use of ICT

H (Audience interested in designing CL processes to be used with an LMS in face-to-face, distant or blended situations or as lesson plans; the design process provides ideas and fosters the design creativity of the users; it is not necessary to be expert

(LD) technologists)

L L


211

Table 6.9 Partial matrix for generating theme-based assertions from case findings, case B (adapted from the worksheet 5A of (Stake, 2005)) H = high importance; M = middling importance; L = low importance. A high

mark means that for this theme, the case findings are or high importance.

Case B:


contextualized CSCL scripts reusing CLFPs and focusing on CSCL

critical elements

Original theme 2 IMS LD computational

representation of CLFP-based scripts



Finding I: the main aspects of the script can be designed with Collage

H (The main aspects of the script can be modelled with

Collage)

H (There are only two details that cannot be rigorously authored with

Collage; However, it is possible to add the necessary IMS LD constructs to the

script using Reload LDE or another non-constrained IMS LD compliant

editor)

H (general way of specifying a group-service (not necessarily

dedicated to conferences) is using the conference service element of IMS LD and an external binding document that indicates which

groups need a different instance of the service)

Finding II: the script created with Collage can be enacted using Gridcole system

M H (Gridcole system, which includes the CopperCore IMS LD engine, is

capable of interpreting the UoL created using Collage)

L

Finding III: on the comparison regarding the computational representation

L

H (IMS LD supports the implementation of this script, with the

interoperability advantages that it implies; many possibilities of the

specification which is flexible enough to describe scripts with the same core design but with different details; the

determination of the actual number of groups at instantiation time provides flexibility and generality, however a service making available the clues to

users depending on their group is necessary)

H (Group formation policy which could be instead supported by

complementary specifications and administration tools; more

(collaborative) tools and description of tools are needed in IMS LD or

other eventual related specifications)

Finding IV: on the comparison regarding design

H (Our approach is pioneering in this trend: hiding the concepts of IMS LD by

providing a design process that offers templates based on

sound educational practice)

L

H (Incorporate IMS LD level B and C constructs in Collage by means of

reusable elements that represent specific pedagogical-founded design

solutions), using more general representations, other types of

reusable elements)

Finding V: on the comparison regarding enactment

L M

H (Service integration (according to a Service-oriented Architecture) into Coppercore-based environments can

solve the problem of limited availability of tools; an aspect that is not satisfactorily covered by current

IMS LD tooling is user-friendly administrative facilities needed

when instantiating UoLs; mobile devices and tighter integration of design and enactment systems to increase flexibility at runtime)

Finding VI: on the comparison regarding observation and traces

L L

H (Observation and traces aspects are limited to the facilities provided

by external tools integrated according to the UoL and by the use

of the monitor service element)

Finding VII: on the comparison regarding re-use/adaptation

H (We are already reusing a general structure (JIGSAW CLFP). Besides, since the JIGSAW–based template implemented in Collage specifies the groups in a

general manner (the actual groups need to be created at

instantiation time), the possibilities of re-using the

script increases)

L M


212

Table 6.10 Partial matrix for generating theme-based assertions from case findings, case C (adapted from the worksheet 5A of (Stake, 2005)) H = high importance; M = middling importance; L = low importance. A high

mark means that for this theme, the case findings are or high importance.

Case C:


contextualized CSCL scripts reusing CLFPs and focusing on


Original theme 2 IMS LD

computational representation of

CLFP-based scripts



Finding I: the script is adequate for the task and objectives of the situation

H (Reach the objectives directly related with the content of the course; the structure of the

experience is considered adequate for the type of task undertaken in the experience)

L

H (Provide more support regarding the preliminary stages

of enacting this type of computer-supported

experiences)

Finding II: the script guides the students according to the CLFPs

H (The experience proceeds to a large extent as it is designed in the script; the guidance in the sequence of activities, as indicated by the

script, is effective and efficient)

H (The experience proceeds to a large

extent as it is designed in the script)

L

Finding III: the script promotes the objectives related to collaborative learning

H (The script fosters the desired objectives related to collaborative learning) L L

Finding IV: the students follow the script using Gridcole

M (Students follow the script successfully using the Gridcole system; students find the system very useful supporting collaboration and indicating what to do and which tools to use in each activity, and they consider that

having access to the system from home provides flexibility)

H (Students follow the script successfully

using the Gridcole system)

L

Finding V: improvements associated to flexibility, awareness and authentication

L L

H (Needs of flexibility - change the composition of groups;

enhancing the intuitiveness of the interface and the addition of

awareness and authentication utilities)

Finding VI: new opportunities for structuring collaboration using ICT

H (Students value very positively the learning scenario in terms of structuring

collaboration and use of supporting technology; usefulness of the approach to

increase the probability of reaching learning outcomes)

L L

Next subsections gather the high-importance findings for each theme, as suggested by the

entries in Table 6.8, Table 6.9 and Table 6.10, and formulate assertions that can help to

satisfactorily understand the quintain. The formulation of the multicase assertions, as the

formulation of case findings, is the result of an interpretative process (Stake, 2005, pp. 72).

6.6.1 Assertion I: the design process facilitates the high-level generation of contextualized

CSCL scripts reusing CLFPs and focusing on CSCL critical elements

Theme 1 appears significantly in case A, what indicates the relevance of this case for

understanding the quintain (regarding this theme). This fact is also highlighted in the formulation of

the multicase study, which anticipates that the main activity enabling the evaluation of the design

process refers to the functioning of creating CSCL scripts by the target audience. On the other hand,

the findings of case B and case C provide additional evidence that combined with case A’s findings


213

lead to the assertion: “the design process facilitates the high-level generation of contextualized

CSCL scripts reusing CLFPs and focusing on CSCL critical elements.”

First of all, case B’s findings I and IV (cf. Table 6.9) indicate that the main aspects of a script

proposed by a third-party can be modelled with Collage, which implements a pioneering approach:

hiding concepts of LD by providing a design process that offers templates based on sound

collaborative learning practices. These practices are formulated in terms of reusable patterns

(CLFPs) that capture expertise regarding CL flow structures. The originality of our approach is also

emphasized by case A’s finding VI, which concludes that Collage is easier to use, specific to

collaborative learning, and it is the first LD editor providing pattern-based templates. As expected,

the audience of the design process (and the authoring tools implementing it) are teachers interested

in applying CL and using ICT in their practice. They do not need to be CL experts (the design

process provides ideas and fosters the design creativity of users) or expert LD technologists.

According to finding I of case A (cf. Table 6.8), a critical phase in the design process is the

selection of CLFPs, which promotes the understanding of the patterns. Besides, the analysis of the

different experiences studied in case A shows that the steps of the authoring phase facilitate the

reuse of CLFPs in such a way that the resulting script is particularized according to the needs of a

particular situation (finding II). The design process achieves a trade-off between flexibility (in

which the combination of patterns plays an important role), keeping the essence captured in the

CLFPs and hiding LD-specific technological details and providing a clear (but limited) set of design

options. Several findings of case C also imply that the generated script (using Collage) which is

experienced by the students is contextualized according to their particular situation. For example,

case C’s findings VI and I (cf. Table 6.10) indicate that students value very positively the scripted

experience whose structure is considered adequate for the undertaken type of task and which

facilitates reaching the objectives directly related to the concepts of the course.

The learning objectives and the type of task are in effect important CSCL critical elements

according to which the design process enables the conceptualization of the expected interaction in

advance. Case A’s finding III and case C’s finding III provide evidence in this sense. Moreover,

findings IV and V of case A reveal that the design process makes teachers focus on other CSCL

critical elements. In this way, its support regarding the determination of the structure of the activity

flow and the group hierarchy is very satisfactory. The design process also helps to establish the

group size, the textual description of the (eventually micro-structured) activities, the distribution of

resources and the computer support.

The fact that the script created within case B can be successfully (and as expected) enacted with

a system indicates that Collage scripts actually reflect the design intentions. A stronger statement in


214

this sense occurs in case C’s findings, especially in findings II and IV which testify that the

experience with real students proceeds as it is designed in the script.

6.6.2 Assertion II: LD is suitable to a large extent for computationally representing CLFP-

based scripts

The analysis accomplished in Chapter Four regarding the support of LD specification to

computationally represent CSCL scripts is completed with the multicase study presented in this

chapter. In effect, the assertion concerning theme 2 is: “LD is suitable to a large extent for

computationally representing CLFP-based scripts.”

This assertion is supported by findings of the three cases studied. The fact that CSCL critical

elements can be determined along the design process when selecting and refining a CLFP-based

template into an LD compliant script (findings I, IV, V of case A) is an important evidence in this

sense. This implies that these elements can be computationally represented with LD. Additional key

indications are provided by case C. Gridcole system interprets the LD script and guides students

reflecting what the teacher expects according to the design accomplished with Collage (case C’s

findings II and IV).

Furthermore, Case B adds significant findings supporting this assertion. In this case, Gridcole

also interprets a Collage UoL representing the main aspects of the scenario proposed by a third-

party (finding III). Besides, the details that cannot be rigorously authored with Collage (due to its

constrained design options) can be added however using other lower-level editors (finding I). This

statement is also supported by other participants in the workshop studied in case B. Its finding III

shows that it is even possible to model the scenario with LD using different approaches. This

indicates the possibilities of the specification, which also supports flexibility by enabling the

determination of the actual number of groups at instantiation time (instead of at design time).

However, this support can be improved as discussed in next assertion.

6.6.3 Assertion III: the design process can be extended with more design options and

further facilities for instantiation and execution will enhance LD-based scripted

CSCL

An additional theme that emerges analyzing the cases is related to the need of interesting future

developments for enhancing the support of CSCL scripts represented using LD. The combination of

case findings regarding this theme originates the assertion III: “the design process can be extended

with more design options and further facilities for instantiation and execution will enhance LD-

based scripted CSCL.”


215

Future work regarding the design process is pointed out in findings of case A and case B. It

includes incorporating other types of patterns, such as those related to assessment (case A’s finding

I). This idea is in line with the possibility of integrating other types of reusable elements at other

levels of granularity and completeness (case A’s finding II). Case A’s finding VII introduces the

prospect of extending Collage with support to LD levels B and C. In this sense, finding IV of case

B discusses that this extension should be accomplished by means of reusable elements that

represent specific pedagogical-founded design solutions. In this sense, a research line is the study of

the trade-offs between using more general graphical representations instead of the specific diagrams

of the patterns’ solutions (among other things, to facilitate the integration of more reusable

elements) vs. the intuitiveness.

Further facilities can be added to the implementation of the design process in authoring tools

(such as Collage). For example, case A’s finding V suggests improving the support of the

determination of the group size limits in group hierarchies with automatic checkups. Another

example is indicated in finding VII. It deals with enhancing the selection of patterns enabling more

sophisticated recommendations (using ontologies, for instance).

It is also important to mention here the need of (generally) specifying group services (not

necessarily dedicated to e-mail or conferences) within the environments supporting collaborative

activities. This need appears in cases B (cf. findings I and III) and C that adopt a preliminary

solution which should be enhanced and formulated as an extension of LD or as a complementary

(interoperable) specification (cf. section 4.5.2 of Chapter Four).

Moreover, Case B’s finding V indicates that service integration (according to a Service-oriented

Architecture) into Coppercore-based environments can solve the problem of limited availability of

tools in current LMSs, what constrains the design possibilities. A related problem in this sense is

searching the (service-based) tools that are adequate for supporting the script and which can be

integrated by these systems. This need also appears in case A’s finding V which points out that

Collage does not provide suggestions of recommended content or tools.

The teacher interviewed in case C (cf. finding I) manifest that enhanced support regarding the

preliminary stages of enacting this type of computer-supported experiences is needed. It refers

mainly to user-friendly administrative facilities needed when instantiating UoLs (e.g. creating the

actual number of groups, associating users to groups), which is an aspect not satisfactorily covered

by current LD tooling. This is also a conclusion of the workshop under study in case B’s finding V.

On the other hand, observation and traces aspects are currently limited to the facilities provided

by external tools integrated according to the UoL and by the use of the monitor service element

(case B’s finding VI). Further research and development is needed in this sense, on for example


216

how the awareness of the students about their own and their group partners progress along the script

can be supported (case A’s finding VII).

Needs of flexibility are patent in CSCL scripted situations, LD tooling should consider

flexibility facilities such as easily changing the composition of groups at runtime. This idea occurs

for example in case C’s finding V and is enforced in finding V of case B. It can be envisaged that

tighter integration of design, instantiation and enactment systems increases the flexibility.

6.7 Discussion

This chapter undertakes the evaluation phase of this dissertation using a multicase study

methodology. The multicase comprises three cases devoted to different activities that enable the

evaluation of the dissertation’s global contribution (the quintain) from different perspectives. The

conceptual structure of each case and the corresponding data analysis and interpretation remark that

the emphasis of the approaches in the three cases is different. The findings concluded in each case

study are decidedly important to understand the application of our proposal in their specific

situations. However, taking evidence from the case studies in a way that their findings are combined

to understand the aspects of interest (themes) is what enables to characterize the quintain. Although

case A’s findings appear to be the most relevant in this sense, the role of case B and case C is

crucial to illuminate the potential of our contributions as a whole. That is to say, the multicase

involves workshops with the target audience (teachers, case A) and with students (case C), which

experience a resulting product of the approach; but it also involves experts in the CL or LD field

(mini-cases of case study A) and researchers proposing related approaches (case B).

Moreover it is worth mentioning that successful expert review of the contributions is also

manifested as a result of the articles accepted for publication which introduce the main results of

this dissertation. The number of special issues in which they are included together with the fact that

a conference includes a workshop on this topic (case B) manifest that the interest in the domain

problem undertaken by this dissertation is noteworthy.

In addition, the research accomplished in related work, within our GSIC/EMIC group, points to

more indications about the validity and usefulness of our approach as well as to further

considerations for future work. This is the case of the preliminary study about the design tensions

related to socially-mediated vs. computer-mediated scripting in CSCL (Dimitriadis et al., 2007)

which aims at identifying under which circumstances computer-mediated scripting is more efficient

and effective and how the flexibility needed to regulate this tension can be supported by LMSs. On

the other hand, additional evidences about the LD support to represent CLFP-based scripts to be

executed with Gridcole system can be found in (Bote-Lorenzo, 2005). Besides, the influence of the

context in which the dissertation is developed is also noticeable in the accomplishment of the


217

multicase study itself, since it benefits from the know-how accumulated in the group in previous

work around case studies (Jorrín-Abellán et al., 2006).

Several reflections can be done after the analysis of the multicase. This chapter and the

associated appendixes C and D make apparent the high load of data involved in the qualitative

perspective of this type of evaluation methodology. Though quantitative data help to indicate

tendencies around the issues of interest, it is the comparative analysis of the qualitative data what

lead to understand and explain the achievements of the contributions as they appear in the several

cases studied. In this sense, the results presented in this chapter imply a devoted work of data

analysis and interpretation which is significantly demanding. However, the conclusions in form of

case findings and multicase assertions show their value beyond the “simple” illustration of having

reached the objectives of the dissertation and entail an additional contribution which offer relevant

founded clues for future work. Next chapter deepens in this sense presenting the conclusions of the

dissertation.


218

CHAPTER SEVEN

CONCLUSIONS AND FUTURE WORK

This chapter includes the main conclusions of the dissertation along with its main contributions as well as the main directions of future work. Both conclusions and future research directions are related to the three main topics combined in this dissertation: scripting CSCL, patterns in TEL and IMS LD. The global perspective is on fostering their adoption by teachers.

The number of publications related to the work presented in this dissertation (four international journal papers, one book chapter and seven international conference papers), the awards that it has co-received (ICALT 2004 best paper and 2006-2007 European CSCL Award for Excellence in the Field of CSCL Technology) together with the new associated projects that have started or are being currently proposed (MosaicLearning TSI2005-08225-C07-04, VA009A05, EU COST proposal, etc.) are relevant indicators of the success of this research and the importance of its derived future work.

7.1 Conclusions and main contributions

Throughout this dissertation we have tackled several challenges around the problem of

facilitating teachers the design of potentially effective CSCL situations that make use of learning

ICT environments. The literature review presented in Chapter Two has led us to conclude that the

number of individual and group level variables that affect collaborative processes demands a less

rigid design approach than those proposed by Instructional Design solutions for individual learning.

Instead, we have adopted a perspective which relies on the CSCL concern for promoting the

elicitation of productive social interactions as the core mechanisms to reach learning objectives.

From the design viewpoint it involves planning a set of instructions that scaffold the collaborative

situation. When these sets of scaffolding rules are embedded in learning environments, they are

called CSCL scripts. Our focus has been specifically on the scripts reflecting pedagogical methods

that structure CSCL situations at the macro-level, i.e. scripts that basically specify flows of

activities and the groups (and roles) that perform these activities.


220

Chapter Two has also described the problems derived by the fact that existing scripting

approaches are typically “hardwired” in purposely devoted environments. These problems include

reusability limitations of the scripts in different situations, significant time and cost efforts when a

new script needs to be supported as well as hindering Participatory Design approaches through

which teachers could actively determine the behaviour and functionality of the scripting

environments. In this sense, the dissertation has advocated the use of suitable computational

representations that enable the formalization of the scripts in such a way that they can be interpreted

and executed by (general) learning management systems (LMSs). In particular, this dissertation has

undertaken three challenges that draw from this approach.

The first challenge dealt with the design of the scripts so that they potentially elicit the desired

interactions. As deeply exposed in Chapter Two, the design of successful scripts considering CSCL

critical elements is not trivial, especially for teachers who are new to CSCL. That chapter also

identified the use of design patterns as the research direction to face up to this challenge. Patterns

formulate design solutions grounded in successfully-proven practice in such a way that they can be

applied and creatively adapted to many different situations (Goodyear, 2005).

Chapter Three has followed an iterative process in which several case studies are analyzed and

relevant literature on design and scripting is reviewed with the objective of identifying the types of

patterns and relationships between patterns that can be used for generating CSCL scripts. As a

result, the chapter has proposed a conceptual model for describing CSCL scripting Pattern

Languages (PLs). The model defines the different types of patterns and relationships among them

so that it is possible to specify numerous meaningful paths enabling the generation of many pattern

sequences that shape the design of specific scripts. Besides, Chapter Three has also discussed

general guidelines for applying the PLs that can be described with the proposed conceptual model.

Moreover, Appendix A includes a PL which is compliant with the model. It comprises only 18

patterns but it illustrates the different types of patterns and relationships considered in the model.

Furthermore, Chapter Three presented three scripts applied in real situations that can be generated

using this PL. Though two of these situations were employed to identify the patterns, the examples

show the PL properties of moral preoccupation, coherence, generativeness and creativity. Yet, a

third authentic situation has been designed explicitly applying the patterns from scratch. The

evaluation results of this third situation provided additional evidences of the fruitful results

associated to the practices formulated in the patterns. The conceptual model, the illustrating PL

and the three scripts reflecting actual interconnected patterns selected from the PL are

contributions that represent a starting point towards agreed high level structures for the production

of CSCL scripting patterns and PLs which enable the sharing of good scripting practices within and

between communities.


221

The second challenge coped with the computational representation of the scripts so that they can

be interpreted by software engines integrated in LMSs. CSCL scripts comprise complex

mechanisms and components (interrelations of groups, synchronization of collaborative activity

sequences, etc.) which impose demanding requirements for their machine-readable specification to

candidate Educational Modelling Languages (EMLs). Chapter Two discussed the reasons that

motivate the selection of IMS Learning Design (LD) specification as a potential candidate to

computationally represent the scripts. Besides its declaration of intent to describe collaborative

learning situations, the fact that it is an open specification agreed upon by domain and industry

experts envisages promising interoperability prospects. If the scripts are formalized using LD they

may be executed in any LD compliant environment, with the associated benefits in terms of

reusability, adapted reproducibility, etc.

Chapter Four has faced up to this second challenge tackling the objective of analyzing the

suitability of IMS LD for computationally representing CSCL macro-scripts, as formulated in the

introduction of this dissertation (Chapter One). The first step so as to fulfil this objective was to

identify common CSCL requirements in the scripts. The importance of the requirements was

justified according to a large number of literature sources, one of which aims at providing an

inclusive framework for specifying the characteristics of the scripts. Though it is however not

possible to state that the list of requirements is complete, they comprise characteristics that enable

the description of significant scripts, including the script flow structures suggested by the patterns

presented in Appendix A which are of type CLFP (Collaborative Learning Flow Pattern, as

indicated of the conceptual model proposed in Chapter Three). Chapter Four tested and illustrated

the capacity of the LD notation to support each requirement by means of representative cases. The

chapter also concluded that the LD limitations for computationally representing some

characteristics are minor and discussed how these deficiencies could be supported by

complementary specifications and tools. This discussion did not intent however to provide

definitive proposals but to indicate interesting future research directions (related to the development

of new specifications and system architectures) which would require important efforts in terms of

devoted work and community consensus if we want to foster interoperability. Therefore, the

contributions of the dissertation regarding the second challenge consist of the lessons learned from

this analysis showing the capacity of the LD notation to express CSCL macro-scripts and

pointing out the role of related tooling and eventually complementary specifications.

The third challenge involved the fact that computational representations are not familiar to most

of the teachers. LD is a set of dense technical documents, intended for a technical audience and do

not enforce design processes that support the creation of pedagogically sound designs. The majority

of the teachers do not have advanced technological skills (particularly of LD), nor do they always

have the support of technical experts to create the computationally represented scripts which satisfy


222

the requirements of their actual educational situations. Concerning this challenge, Chapter Two

pointed out as a potential solution the provision of graphical authoring tools that generate

computationally-represented scripts.

With the aim of facing up to this challenge in a comprehensive manner that also considers the

previous challenges, Chapter Five has undertaken the objective of proposing a design process that

facilitates the reuse of CLFPs in the creation of CSCL macro-scripts (computationally represented

with LD) in a way that allows teachers to particularize the patterns according to the needs of their

educational situation, making explicit the CSCL critical elements. That chapter discussed how new

approaches to facilitate the creation of LD Units of Learning (UoLs) based on pre-existing learning

design solutions are emerging. The target audience of these approaches is teachers who use

concepts and structures more relevant to their community of practice than the terms of the LD

specification. The chapter introduced a framework that conceptualizes different (existing and yet-to-

come) approaches that drive the creation of full-fledged UoLs by reusing different types of design

solutions. This framework enabled us to situate our contributions and compare it to related work. It

is also in Chapter Five where we proposed a design process using CLFP-based LD-represented

templates as the units of reuse for the creation of scripts. Our target audience is therefore teachers

that are interested in practicing scripted CSCL but not necessarily experts in CSCL and LD. Chapter

Five argued that instead of collecting CSCL scripting patterns in repositories, these patterns can be

explicitly incorporated in design processes embedded in authoring tools as assistants (advising

mechanisms) or as refinable templates and building blocks (partially completed designs or design

chunks). In the case of CLFPs which provides structures of CL flows in their solution, they can be

offered as LD templates. These templates, which are partly completed re-usable designs, include the

description of the flow of activities and associated groups (or roles) that potentially elicit expected

interactions leading to certain well-known CL benefits. The diagrams of the CLFP solutions are

used as the graphical representations of the templates, which include configurable aspects that

depend on the pattern.

The design process includes two main phases: selecting a CLFP-based template and authoring a

CLFP-based UoL. Apart from providing examples and detailed information of the CLFPs, the

selection of templates is supported by using metadata whose elements include the learning

objectives that the CLFP elicits, the type of problems (task-type) that the CLFP best serves, and the

complexity or risks in terms of CL experience needed to put in practice a script based on the

pattern. When teachers select a CLFP-based template, they automatically determine the level of

structure that is necessary to accomplish a set of learning objectives, to elicit certain expected

interaction and to undertake specific task-types. In the authoring phase, teachers refine the

templates, according to the needs of actual educational situations, and indicate further objectives

(beyond those depending on the CLFP) that are typically related to the subject matter or content to


223

which the CLFP is applied. Besides, the learning flow can be enriched by combining or

concatenating CLFPs into templates of CLFP hierarchies. In this case, the selection phase is

repeated. Defining the description of the activities and determining the resources needed to support

them implies specifying the structure of the interaction process within activities. This is related to

defining the computer support that is used to sustain learning and expected interaction (face-to-face

or computer mediated). Moreover, determining the group-sizes and providing further information of

groups can be also realized in this phase.

In this way, the design process for the generation of LD scripts reusing CLFPs is the central

contribution of this dissertation since it links several aspects of the previous contributions. As

targeted in the objective tackled in Chapter Five, the design process reduces the technical

complexity by hiding the details of LD and guarantees potential effective results, since it is based

on the reuse of good practices in CL. Consequently, it fosters the reuse of the patterns as a way of

communicating CL expertise to others eventually novice practitioners. In sum, it offers a trade off

between generality and unrestricted design options vs. adequate reuse and particularization of

CLFPs (and hierarchies) and an easy edition of collaborative LDs.

Moreover, the CLFP-based design process is implemented in an authoring tool (Collage).

Chapter Five has thoroughly illustrated the design process, as implemented in Collage, with several

scripts drawn from real practice, what shows the feasibility and usefulness of the whole approach.

Furthermore, these examples have been also employed in Chapter Six within a set of experiences

with teachers, students and educational technologists and researchers with the objective of

evaluating the proposed pattern-based design process for CSCL macro-scripts computationally

represented with LD. In this sense, the evaluation accomplished in Chapter Six is organized as a

multicase study that comprises three different cases. They aim at assessing the design process

but from different perspectives. The first case study is devoted to workshops where mainly the

target audience uses the design process implemented in Collage authoring tool. A second case study

implies the design of a scenario proposed by a third-party using our approach. Both cases also

involve the participation of CSCL and LD experts as well as researchers proposing related

approaches, what enabled us to provide further evidence about the originality of our contributions.

The last case study analyzes an authentic educational situation where students follow a script

created according to the CLFP-based design process. It is worth recapitulating that the findings

concluded in each case study are decidedly important to understand the application of our

contributions in their specific situations. In addition, the combination of the case findings

enabled us to point out the main conclusions about the global aspects of interest of the

proposed design process. These conclusions can be summarized as follows: the design process

facilitates the high-level generation of contextualized scripts reusing CLFPs and focusing on CSCL

critical elements; LD is suitable to a large extent for computationally representing CLFP-based


224

scripts; the design process can be extended with more design options; and further facilities for

instantiation and execution will enhance LD-based scripted CSCL.

Moreover it is worth mentioning that successful peer-review of the original contributions of this

dissertation is also manifested as a result of the articles accepted for publication which present the

main results of this dissertation. The achieved publications are mainly four international journal

papers (Hernández-Leo et al., 2005; Hernández-Leo et al., 2005a; Hernández-Leo et al., in 2006e;

Hernández-Leo et al., in press), one book chapter (Hernández-Leo et al., in pressa) and seven

international conference papers (Hernández-Leo et al., 2004; Hernández-Leo et al., 2005b;

Hernández-Leo et al., 2006; Hernández-Leo et al., 2006a; Hernández-Leo et al., 2006b; Hernández-

Leo et al., 2006c; Hernández-Leo et al., 2006d).

Overall, it is possible to state that the dissertation has achieved its global objective which, as

formulated in Chapter One, refers to proposing and evaluating a design process based on patterns

for facilitating the creation of potentially effective CSCL macro-scripts computationally represented

with IMS Learning Design so that they can be interpreted by learning environments such as

Learning Management Systems.

The work accomplished in this doctoral dissertation has also enabled the identification of new

research direction for future work. They are introduced in next section.

7.2 Future research directions

The objectives of the dissertation have been achieved. Besides, this work has identified a

number of open research problems that are summarized next.

Regarding the general use of design patterns in TEL and CSCL, future work includes:

- Using the conceptual model proposed in Chapter Three as a starting point towards an

agreed high level framework that interrelates different types of TEL and CSCL design

patterns. The common framework could include and connect patterns related to different

pedagogical approaches and practices, specific subject matters, and the design of supporting

systems. Having such a framework would foster the combined application of different types

of patterns, the sharing and communication between different communities of practice and

institutions, and the production of new (coherent) patterns. In this sense, the patterns could

be provided in on-line repositories, organized according to the framework. This work

should be accomplished in a way that the considered patterns are suitably validated in the

pattern community using shepherding and writer’s workshops. As a first step, the PL

included in Appendix A could be interconnected to other patterns and patterns languages

proposed in the community. This step would provide further insights about the usefulness


225

of the proposed conceptual model and would indicate how it can be extended towards a

wider framework. An opportunity to work in this direction may be provided by an eventual

project (named “SET-uP: Sharing knowledge and expertise in designing learning

environments: the use of design patterns) whose pre-proposal has been recently (March

2007) submitted as a COST (European Cooperation in the field of Scientific and Technical

Research) Action of the EU RTD Framework Programme. This project will bring closer

research and development groups who are working in the area of designing e-learning

experiences in an effort to create, review, refine and evaluate design patterns in a very

systematic way.

- Another interesting direction of future work refers to encoding the semantics of the

relationships between patterns using ontologies. Ontologies will enable explicitly

representing the meaning of patterns (terms) in vocabularies and the relationships between

them. This approach would allow a powerful automatic organization and retrieval of the

patterns.

The proposed CLFP-based design process for the creation of CSCL scripts and its

implementation in Collage can be extended according to several interesting research directions:

- Besides adding more CLFPs, future work concerns the incorporation of other types of

CSCL scripting patterns (not only CLFPs) as assistant or templates depending on the nature

of the pattern, as already discussed in Chapter Five. Moreover, further research should be

accomplished to explore how more flexible design options can be offered with structures

that are not based on patterns but without hindering the proper reuse of patterns’ essence.

This research line is related to a more general challenge in TEL which refers to providing

methods for sharing learning design solutions of different granularity and scope so that

they are reused in a way that facilitate the creation of units of learning (not necessarily

specific to CSCL). In general, chunking pedagogy-based solutions is not trivial, since they

highly depend on context and practice. Furthermore, to facilitate the understanding of the

solutions before their actual reuse, they should be presented to users using different

conceptual approaches as well as different types of graphical representations.

- Supporting pattern-based design of embedded assessment in CSCL script is another

relevant research direction. Formative and summative assessment approaches, considering

individual and group performance, should be integrated in the design process proposed in

the dissertation. The current version of the design process does not explicitly consider the

role of assessment, which is essential in teaching-learning processes. Though assessment is

often used with learning purposes (e.g. peer assessment (Prins, Sluijsmans, Kirschner, &

Strijbos, 2005)), the different possible forms of assessment and their effects depending,


226

among other aspects, on the stage of the script deserve a focused attention. In this sense, the

GSIC/EMIC group has recently started working on the pattern-based design of embedded

assessment (Villasclaras-Fernández, Hernández-Leo, Asensio-Pérez, & Dimitriadis,

accepted). The tasks to be accomplished in this direction are: enlarging the pattern language

of Appendix A with new patterns devoted to assessment; refining the conceptual model if

necessary; incorporating the patterns in the design process and, accordingly, in Collage

authoring tool as templates and assistants. In order to formalize assessment patterns as

templates, an important task will be selecting (or developing) a specification that enables its

computational representation. As an initial result we can mention that the incorporation of a

pattern on “peer assessment” is currently being incorporated in Collage as a template

represented with LD.

- Collage is LD level A compliant at the moment. However, we plan to include level B and C

constructs by means of reusable elements that represent specific pedagogically-founded

design solutions and hide the LD complex details that imply these levels of the

specification. For example, the mechanism for distributing the artefacts to be peer-reviewed

in peer assessment pattern-based design is being incorporated in Collage as a building block

that is in charge of including the LD elements (mainly properties and global elements) that

enable such distribution.

- As discussed in Chapter Three, future work also includes the study of using visual

languages to represent patterns’ solutions. This work envisages a potential chance to face

the challenges of easily adding new patterns and other types of reusable solutions to

Collage. This approach would also enable a more general editor that allows the

visualization and modification of UoLs created with other compliant authoring tools.

However, this topic deserves devoted research that investigates the trade offs between

intuitiveness and generality of the different graphical approaches.

- Chapter Five also envisages an interesting challenge which may be considered in future

versions of the proposed design process: the integration of learning design solutions

formalized with different languages so that they can be assembled in order to generate an

LD-compliant UoL (or eventually other types of units of learning expressed with different

computational representations). The work accomplished in (Dodero et al., 2007) represents

a significant starting point towards abstracting the design solutions from the details of the

underlying specifications so that the (model-driven) generation of units of learning from

several meta-models is enabled.

- This dissertation is mainly devoted to macro-scripts. Similar research efforts should be

accomplished concerning micro-scripts. In particular, the support of LD and other


227

educational modelling languages for representing this kind of scripts needs to be deeply

analyzed as well as their integration in authoring tools supporting design techniques (such

as pattern-based approaches). The interoperability between macro and micro-scripts should

be afforded so that micro-scripts can be applied to scaffold activities within macro-scripts.

With regard to LD tooling and interoperability specifications, it is possible to point out the

following open problems:

- Chapter Four points out to several requirements of CSCL scripts that would require further

support regarding computational representation and tooling. This is the case of the

specification of grouping aspects such as group formation policies and number of group

limits. These aspects should be implemented by administration tools (and also related

supporting tools such as grouping services or player utilities) in charge of instantiating

UoLs (step in which the actual number of groups and members of each group is indicated)

in combination with eventual exhaustive group composition specifications. Therefore, the

results of this research line are expected to feed into suggestions towards standards and into

the conception of new tools. This work is also related to the need identified in the

evaluation accomplished in Chapter Six which indicates that this type of administration

tools should be user-friendly to foster the adoption of LD. Therefore, the GSIC/EMIC group

is currently developing an administration tool associated to Collage, called

InstanceCollage, which will enable the intuitive creation of groups and the further binding

of individuals to groups according to the CLFP hierarchy structure of a UoL created by

Collage.

- An extension of LD or a new interoperability specification to support the description and

binding of external supporting tools should be developed. Some initial ideas in this sense

are pointed out in Chapter Four, which have been employed in the Gridcole service-based

system (Bote-Lorenzo, 2005). Moreover, the problem related to the specification of data

flow between the tools should be addressed. This is a concern currently undertaken by our

research team. Particularly, we plan to use a workflow standard language in a way that

complements LD maintaining interoperability (Palomino-Ramirez et al., 2007).

- Another tool that we are currently developing as a result of the work of this dissertation

refers to a “process awareness tool” for teachers and students that will allow them to be

aware of the collaborative learning flow during execution: which activities have been

accomplished, which are the next ones, in which activities are involved the rest of the

members of the group that they will form in the following activity, etc. In many CSCL

situations, having such awareness is crucial, among other reasons because participants may

change their groups depending on the phase of the learning flow and may need to know the


228

progress of their future team partners. This type of tools is also important for managing

flexibility (accomplishing modifications according to the awareness information).

- The need of enhancing the flexibility support of current LD approaches has been also

manifested in Chapter Six. Several research directions are associated to this problem. They

imply the integration of authoring tools and runtime systems in such a way that execution of

UoLs could be satisfactorily modified according to the evolution of the learning process and

the unexpected circumstances that may appear (e.g. a student, member of a group,

progresses slower or abandons a course). The adaptations could be done by a teacher, an

expert system or artificial intelligent techniques. Regarding CSCL scripts, Dillenbourg et al.

(2007) suggest that the probability of reaching success would be increased if engines handle

the specifications of script constraints that are allowed to be modified along the whole

teaching and learning process. A related challenge entails research about the implications

of using general interfaces of LMSs interpreting computationally represented scripts vs. the

use of specific user interface features specifically designed to satisfy the characteristics of

particular scripts.

- Teachers participating in Collage workshop case study (Chapter Six) missed the fact that

Collage does not provide suggestions of recommended content or tools. In this sense,

semantic search of tools employing an ontology of CSCL tools that uses meaningful

learning abstractions is being researched by our research group (Vega-Gorgojo et al., 2005).

That ontology is the basis of a service discovery facility that is developed for allowing

educators to search service-based CSCL tools using learning concepts (Vega-Gorgojo et al.,

2006).

Finally, additional case studies are needed in order to further evaluate the suitability of the

presented approach in different educational, organizational and technological contexts. Especially

interesting case studies would be those accomplished by external researchers. An initial effort in

this sense is being currently carried out under the MosaicLearning Project (Spanish Ministry of

Education and Science, project TSI2005-08225-C07-04) with an experience that involves doctoral

students of three different Spanish Schools of Telecommunications and Computer Science

Engineering.

APPENDIX A

A CSCL SCRIPTING

PATTERN LANGUAGE

This appendix presents an illustrative CSCL scripting pattern language that enables the generation of CSCL scripts grounded in practice. This pattern language fits in with the conceptual model proposed in Chapter Three.


230

A.1 Collaborative learning flow level

Pattern 1.1 JIGSAW **

… within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which several small groups are facing the study of a lot of information for the resolution of the same problem.

If groups of students face resolution of a complex problem/task that can be easily divided into sections or independent sub-problems, an adequate collaborative learning flow may be planned. The flow of collaborative learning activities to be followed in order to solve a complex divisible task should promote the following educational benefits (Aronson et al., 1992; Clarke, 1994; Johnson & Johnson, 1999):

- To promote the feeling that team members need each other to succeed (positive interdependence) - To foster discussion in order to construct students’ knowledge - To ensure that students must contribute their fare share (individual accountability)

However, the solution for structuring collaboration in order to tackle this problem may be complex and probably more appropriate for collaborative learning experienced teachers and learners. It may be best suited for the end of the semester when the students are comfortable with group work. Therefore: Structure the learning flow so that each student (individual or initial group) in a group (“Jigsaw Group”) studies or work around a particular sub-problem. Then, encourage the students of different groups who study the same problem meet in an “Expert Group” for exchanging ideas. These temporary focus groups become experts in the section of the problem given to them. At last, students of each “Jigsaw group” meet to contribute with its “expertise” in order to solve the whole problem.

Collaborative activity around the problem and solution proposal

Introductory individual (or initial group) activity

Collaborative activity around the sub-problem

Individual or initial group (general representation) Teacher

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this Jigsaw structure preceded or followed by other set of activities, which can be organized as other patterns at the collaborative learning flow level – PYRAMID (Pattern 1.2), BRAINSTORMING (Pattern 1.4), TPS (Pattern 1.3), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6). If necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESS (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1). Patterns that complete this pattern: some of the Jigsaw phases might be planned according to other collaborative learning flows –PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), TAPPS (Pattern 1.6), or activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the Jigsaw structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).

A CSCL SCRIPTING PATTERN LANGUAGE

231

Pattern 1.2 PYRAMID **

(AKA SNOWBALL) … within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which several students face the collaborative resolution of the same problem.

If groups of students face resolution of a complex problem/task, usually without a concrete solution, whose resolution implies the achievement of gradual consensus among all the students, an adequate collaborative learning flow may be planned. The flow of collaborative learning activities to be followed in order to solve a complex task, whose resolution implies the achievement of gradual consensus, might promote the following educational benefits (Davis, 2002; Gibbs, 1995):

- To promote the feeling that team members need each other to succeed (positive interdependence) - To foster discussion in order to construct students’ knowledge - To enable the development of negotiation skills

The risk involved in structuring collaboration so that a gradual consensus is achieved is medium. That is, the experience needed in collaborative learning needed is not too high. Therefore: Structure the learning flow so that the students start (individually or forming an initial small group) studying the problem and proposing an initial solution. Then, encourage groups (usually pairs) to compare and discuss their proposals and, finally, propose a new shared solution. Guide the students so that the groups join in larger groups in order to generate new agreed proposals. At the end, all the students may propose a final and agreed solution.

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this Pyramid structure preceded or followed by other set of activities, which can be organized as other patterns (or even the same pattern) at the collaborative learning flow level – JIGSAW (Pattern 1.1), BRAINSTORMING (Pattern 1.4), TPS (Pattern 1.3), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6). If necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESSES (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1). Patterns that complete this pattern: some of the Pyramid levels might be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), TAPPS (Pattern 1.6), or activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the Pyramid structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).

PHASE N: All propose a final and agreed solution PHASE i: Compare, discuss and propose a shared solution

PHASE 1: Individual (or initial group) study of the problem. Proposes a solution

Individual or initial group(general representation)

Teacher


232

Pattern 1.3 THINK-PAIR-SHARE (TPS) **

… within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which students are paired to solve a challenging or open-ended question.

If groups of students face resolution of a challenging or open-ended question, an adequate collaborative learning flow may be planned. Students are much more willing to respond after they have had a chance to discuss their ideas with a classmate because if the answer is wrong, the embarrassment is shared. Also, the responses received are often more intellectually concise since students have had a chance to reflect on their ideas with the one another. The flow of collaborative learning activities to be followed in order to solve a challenging or open-ended question, might promote the following educational benefits (NISE, 1997; Millis & Cottell, 1998):

- To promote the feeling that team members need each other to succeed (positive interdependence). - To foster discussion in order to construct students’ knowledge. - To focus students’ attention on a particular topic. - To give a chance to formulate answers by retrieving information from long-term memory.

The solution for structuring collaboration in order to tackle this problem may be ideally suited for individuals who are new to collaborative learning. Therefore: Structure the learning flow so that each student has time to think about the question. Then, encourage them to pair and discuss their ideas about the question. Finally, they may comment or take a classroom “vote”.

They comment or take a classroom “vote”

They pair and discuss their ideas about the question

Each participant has time to think about the question

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this TPS structure preceded or followed by other set of activities, which can be organized as other patterns at the collaborative learning flow level – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6). If necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESSES (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1).

Patterns that complete this pattern: some of the TPS phases might be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), BRAINSTORMING (Pattern 1.4), TAPPS (Pattern 1.6), or activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the TPS structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


233

Pattern 1.4 BRAINSTORMING**

(AKA ROUNDTABLE) … within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which several students face the generation of a large number of ideas.

If groups of students face the resolution of a problem whose solution requires the generation of a large number of possible answers/ideas in a short period of time, an adequate collaborative learning flow may be planned. The flow of collaborative learning activities to be followed in order to solve a task, whose resolution implies the generation of a large number of possible answers/ideas in a short period of time, might promote the following educational benefits (NISE, 1997; Millis & Cottell, 1998):

- To encourage learners to take risks in sharing their ideas - To demonstrate students that their knowledge and their language abilities are valued and accepted - To teach acceptance and respect for individual differences - To focus students’ attention on a particular topic

The solution for structuring collaboration so that a large number of ideas are generated may be ideally suited for newly formed groups, since they do not need to clarify their ideas. Therefore: Structure the learning flow so that students in the same group write down their answers to stated question. Explanations, evaluations, and questions are not permitted as the ideas are generated. This process might continue until students run out of possible solutions. After that, encourage each group to review and clarify their ideas. If needed, the group may present the generated ideas to the rest of the class.

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this Brainstorming structure preceded or followed by other set of activities, which can be organized as other patterns at the collaborative learning flow level – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6). If necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESSES (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1).

Patterns that complete this pattern: some of the Brainstorming phases might be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), TAPPS (Pattern 1.6), or activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the TPS structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3). Each brainstorming group may comprise a FACILITATOR (Pattern 4.1).


234

Pattern 1.5 SIMULATION **

(AKA ROLE-PLAY) … within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which the members of one or several groups perform a character in a simulation.

If groups of students face a problem whose resolution implies the simulation of a situation in which several characters are involved, an adequate collaborative learning flow may be planned. The flow of collaborative learning activities to be followed in order to solve a task, whose resolution implies the simulation of a situation in which several characters are involved, might promote the following educational benefits (Paulsen, 1995):

- To promote the feeling that team members need each other to succeed (positive independence) - To ensure that students must contribute their fare share (individual accountability) - To help students feel as well as understand the dynamics of a complex situation

The risk involved in charring out a simulation/role play is medium or high. Role-plays are usually hard to organize in large classes and that students may feel too shy or too time restricted to participate effectively in real-time simulations. Therefore: Structure the learning flow so that each student consults information about the problem/situation to be simulated and prepare the role of their character. Then, encourage the students in the same simulation group (usually small groups) perform a particular situation related to the problem. After that, the trained simulations may be performed to the rest of the class (large group). Finally, the whole class may discuss and share their conclusions about the problem.

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this Simulation structure preceded or followed by other set of activities, which can be organized as other patterns at the collaborative learning flow level – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), TAPPS (Pattern 1.6). If necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESSES (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1).

Patterns that complete this pattern: some of the Simulation phases might be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), TAPPS (Pattern 1.6), or activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the TPS structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


235

Pattern 1.6 THINKING ALOUD PAIR PROBLEM SOLVING (TAPPS) **

… within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, it may be necessary to plan how groups will perform a set interrelated activities. This pattern gives the organization of a collaborative learning flow for a context in which several students are paired and given a series of problems.

If students face a series of problems whose solutions imply reasoning processes, an adequate collaborative learning flow may be planned. The flow of collaborative learning activities to be followed in order to solve a series of problems whose solutions imply reasoning processes, might promote the following educational benefits (NISE, 1997; Millis & Cottell, 1998; Slavin, 1995):

- To foster discussion in order to construct students’ knowledge - To permit students to rehearse the concepts and produce a deeper understanding of the material - To encourage analytical reasoning skills - To support problem solving skills

The risk involved in structuring collaboration so that a series of problems are reasoned in pairs is medium. That is, the experience needed in collaborative learning needed is not too high. Therefore: Structure the learning flow so that students are paired and given a series of problems. Give the two students specific roles that switch with each problem: Problem Solver and Listener. The problem solver reads aloud and talks through the solution of the problem. The other (the Listener) follows the Problem Solver’s steps and catches any errors that occur. The Listener may ask questions if the Problem Solver’s thought process becomes unclear. The question asked, however, should not guide the problem solver to a solution nor should they explicitly highlight a specific error except to comment that an error has been made.

Patterns that complement this pattern: the learning flow of a whole educational unit might comprise this TAPPS structure preceded or followed by other set of activities, which can be organized as other patterns at the collaborative learning flow level – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5). In necessary, the learning flow can be enriched according to ENRICHING THE LEARNING PROCESSES (Pattern 1.7) or preceded by INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1).

Patterns that complete this pattern: the phase N+1 of the TAPPS phases might be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), or activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5). The groups indicated by the TPS structure may be formed according to FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


236

Pattern 1.7 ENRICHING THE LEARNING PROCESS

… within a collaborative learning scenario in which SCRIPTED COLLABORATION (pattern 11 from (E-LEN, 2005)) is seen as a remedy for situations where free collaboration does not lead to learning, this pattern proposes how to enrich the learning process for a synchronous context in which the process of the concurrent activities included in a scrip, which are performed simultaneously by different groups, is not the same.

How can the learning process be designed so that the (group of) students that perform some activities at faster rates can employ the time till the rest of the group finish (note that in collaborative learning synchronization of group activities is a key issue) to escalate the level and quality of the learning experiences? The reasons of why the progress of different group of students is different may be, for instance, the different backgrounds of the members of the groups, their organizational skills or the particular skills needed to perform a particular task (artistic skills, technological skills). Apart from the feeling of boredom that can appear among the groups that finish first (and might wait for the rest of the groups before continuing with the next activity, maybe because they have to form different groups), in Education it is advisable to provide students with the opportunities, resources and encouragement necessary to achieve their maximum potential without decreasing their motivation (Renzulli & Reis, 2005). Therefore: Apply the know-how of gifted education to improve the learning process (the collaboration script) in a way that some enriching challenging complementary activities are provided for the (groups of) students that already completed any (basic or curricular) activity of the design. Note that enriching activities might be planned following an organized approach with clear goals (related to the general objectives of the whole learning design) and a definable structure. Three types of enrichment activities can be considered: type 1 suggest exposing students to a wide variety of topics, hobbies, places that would not ordinarily be covered in the curriculum, type II consists of training general activities that promotes the development of processes such as creative thinking or communication skills, and type III may be devoted to students who become interested in pursuing a self-selected area and have the time necessary for advanced content acquisition and process training in which they assume the role of a first-hand inquirer.

Patterns that complement this pattern: enriching activities may precede, follow or be included within collaborative learning flow structures – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6).

Patterns that complete this pattern: enriching activities may be planned according to other collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), TAPPS (Pattern 1.6), or activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5).


237

A.2 Activity level

Pattern 2.1 INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS *

… within a scripted collaborative learning scenario whose flow of activities may be structured according to patterns at the collaborative learning flow JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6), this pattern proposes to consider an introductory activity explaining the collaborative learning design for a context in which meaningful learning and positive interdependence are desired to be fostered.

Students may be aware of the collaborative learning process that they will perform so that their learning is potentially meaningful and so that positive interdependence among the members of the groups is encouraged. This pattern discusses how this might be accomplished. One of the principles of instruction is that learning is facilitated when learners are shown the task that will be able to do or the problem they will be able to solve as a result of completing a module or course, i.e. learning is facilitated when learners are engaged at the problem or task level not just the operation of action level (the actions and operations that comprise the tasks). Showing learners the task or problem they will be able to solve is more effective that stating abstract learning objectives (Merril, 2002). On the other hand, many researchers (such as (Dillenbourg, 1999a)) in order to differentiate cooperation vs. collaboration, emphasise the contributions of group members and associate cooperative with division of labour procedures and collaborative with equality of contributions to the same problem solution. In this sense, collaboration scripts, which are often complex (risky) learning processes, need a high degree of positive interdependence so that the performance and interaction in a collaborative learning setting is successful (Strijbos et al., 2004). Positive interdependence refers to the perception that a member of the group is linked with others in a way so that (s)he cannot succeed unless they do (and vice versa); i.e., their work benefits (s)he and her/his work benefits them (Johnson & Johnson, 1999). It promotes cohesion and a heightened sense of belonging to a group. In order to promote the feeling that team members need each other to succeed, it is necessary to let students be aware of the whole collaborative learning process they will perform, so that they understand: Why the are going to collaborate? How is going to be the collaboration (coordination among groups, etc.)? How dependent is their performance on the performance of the others? Therefore:

Include in the learning flow an introductory activity that explains the whole learning design: present the task (or problem) they will solve and the flow (sequence) of activities they will perform (including the different groups they may form) in order to complete the task.

Patterns that complement this pattern: this type of introductory activity might precede other activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5).

Patterns that complete this pattern: a FACILITATOR (Pattern 4.1) may be in charge of keeping awareness of the whole learning design. Use GUIDING QUESTIONS (Pattern 3.3) to check if the learning design has been understood.


238

Pattern 2.2 DISCUSSION GROUP

A version of this pattern appears in (Goodyear, 2005).

… within a scripted collaborative learning scenario whose flow of activities may be structured according to patterns at the collaborative learning flow JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), there may be activities devoted to discussion. This pattern is mainly concerned with the establishment of appropriate organizational forms for knowledge sharing, questioning and critique.

Discussion groups are the most common way of organizing activity in networked learning environments. The degree to which a discussion is structured, and the choice of structure, is key in determining how successfully the discussion will promote learning for the participants. Discussions can be relatively structured or relatively unstructured, and they may also change their character over a period of time. It is not uncommon for a teacher to set up a discussion in quite a formal or structured way, and for the structure then to soften as time goes by - for example, as the participants take hold of the conversation, opening up and following new lines of interest. The structure of a discussion should be such that it increases the likelihood of:

a) an active and substantial discussion, with plenty of on task contributions b) the students coming away from the discussion with a good understanding of the contributions made c) contributions being made by all members of the group and 'listened' to by all other members of the

group. Unstructured discussions run the risks of (for example)

• not getting going properly within the time available • dissipating into a number of loosely related strands that fail to engage effectively with subject being

studied • dissolving into monologues or two way conversations that fail to involve the whole group.

(Pilkington & Walker, 2003) have demonstrated the value of assigning explicit group roles in online discussion groups. Some writers, for example, (McConnell, 2000) are not sure about the validity of the teacher setting specific structuring devices, preferring to make the group itself responsible for determining how it wants to discuss things, or carry out its work more generally. Therefore:

Start the discussion by establishing its structure. Make the rules and timetable for this structure explicit to all the members of the group. Where there is little time available to the group for the discussion, and/or the members of the group are inexperienced at holding online discussions, the teacher/facilitator should set the structure. Where the students are to set their own structure, the teacher/facilitator should give them support and ideas about how to do this, and encourage them to do so in a fair and timely way.

Patterns that complement this pattern: this type of activity might follow or precede other activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5).

Patterns that specialize this pattern: PREPARING FRUITFUL DISCUSSIONS USING SURVEYS (Pattern 2.3), ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4).

Patterns that complete this pattern: GUIDING QUESTIONS (Pattern 3.3), FACILITATOR (Pattern 4.1), FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


239

Pattern 2.3 PREPARING FRUIFUL DISCUSSIONS USING SURVEYS *

… an activity organized according to DISCUSSION GROUP (Pattern 2.2) may consider the results of a previous activity or their ideas about a subject posed by the teacher.

The exploration of contradictory views in a discussion can promote a deeper understanding of a subject. It can stimulate each participant to develop their own opinions and explore their reason for them. Discussions are not sometimes much fruitful because of a lack of structure of the ideas to debate. Another reason that causes this problem is that very often participants do not know the opinions and ideas of the rest of participants (Gómez et al., 2002; Martínez-Monés et al., 2005). Unstructured discussions run the risks of:

• not getting going properly within the time available, • dissipating into a number of loosely related strands that fail to engage effectively with subject being

studied, • dissolving into monologues or two way conversations that fail to involve the whole group.

Therefore: Before the discussion takes place, prepare a survey or questionnaire with questions related to the topics that might be particularly discussed. The students might answer the survey thus enabling them to organize their ideas and helping them to find arguments to defend their opinions on the main topics.

Patterns that complement this pattern: this organization may precede ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4) and might follow or precede other types of activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5).

Patterns that complete this pattern: MANAGING OF ON-LINE QUESTIONNAIRES (Pattern 3.2), GUIDING QUESTIONS (Pattern 3.3), FACILITATOR (Pattern 4.1).


240

Pattern 2.4 ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS *

… an activity organized according to DISCUSSION GROUP (Pattern 2.2) may consider the results of a previous activity or their ideas about a subject posed by the teacher.

Sometimes students are reluctant to challenge each other’s different views on a particular subject or the results from a particular activity during a discussion. When a student raises a different view or result after having being asked by the facilitator, the others have not reflected on the potential causes of the different approaches. Therefore, the other students avoid being involved in the discussion as they are not confident on what to argue (Johnson & Johnson, 1999; Gómez et al., 2002; Martínez-Monés et al., 2005). Therefore: Before the discussion takes place, the students should know, in advance, the others’ point of views or their outcomes from the learning activity they are going to discuss about. Also, they should have enough time to reflect on why there are different approaches. Sometimes, those reflections may generate cognitive conflicts enabling the students to notice that their opinions or their results may be wrong, thus generating new questions and new approaches to the discussed issue they had not thought of and thus generating learning. Sometimes (especially when there is not a unique answer to a question) the reflection on the differences may help the students to think of arguments to reinforce their opinions or to defend their results. The availability of those arguments may motivate the student to take part in the subsequent discussion.

Patterns that complement this pattern: this organization may follow PREPARING FRUIFUL DISCUSSIONS USING SURVEYS (Pattern 2.3) and might follow or precede other types of activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5).

Patterns that complete this pattern: STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1), GUIDING QUESTIONS (Pattern 3.3), FACILITATOR (Pattern 4.1), FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


241

Pattern 2.5 THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING

A version of this pattern appears in (TELL, 2005a) by S. Bartoluzzi and P. Goodyear.

… within a scripted collaborative learning scenario whose flow of activities may be structured according to patterns at the collaborative learning flow JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), there may be activities devoted to creation of an artifact to be assess. How we assess students’ work is one of the most significant decisions we make in educational design, not just because of issues of fairness and accuracy but also because how we set out to test students affects how they approach their work as learners. Assessment techniques need to be valid and reliable but they also drive learning.

Assessment regimes which prioritize technical measurement issues, such as validity and reliability, may ignore the effects of the test on students’ approaches to learning. On the other hand, we do need to assess students’ work, and our approaches must be fair and reasonable. Students take assessment tasks very seriously, especially when the grade they get for a task affects their final qualification, or the speed with which they progress to the completion of their studies. The nature of the assessment regime on a course unit affects how students approach their study (Biggs, 1999). For example, if the assessment regime consists mainly or exclusively of an end-of-course formal examination (time limited, unseen exam paper, no access to books or notes, etc), then students are much more likely to take a surface approach to study. This tendency will be strengthened further if they feel the curriculum is overloaded with content. Surface study strategies include rote memorisation and avoidance of reading material that is outside the core of the course. The knowledge developed through surface approaches tends to be inert and fragmented – hard to apply (Renkl, Mandl, & Gruber, 1996). Since we value the acquisition of flexibly organised, well-integrated and applicable knowledge – what can be called working knowledge – then we need assessment strategies that favour deep rather than surface learning (Biggs, 1999). Deep learning involves the personal construction of meaning, learning for understanding, learning processes which transform current conceptions, etc). Rather than leaving assessment till the finish of the course, where it marks the end of learning, it can be possible, and advantageous, to make the assessment task the vehicle for learning (Knight, 1995). An example is where the students’ main activity on a course is a project-like assessment task. This can also change the relationship between the materials students read and the assessment task. Instead of the assessment being a test of how well the materials have been memorised, the materials become a resource for the assessment project. A key aspect is that the assessment task must be one of the main things – if not the main thing – on which the students focus during their period of study. It is possible for the students’ work to be distributed across a small number of such tasks, but too great a number will create an incoherent learning experience. Therefore: Put a project-like assessment task at the heart of your course unit. Give students a lot of control over how they will carry out the task and make sure they have sufficient time to plan and execute it well. Let them have a strong voice in deciding exactly what the assessment task will consist of. Do not introduce other assessment tasks unless really necessary – such things can easily act as distractions and will dissipate the student’s intellectual energy.

Patterns that complement this pattern: this type of activity might follow or precede other activities – INTRODUCTORY ACTIVITY: LEARNING DESIGN AWARENESS (Pattern 2.1), DISCUSSION GROUP (Pattern 2.2).

Patterns that complete this pattern: STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1), GUIDING QUESTIONS (Pattern 3.3), FACILITATOR (Pattern 4.1), FREE GROUP FORMATION (Pattern 4.2).


242

A.3 Resource level

Pattern 3.1 STRUCTURED SPACE FOR GROUP TASKS

A version of this pattern appears in (TELL, 2005a) by S. Bartoluzzi and P. Goodyear.

… a collaborative activity – ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 1.7), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) – may require an online space to facilitate their work.

Sometimes students require an online space that facilitates their collaboration. In some situations, it makes sense to leave to each group the decisions about what tools, etc, they will use. This is particularly important where one of the intended learning benefits is that students become more capable at organising and managing their own online collaborative activity. (Learning to become a virtual team-worker, etc). However, in many cases, it simply distracts the group’s attention from the main task at hand and can make the early part of their work together much less effective. This is another case where getting the right balance between structure and freedom can be achieved through providing an adequate starting framework – in this case, a reasonably well-configured online space for a small group task – but ensuring that groups can modify the space to suit their own preferences and emerging needs. In many small group tasks, the group members need (i) somewhere to discuss their work (a space for planning, and monitoring the work as it goes along), (ii) somewhere to share a growing pool of relevant resources (e.g. useful papers they have identified, etc), (iii) somewhere to lodge the evolving versions of their joint product. Neither a discussion-oriented tool nor a shared editing tool is quite right for all purposes. Discussion tools, such as a threaded discussion forum are good for helping with the structure and flow of a discussion about planning, but don’t help much with document management, version control, etc. Document repositories can be good for sharing resources, and some will allow annotation. But they aren’t good for discussion. Collaborative writing tools, such as a wiki, are good for some kinds of joint document production, but aren’t so useful for discussing the process of document production. Ideally, one needs to be able to provide each of these things, in some reconfigurable, customisable environment. If students do not have the will or the skills to do the customisation, then what you provide must be adequate for their task. But it should not imprison those students who do have the skills and the will to improve the tools to hand. Therefore: Ensure that the set of collaboration tools you make available to students can support sharing of resources and products and group processes.

Patterns that complement this pattern: MANAGING OF ON-LINE QUESTIONNAIRES (Pattern 3.2).

Patterns that complete this pattern: depending on the collaboration tools available in the structured space the principles of other patterns may be considered – FACILITATOR (Pattern 4.1), FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).�


243

Pattern 3.2 MANAGEMENT OF ON-LINE QUESTIONNAIRES**

A version of this pattern appears in (Avgeriou et al., 2003; TELL, 2005a)

… a collaborative activity – PREPARING FRUIFUL DISCUSSIONS USING SURVEYS (Pattern 2.3), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) – may require the use of web-based questionnaires.

How can web-based questionnaires be created, delivered and graded? The administration of on-line tests for the assessment of students is a common task for the majority of learning systems. The creation and delivery of questions and tests over the Web is a complicated task due to the interactive, sophisticated nature of the web-based questionnaires. Therefore: Provide a mechanism for the creation of on-line questions: closed-end questions with predefined answers, that are able to be automatically graded and open-end questions, which need to be graded by an instructor. Allow the Instructors that create the questions, to be able to allocate a grade to each question. Also give them the ability to announce the schedule of on-line tests so that students are informed in time. Develop a run-time system for the delivery of the tests at the time scheduled, the automatic grading of closed-end questions, the automatic submission of answers to open-end questions to the Instructors and the storage of the results into the students’ records. In case of self-assessment questionnaires, assign particular questions to learning units where the student should check the knowledge she/he is supposed to have obtained. The run-time system should make these questions available to the students whenever they access the particular learning units.

Patterns that complement this pattern: STRUCTURED SPACE FOR GROUP TASK (Pattern 3.1).

Patterns that complete this pattern: GUIDING QUESTIONS (Pattern 3.3).


244

Pattern 3.3 GUIDING QUESTIONS

… a collaborative activity – DISCUSSION GROUP (Pattern 2.2), ENRICHING DISCUSSIONS BY GENERATING COGNITIVE CONFLICTS (Pattern 2.4), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) – may provide some hints supporting decision making about the completion of tasks.

A group of students that collaboratively perform a learning task are not sure on the criteria for deciding whether they have completed it or whether it fulfils the expected results. For some learning tasks in which the students do not have a clear knowledge of the expected outcomes, it may be difficult for them to decide when the task is completed. This may be due to their fear of not having done enough work or lack of ability for judging themselves. Some kind of conflict resolution (Johnson & Johnson, 1999) might be used for achieving a consensus on that but it would just be based on personal opinions and therefore the students, for the same reasons, would not be very confident on that. Also, the teacher might take the decision (or even just impose a time constraint) but the students would still not know why the task is completed. Therefore: Provide the students with a list of questions that they might be capable of answering as they advance with the task. The questions might not only deal with procedural issues (e.g., have you finished the introductory section of the document?) but mainly with the content of the activity itself. These questions would help the students to focus on important issues of the task as well as potentially generate cognitive conflicts with their previous knowledge or with the knowledge they are producing by means of the activity itself. Also, the students may be aware of the importance of self-posing questions on what they are learning as a way of enhancing and enlarging their knowledge (i.e. there might not only be an improvement of the task “tangible” outcome but also an improvement of the learning process itself).

Patterns that complement this pattern: STRUCTURED SPACE FOR GROUP TASK (Pattern 3.1), MANAGING OF ON-LINE QUESTIONNAIRES (Pattern 3.2).


245

A.4 Roles and common collaborative mechanisms level

Pattern 4.1 FACILITATOR*

Facilitating the work that a group has to collaboratively accomplish is a recurring problem in the context of collaborative learning flows – BRAINSTORMING (Pattern 1.4), activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) and supporting tools – STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1), MANAGEMENT OF ON-LINE QUESTIONNAIRES (Pattern 3.2).

Students might be guided towards greater independence (autonomous learning) in collaborative learning situations and, at the same time, towards effective collaboration. Autonomous learning is an important issue in education, which foster a greater independence of the students. This issue may be also considered in collaborative learning. Promoting self-organization helps to a large extent the achievement of greater independence. A group self-organizes by developing and sharing roles for team members, sharing workloads, etc. (Martínez-Monés et al., 2005). Allowing students to freely form groups and organize the work within groups might also promote students’ responsibility. However, fostering this independence should not damage effective collaboration (Paulsen, 1995; Davie, 1989). Therefore: Become a facilitator: motivate, introduce deadlines, help people get started, give them feedback, weave the contributions of different participants together, get it un-stuck when necessary, make sure all have opportunity to participate and learn, deal with individuals who are disruptive or get off the track, bring in new material to freshen it up periodically, and get feedback from the group on how things are going and what might happen next.

Patterns that complement this pattern: FREE GROUP FORMATION (Pattern 4.2), CONTROLLED GROUP FORMATION (Pattern 4.3).


246

Pattern 4.2 FREE GROUP FORMATION

Partly based on the patterns FORMING GROUPS FOR GROUP WORK WITHIN A CLASSROOM CONTEXT and FORMING GROUPS FOR COLLABORATIVE KNOWLEDGE BUILDING included in (E-LEN, 2005) by Gaby Lutgens.

Forming groups is necessary to comply with the types of groups indicated by collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6), the specific groups demanded by activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) and, even, the functionality provided by supporting tools – STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1). This pattern gives an approach to group formation for a context in which a group of students tackle a large demanding assignment.

How can a group of students be formed when they are asked to work on a large demanding assignment? Generally, groups should be heterogeneous, should not isolate minority students and should be formed by the teachers (NISE, 1997). However, it is important that students feel comfortable, especially when the assignment is large, demanding and product-oriented and have a strong importance related to grading. Simply allocating the same mark of every student in a group can lead to the problem of free-riders (Cronholm & Melin, 2006). In fact, there are many problems related to difference preferences that may emerge and obstruct learning. Different wishes about working times, geographical distance between the students, diverse study techniques or ways of thinking, differences in motivation (different level of ambition related to grading, commitments to the task and to the goal of the course) may lead to group conflict and non-creative group climate. Social sensitivity is an important aspect when assembling groups (Cronholm et al., 2006). Therefore: Ask students their opinion related to group formation. Let them form the groups themselves, if they prefer so. You might instead opt for a semi-free group formation approach where the students only select part of the members of their group.

Patterns that complement this pattern: FACILITATOR (Pattern 4.1).

Patterns that are alternative to this pattern: CONTROLLED GROUP FORMATION (Pattern 4.3).


247

Pattern 4.3 CONTROLLED GROUP FORMATION

Partly based on the patterns FORMING GROUPS FOR GROUP WORK WITHIN A CLASSROOM CONTEXT and FORMING GROUPS FOR COLLABORATIVE KNOWLEDGE BUILDING included in (E-LEN, 2005) by Gaby Lutgens.

Forming groups is necessary to comply with the types of groups indicated by collaborative learning flows – JIGSAW (Pattern 1.1), PYRAMID (Pattern 1.2), TPS (Pattern 1.3), BRAINSTORMING (Pattern 1.4), SIMULATION (Pattern 1.5), TAPPS (Pattern 1.6), the specific groups demanded by activities – DISCUSSION GROUP (Pattern 2.2), THE ASSESSMENT TASK AS A VEHICLE FOR LEARNING (Pattern 2.5) and, even, the functionality provided by supporting tools – STRUCTURED SPACE FOR GROUP TASKS (Pattern 3.1). This pattern gives an approach to group formation for a context in which a group of students tackle an assignment for limited duration and which benefits from diverse or conflict knowledge.

How can a group of students be formed when they are asked to work on an assignment to collaborative build knowledge? Heterogeneous groups with members with different skills and knowledge are considered to be more effective that homogenous groups in terms of sharing ideas and experiences to learn about topics or gain new insights. Heterogeneous groups provide opportunities to meet new people o people with different profile or divergent knowledge. This improves skills such as conflict management, ability to understand other people’s needs, communication and the ability to collaborate. They also prevent the isolation of minority students. However, it is difficult to find a method that guarantee balanced groups (NISE, 1997). On the other hand, randomly allocating students to groups is considered closer to real future professional life of the students (Cronholm et al., 2006). Therefore: There are many ways to form potentially heterogeneous groups. Assemble heterogeneous groups taking into account student outcomes in previous activities, their profile, or their academic strengths. You may instead form the groups randomly (e.g. count off students with numbers and ask the students who have each number meet), or considering a common characteristic not related with the task (e.g. ask the students born in the same month join).

Patterns that complement this pattern: FACILITATOR (Pattern 4.1).

Patterns that are alternative to this pattern: FREE GROUP FORMATION (Pattern 4.2).

APPENDIX B

WELL-KNOWN CSCL SCRIPTS:

RUNNING UNIVESANTÉ AND

ARGUEGRAPH WITH AN LD ENGINE

This appendix collects three well-known CSCL scripts. They have been extracted from (Kobbe et al., submitted) and are described according to a framework proposed in the same document. More information about the scripts can be found in (Dillenbourg, 2002). The appendix also includes the narrative use case descriptions and UML activity diagrams of Universanté and ArgueGraph scripts (as proposed in the basic design procedures when creating UoLs described in (Sloep et al., 2005)). Besides, it shows screenshots of their LD representations (UoLs) running in CopperCore (Martens et al., 2005; Vogten et al., 2006). Chapter Four provides the detailed analysis of their computational representation with LD. The ready-to-run Units of Learning are available at http://gsic.tel.uva.es/collage/scripts and in the attached CD-ROM.


250

B.1 Well-known CSCL scripts

Table B.1 Universanté script specified according to the framework proposed by (Kobbe et al., submitted)

Components Resources: Case descriptions from at least 2 themes, with at least 2 case descriptions per theme.

Participants: Participants from at least 2 nations, with at least as many participants per country as

there are case descriptions.

Groups: Theme groups, case groups and country groups.

Roles: none

Activities: analyzing, discussing; summarizing/synthesizing, presenting; analyzing, comparing,

discussing; presenting; giving feedback, critiquing; problem solving

Group formation and component distribution

For each case description one “case group” is formed, composed of at least 1 participant per

country, balanced among the groups. All case descriptions are distributed evenly among all case

groups. For each theme, one “theme group” is formed, composed of at all “case groups” with case

descriptions of this theme.(Universanté, 2002; Dillenbourg, 2002)

Sequencing Within each case group, all participants discuss the case.

Within each country group, the members of each theme group in turn present a synthesis of their

case experience.

Within each theme group, the members of each country group create a fact sheet concerning the

theme’s status in their country.

Within each theme group, all participants discuss the similarity and difference between the fact

sheets of different countries.

Within each country group, and for each theme group in turn, …

… the members of the theme group present their fact sheet

… everybody else provides comments on the fact sheets

… the members of the theme group modify their fact sheet according to the comments.

Within each case group, all participants propose a solution for the case problem.

RUNNING UNIVESANTÉ AND ARGUEGRAPH WITH AN LD ENGINE

251

Table B.2 ArgueGraph script specified according to the framework proposed by (Kobbe et al., submitted)

Components Resources: One copy of a questionnaire for each participant and another copy for each small group.

The questionnaire consists of multiple-choice questions (all choices are equally valid) with space

for providing an argument to justify the choice. One argument sheet per question of the

questionnaire and as many copies of these sheets as are needed to provide one copy for each

participant.

Participants: An even number of at least 4 participants (works best with 20-30 participants) and a

tutor.

Groups: Class group and small groups.

Roles: none

Activities: judging, writing arguments; comparing, interpreting, discussing; negotiating, writing

arguments; explaining, justifying; summarizing


In the “survey” phase, all participants together form the class group and receive one copy of the

questionnaire. In the “conflict” and “elaboration” phase, all participants are distributed evenly

among groups of two, composed of participants with maximal difference in their responses to the

questionnaire. Each small group receives another copy of the questionnaire.

Sequencing Within the class group, all participants individually fill out the first copy of the questionnaire.

The tutor displays the aggregated results of the questionnaire (the participants’ choices are plotted

as anonymous points in the graph) to the participants.

Within the class group, all participants jointly discuss the displayed results of the questionnaire.

“Conflict” phase:

Small groups are formed based on each participants’ responses to the questionnaire.

Within each small group, all participants jointly fill out the second copy of the questionnaire, i.e.

they negotiate on a single choice and generate a shared argument for this choice.

“Elaboration” phase:

The tutor collects all questionnaires.

For each question of the questionnaire, the tutor compiles all choices and arguments from each

small group on an argument sheet.

For each small group in turn, the tutor asks the participants to comment on their arguments and

gives advice on how to relate their arguments to theories and concepts.

“Reflection” phase:

The tutor distributes all copies of the argument sheets among all participants.

Within the class group, each participant individually writes a synthesis of all arguments on the

argument sheet, taking into account the advice of the tutor.


252

Table B.3 ConceptGrid script specified according to the framework proposed by (Kobbe et al., submitted)

Components Resources: a) Background information for each role, b) list of 16 concepts that relate to the

background information (4 concepts per background information) c) grid table, in which the

concepts and the relations between concepts can be inserted.

Participants: 4 participants for each group. A tutor.

Groups: Small groups.

Roles: 4 roles, based on theoretical background materials (names of real-life scientists are useful).

Activities: reading; linking concepts to theories; defining concepts from a theoretical point of view;

relating concepts to each other

Group Formation: Small groups of four are formed.


Each small group gets a copy of the grid table, the background information for each role and the list

of concepts. The roles are distributed among small group members by themselves.

Sequencing Within each group, …

… each participant reads background information related to his/her personal role.

… all participants decide on how to distribute the sixteen concepts among their roles.

… each participant writes a definition of the concepts allocated to his/her role.

… all participants discuss and decide how to arrange the concepts into a grid table and

how to relate them to each other.

… the tutor reviews the grids and points out omissions, inconsistencies and errors.


253

B.2 Universanté Unit of Learning

Table B.4 Narrative use case description of Universanté

Title Universanté Provided by Davinia Hernández-Leo, using work from P. Dillenbourg et al. (Berger et al.,

2001; Universanté, 2002; Dillenbourg, 2002) Pedagogy/type of learning Collaborative learning Description/context The Universanté script exploits socio-economic and cultural differences

between countries for teaching community health to medical students of different countries. The activities take place in a hybrid F2F and virtual space. (Supposing that there are 2 countries and 4 participants per country)

Learning objectives To learn (and teach) community health (by the confrontation of different national health context and different health issues) To introduce and reinforce general concepts of public health starting from particular clinical cases To investigate a health problem as a whole by presenting, among others, the epidemiological data and the prevention strategies for a given country To reinforce the previous objective by introducing methodological principles about public health data validation To familiarise the students with strategy planning intervention

Roles Thematic-group, Case-group, Country-group, Tutor Different types of learning content used:

Clinical cases (different for each case group) and forms to enter fact sheets and intervention strategies (The collection of links and the glossary have not been considered. However, the webpages including them (Universanté, 2002) could be easily added as learning objects in an environment)

Different types of learning services/facilities/tools used:

Discussion forums

Learning activity workflow (how actors/content/services interact):

Within each case group, all participants discuss a clinical case using a discussion forum; regularly the case groups with the same thematic gather in the same discussion forum and identify common points and differences between the cases; The tutor stimulates and guides both discussions; Within each country group, the members of each thematic group in turn present (F2F) a synthesis of their case experience; Within each thematic group, the members of each country group create a fact sheet concerning the thematic status in their country; Within each thematic group, all participants discuss the similarity and difference between the fact sheets of different countries using a forum; Within each country group, the members of each thematic group in turn present their fact sheets; The tutor prompts some methodological issues; Within each country group, the member of each thematic group (i.e. within each thematic group, the members of each country group) modify the fact sheet according to the methodological comments; Within each case group, all participants propose a health strategy to cope with the case problem; The tutor has access to the final fact sheets and health strategies.


254

Thematic-group TutorCountry-group

Guide the discussion

Case-group

Discuss the case

The members of each thematic group in turn present a synthesis

The members of each country group create a fact sheet

Discuss the fact sheets

The members of each thematic group in turn present the fact sheets

Prompt methodological issues

The members of each thematic group modify the fact sheet

Propose a health strategy Access to fact sheets and health strategies

Figure B.1 UML activity diagram of Universanté script

Table B.5 Users and their associations to roles (groups): “country group”

Country-group- Switzerland

Country-group- Cameroon

Colin Rob Daniel Hans

Juanma David Gemma Davinia


255

Table B.6 Users and their associations to roles (groups): “thematic group” and “case group”

Thematic-group-Cancer Thematic-group-AIDS Case-group- breast-cancer

Case-group- lung-cancer

Case-group- pregnant-aids

Case-group- drug-addicted-aids

Colin Rob Daniel Hans Juanma David Gemma Davinia

Table B.7 Screenshots of CopperCore running the LD-represented Universanté script

Case-group: Discuss the case Tutor: Guide the discussion

User: colin

Country-group: The members of each thematic group in turn present a synthesis

User: colin

User: Daniel


256

Thematic-group: The members of each country group create a fact sheet

User: colin

User: daniel

User: juama

User: gemma

User: rob (similar to colin’s, they share the fact sheet) (User: hans – view similar to daniel’s -, user: david – view similar to juanma’s -, user: davinia – view similar to gemma’s)


257

Thematic-group: Discuss the fact sheets

User: rob (idem for colin, juanma and david)

User: daniel (idem for hans, gemma and davinia)

Country-group: The members of each thematic

group in turn present the fact sheets Tutor: Prompt methodological issues

User: david


258

Country-group: The members of each thematic group modify the fact sheet

User: daniel

(Similar to the activity in which they create the fact sheet) Case-group: Propose a health strategy

User: colin

User: rob


259

User: juanma (like user: colin, same case)

(User: rob and user: david shared the health strategy as well, etc.)

Tutor: Access to the final fact sheets and health strategies


260

B.3 ArgueGraph Unit of Learning

Table B.8 Narrative use case description of the ArgueGraph script

Title ArgueGraph Provided by Davinia Hernández-Leo, using work from P. Jermann and P. Dillenbourg

(Jermann et al., 2002; Dillenbourg, 2002) Pedagogy/type of learning Collaborative learning Description/context The ArgueGraph script is used in the beginning of a master course on the design

of educational software. (Supposing that we only have 4 participants and the questionnaire includes only a question.)

Learning objectives To make students understand the relationship between learning theories and design choices in courseware development.

Roles Student, (Pairs: PairA and PairB,) Tutor Different types of learning content used:

Online questionnaires and forms to enter arguments

Different types of learning services/facilities/tools used:

Monitor (The informal discussion (survey phase) and the discussion between pairs are considered to be F2F.)

Learning activity workflow (how actors/content/services interact):

Survey phase: Students individually fill in a questionnaire about design principles in courseware development. They provide also a short written argument for their choices; This process is monitored and ended by the tutor; All learners then see all responses and jointly discuss them informally; The tutor forms pairs of students by selecting peers with the maximal difference in their responses to the questionnaire; Conflict phase: Within each small group (pairs), the participants negotiate on a single choice to the same questionnaire and generate a shared argument. They can read their individual previous answer; Meanwhile the tutor reviews all arguments produced by individuals and relates them to the various theoretical approaches in the domain (behaviourist, constructivist, socio-cultural…) Elaboration phase: The tutor reviews all arguments produced by pairs and relates them to the various theoretical approaches in the domain; Students read tutor’s comments on how to relate their arguments to theories and concepts; Reflection phase Each student individually writes a synthesis of all arguments, taking into account the advise of the teacher; This process is monitored and ended by the tutor.


261

Students/ Class Tutor Pairs

Fill in a questionnaire

Discuss informally

Answer the questionnaire

Form pairs

Read tutor’s comments Relate pairs’ arguments to theories

Write a synthesis

Monitor questionnaires

Relate individual arguments to theories

Monitor questionnaires

Figure B.2 UML activity diagram of ArgueGraph script

Table B.9 Screenshots of CopperCore running the LD-represented ArgueGraph script

Tutor Student Survey phase

User ines, select 1 (He’ll remember it the next time!); User bart, select 2 (The student is stimulated to think about his error); User asen, select 3 (Interacting with the teacher is always helpful);


262

User ale, select 4 (Now that he knows his previous answer is wrong)

Pair A: ines y asen Pair B: bart y ale

Conflict phase

Elaboration phase


263

Reflection phase


264

APPENDIX C

SUPPORT EVALUATION DATA OF THE

COLLAGE WORKSHOPS CASE STUDY

This appendix includes the support data regarding the evaluation of “Collage workshops” case study. The data are organized according to the topics and information questions forming the conceptual structure of this case. Each supporting argument has associated a coding, which identifies its data source (the raw data is available in the attached CD-ROM).


266

C.1 Pattern-based design process

Table C.1 Partial results and support data concerning the information question “is the selection of the CLFP-based LD templates and their representation useful and satisfactory?”

Partial results Coding of the data source Support data Document ‘UCA-quest-final-comments’, 1 passages, 33 characters, Paragraph 77, 33 characters (in Spanish).

“Collage systematizes the selection of patterns.”

Document ‘UCA-quest-final-selection-2’, 5 passages, 532 characters, Paragraph 77, 33 characters (in Spanish).

“The selection utility is a guide very useful in the process of building the educational design…”

Document ‘GSIC-EMIC-final-quest-engineering-teachers’, 3 passages, 390 characters, Paragraph 11, 52 characters (in Spanish).

“I like especially the suggestion of patterns…”

Document ‘GSIC-EMIC-final-quest-pedagogy-teachers’, 3 passages, 490 characters, Paragraph 3, 104 characters (in Spanish).

“… It is totally necessary to perfectly know the CL techniques, their phases and implications, in order to design the learning process correctly.”

Document ‘UVA-discussion-transcription’, 5 passages, 3302 characters, Paragraph 175, 597 313 characters (in Spanish).

“… a minimum formation on patterns is necessary, for which Collage is helpful…”

Document ‘UCA-quest-final-representation2’, 1 passages, 66 characters, Paragraph 23, 66 characters (in Spanish).

“The graphical representations and the examples are very illustrative.”

Document ‘UVA-discussion-transcription’, 9 passages, 2473 characters, Paragraph 14-18, 404 characters (in Spanish).

“Another thing that I like, independently that the result is later interpreted by an LMS, is the recommendation of patterns… The description of the patterns, their usefulness, etc. offer ideas to people that are not used to collaborative learning…”

Document ‘UVA-discussion-transcription’, 9 passages, 2473 characters, Paragraph 44, 160 characters (in Spanish).

“I think it is very well structured: the patterns, with the examples, the information, which is not large but sufficient to present the models…”


“I like the information help of each pattern, it facilitates a lot. Despite the fact of knowing the collaborative learning techniques, many doubts appear during the design that are easily solved thanks to this information.”


“I selected several desirable objectives and finally any pattern was suggested. I do not think such a rigid matching is necessary.”


“You might revise the selection of patterns… There can be patterns that do not strongly serve an objective but that do not constraint its achievement. Then, these patterns should not be removed from the recommended list”

Document ‘UCA-discussion-transcription’, 1 passages, 1067 characters, Paragraph 187-193, 1067 characters (in Spanish).

“I find the approach very flexible. However, it is not such when selecting a pattern. I would design the selection of patterns the other way around. That is, if I select a pattern, the types of objectives, types of problems, etc. that can be done with this pattern will be automatically shown…”

The “selection phase” (including information and an example of each pattern) is critical and promotes the understanding of the patterns. However, the suggestion of patterns in terms of their matching with the educational benefits should be made more flexible.

Document ‘UVA-quest-final-selection-2’, 1 passages, 237 characters, Paragraph 17, 237 characters (in Spanish).

“A possibility to improve the selection of patterns is to show the characteristics of the chosen pattern…”

SUPPORT EVALUATION DATA OF THE COLLAGE WORKSHOPS CASE STUDY

267

10 (out of 14) UCA interviewees select a), 3 selects b) and 1 select d).

3 (out of 5) UVA interviewees select a), 1 selects b) and 1 selects d).

5 (out of 7) UNFOLD interviewees select a) and 2 select d).

Closed question of the final questionnaires.

The participants are asked to value the CLFPs provided by Collage. They can answer: a) they are significant, they are relevant examples of CL techniques, b) they are adequate, there are more relevant examples but these are OK, c) they are not adequate, I do not know these techniques but I know others, d) I do not know any CL technique but it seems to me that these are adequate, or e) I do not know any CL technique and it seems to me that these are not adequate. 5 (out of 5) GSIC-EMIC interviewees

select a).


“I think that the flow patterns provided by Collage are the most relevant in the CL field. These patterns are often used…”

Document ‘UCA-quest-final-relevance-CLFPs2’, 1 passages, 88 characters, Paragraph 29, 88 characters (in Spanish).

“They are significant patterns, and can positively assist the design of a course”

The CLFPs in which the templates provided by Collage are based are significant, though there are more examples of well-known CL strategies (e.g. assessment techniques).

Document ‘UVA-quest-final-relevance-CLFPs2’, 3 passages, 654 characters, Paragraph 17, 255 characters (in Spanish).

“I think that Collage includes the most relevant techniques. However I miss a “pattern” that I employ in my classes, which is randomly selecting a member of a group to explain what the group has done.”

6 (out of 14) UCA interviewees select a), 6 selects b) and 2 select d).

3 (out of 5) UVA interviewees select a), 2 select b).


The participants are asked to value the CLFPs representations used in Collage. They can answer: a) adequate, my ideas about the techniques coincide with what is presented in Collage, b) acceptable, some of my ideas coincide with what is presented in Collage, c) they are not adequate, my ideas about the techniques do no coincide with what is presented in Collage, or e) I do not know the techniques formulated in the CLFPs.

4 (out of 5) GSIC-EMIC interviewees select a) and 1 selects b).

Closed question of the final questionnaire.

UNFOLD members are asked to grade in the range of 1 (not adequate) to 5 (very adequate) the way in which the patterns are represented.

6 (out of 7) choose 4 and 1 chooses 3.

Document ‘GSIC-EMIC-final-quest-teachers-teachers’, 3 passages, 418 characters, Paragraph 103, 161 characters (in Spanish).

“I think that the patterns are perfectly transferred to the user workspace, reproducing the needed roles and activities for their execution.”

Document ‘UVA-quest-final-reuse2’, 4 passages, 326 characters, Paragraph 14, 100 characters (in Spanish).

“The tool applies well the principles of the patterns.”

Document ‘GSIC-EMIC-final-quest-engineers-teachers’, 2 passages, 279 characters, Paragraph 3, 114 characters (in Spanish).

“It may be debatable whether there is just one way of presenting the patterns. This fact has to do with their flexibility.”


“I think that the patterns are adequate but a bit rigid…”


“The graphical representations of Collage help to understand the collaborative learning techniques.”

Document ‘UCA-discussion-transcription’, 1 passages, 271 characters, Paragraph 53, 271 characters (in Spanish).

“I think that one of the things of the tool that I like the most is the graphical representations of the patterns. I think they are very good…”

The representation of the CLFPs as LD templates is quite adequate (since a CLFP could be represented as several LD templates in terms of modelling and visualization). The visualizations as well as the interactive possibilities are especially useful. However, new (drag and drop) elements that enable to slightly modify the modelled learning flows would provide flexibility in the design.

Document ‘UNFOLD_quest&observations_report’, 2 passages, 354 characters, Paragraph 69-70, 225 character.

“Probably the representations will be improved in the future with other mapping elements and drag-and-drop elements.”


268

Table C.2 Partial results and support data concerning the information question “does it achieve a satisfactory

trade-off between reuse of CLFPs and the creation of scripts contextualized according to the situational needs?”

Partial results Coding of the data source Support data

12 (out of 14) UCA interviewees select Yes, while 2 select N/A.


The participants are asked to value if Collage helps to reuse the collaborative learning strategies proposed in the CLFPs. They can select: N/A, Yes or No. 4 (out of 5) UVA interviewees select

Yes, while 1 selects N/A.


“The patterns enables a better understanding regarding the design of collaborative learning situations”


“I think that the tool imposes an elaboration process that impedes the non-reuse.”


“The patterns provide a guide to structure the activity and prevent users forgetting important aspects.”

The steps in the “authoring phase” of the design process implemented in Collage facilitate the reuse of CLFPs when structuring collaborative learning designs (cf. also the last partial result of Table C.1).


The teacher rate the experience creating the design as successfully, “… in general I was able to create the design, though I did not thought-written it on paper before… The experience was quite dynamic and offered me new design ideas…”

UCA interviewees rate the utility of the combination of CLFPs with an average of 4.21 (deviation of 0.86).


The participants are asked to rate the possibility of combining CLFPs provided by Collage (in the range of 1 (it is not useful) to 5 (it is very useful)).

UVA interviewees rate the utility of the combination of CLFPs with an average of 4.20 (deviation of 0.75).

Document ‘UCA-quest-final-combination-2’, 7 passages, 557 characters, Paragraph 20, 47 characters (in Spanish).

“The combination of patterns enlarges significantly the possibilities.”


“I think that the combination of patterns allows a better adaptation of the activity to the problems and methods that we want to develop, making the activity more complete…”


“A positive aspect of Collage is the flexibility when combining patterns…”

Document ‘UNFOLD_quest&observations_report’, 2 passages, 317 characters, Paragraph 65, 136 characters.

“… it would be nice if other patterns could be added! However, as patterns can be combined, these already offer quite a lot of flexibility.”

The combination of patterns provides outstanding design flexibility.

Document ‘UVA_observations_byInesRuizRequies’, 2 passages, 251 characters, Paragraph 44, 164 characters (in Spanish).

“… they select and combine the patterns easily… in general the participants experiment by themselves…”


269


“I find especially interesting the possibility of combining patterns. It is very interesting from my perspective not to be constrained with just one option that we select at the beginning and it is closed when designing the activities… but being able of combining different patterns seems to me very, very interesting…”

Document ‘UVA-quest-final-combination-2’, 4 passages, 565 characters, Paragraph 11, 178 characters (in Spanish).

“Combining patterns is quite frequent when using active learning; I have often combined several techniques…”

UCA interviewees rate the adaptation of CLFPs with an average of 3.79 (deviation of 0.86).


The participants are asked to rate to what extent they would be able to adapt the CLFPs to the characteristics of their courses using Collage (in the range of 1 (almost nothing) to 5 (totally adapted)).

UVA interviewees rate the adaptation of CLFPs with an average of 2.80 (deviation of 0.40).

Document ‘UCA-quest-final-context-2’, 4 passages, 414 characters, Paragraph 38, 184 characters (in Spanish).

“The possible combinations of patterns offer a degree of flexibility sufficient to adapt a large variety of activities…”


“I find the patterns very interesting and adaptable to my practice as a docent…”

Document ‘UCA-final-reuse-2’, 1 passages, 159 characters, Paragraph 38, 159 characters (in Spanish).

“The same structure may be useful for different courses / environments, simply changing the resources, the definition of groups, objectives and activities.”


“For the design I had in my head, it was adequate… I was able to do what I wanted, at least at the flow level…”


“It enables the generation of contextualized learning processes that each teacher should prepare when planning…”

Document ‘UVA-discussion-transcription’, 5 passages, 3302 characters, Paragraph 169-171, 743 313 characters (in Spanish).

“No example is directly transferable… I think that it is good to know an example in order to have something like a “demo”… but, at the end, each teacher has his circumstances and everything changes…”

Document ‘UCA-discussion-transcription’, 3 passages, 965 characters, Paragraph 119 and 123, 354 and 313characters (in Spanish).

“… with this example we have a “pattern” of how to accomplish a collaborative learning experience in which the student work with an article or whatever… we can have general schemas of different level of depth, because when we want we can incorporate a new current resource so that the unit of learning is updated and more attractive… and can be reused.”


“When the use of the tool would be broadly adopted, it will be helpful to have previous experiences, if a viewer of examples exists… ”

Collage templates (reflecting a CLFP or a combination of CLFPs) can be to a large extent particularized according to the needs of the learning situations. Pattern-based templates are probably more useful in the process of customizing a new situation than examples, but complete (or partly complete) examples are also helpful.


“The examples are useful especially if the results are available… like in a community of practice…”


270

10 (out of 14) UCA interviewees select Yes, while 4 select N/A.


The participants are asked to value if Collage achieves a satisfactory trade-off between flexibility and keeping the essence captured in the CLFPs. They can select: N/A, Yes or No.

4 (out of 5) UVA interviewees select Yes, while 1 selects N/A.


“… Collage is flexible and keeps the essence of the patterns. I think that the most important aspect of the tool is that it allows us to rationalize our labour without the need of too much technological effort… ”

Document ‘UCA-quest-final-comments’, 1 passages, 253 characters, Paragraph 53, 236 characters (in Spanish).

“Designing with Collage is easy (at least it is not complex at all). It allows articulating group work processes with a flexibility margin which is not small…”

Document ‘UVA-quest-final-comments’, 2 passages, 211 characters, Paragraph 11, 96 characters (in Spanish).

“It provides a very good guide to plan collaborative learning activities.”

Document ‘UVA-quest-final-design-4’, 3 passages, 515 characters, Paragraph 14, 158 characters (in Spanish).

“Collage pushes the user to make decisions that any good professional would determine…”


“It saves the teacher a lot of specification workload.”


“In principle I do not want more flexibility, … if we have to apply something, let’s apply it truly, we should reflect a bit on the steps…”


“I do not find the process too constrained since it enables the combination of patterns… and I think that it is easier for a novice to have an already structured model…”


“The guidance by the tool to me is fundamental. With just a paper, I forget things… It is very important to know that there is control in the learning process that I have design because I have filled everything…”


(Regarding the fact that the tool does not enforce a sequence of authoring steps.) “… all the parameters are at the same level of importance… it is necessary to manipulate all at the same time when we are making the design decisions…”


“The number of things to be done, and the various places where they have to be put, make this tricky for teachers…”

Collage achieves a satisfactory trade-off between flexibility, keeping the essence captured in the CLFPs, hiding technological LD-specific details and providing a clear (but limited) set of design options. However, since these design steps are not presented in a rigorous sequence, some of them may be forgotten.


“… the fact that Collage helps to structure, and indicates what to fill … because sometimes when I make a design..., I have fear of missing details… and here the structure is provided, because it is based on patterns, what offers you security in the sense that the aspects already filled are already thought… and if this table indicating what is missing is offered by the tool, it is even better”


271

C.2 Focus on CSCL critical elements

Table C.3 Partial results and support data concerning the information question “does the design process implemented in Collage help to determine the learning objectives, task-type and expected interaction that will

be develop?”


UCA interviewees rate the help to determine the learning objectives with an average of 3.86(deviation of 0.74).


The participants are asked to rate if the selection of CLFPs and the associated information help to determine the educational objectives that will be promoted (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the learning objectives with an average of 3.40 (deviation of 0.49).

UCA interviewees rate the help to select the task-type that will be solved by the students with an average of 3.43 (deviation of 0.73).


The participants are asked to rate if the selection of CLFPs and the associated information help to determine the task-type that will be solved by the students (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to select the task-type that will be solved by the students with an average of 3.20 (deviation of 0.75).


“… in this way we are not just reflecting on what we are already doing in our practice… but also developing transversal competences and other things. Then, it is clear that this tool can facilitate… because it considers many competences that the students should develop independently of the courses…”


“It helps to determine the CLFP more suitable to foster particular objectives.”


“It helps quite a lot… they are checked proposals which are largely objective … ”

The design process helps to determine the learning objectives related to collaborative learning that will be promoted (cf. also the first partial result of Table C.1) and to select the task-type that will be solved by the students.


“The help is useful but I think that it might constrain… (it is good as an orientation)”

UCA interviewees rate the help to determine the expected interaction with an average of 3.62 (deviation of 0.62).


The participants are asked to rate to what extent Collage helps to determine the expected desired type of interactions (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the expected interaction with an average of 3.80 (deviation of 0.75).

Document ‘UCA-quest-final-design-2’, 1 passages, 67 characters, Paragraph 11, 67 characters (in Spanish).

“It helps to envisage how the interactions will develop…”

The design process helps to determine the expected interaction (discussing, reasoning…).


“When we describe the activities, we need to reflect on how we will achieve the objectives we have in mind…”


272

Table C.4 Partial results and support data concerning the information question “does the design process implemented in Collage help to understand and determine the structure regarding the flow of activities and

the hierarchy of groups?”


UCA interviewees rate the help to understand the flow of activities with an average of 3.86 (deviation of 0.74).


The participants are asked to rate to what extent the selection of CLFPs and the associated information help to understand the flow of collaborative learning activities proposed by the CLFPs (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to understand the flow of activities with an average of 4.00 (deviation of 0.00).

UCA interviewees rate the help to determine the flow of activities with an average of 3.64 (deviation of 1.04).


The participants are asked to rate to what extent Collage helps to determine the flow of coordinated activities of the final script (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the flow of activities with an average of 4.40 (deviation of 0.80).

Document ‘GSIC-EMIC-final-quest-pedagogy-teachers’, 3 passages, 327 characters, Paragraph 53-55, 327 characters (in Spanish).

“Collage is useful because it helps to think in terms of collaborative learning and its previous arrangement.”


“To me the best part is the one related to the “flow”…”

Document ‘UCA-quest-final-selection-6’, 5 passages, 262 characters, Paragraph 29, 85 characters (in Spanish).

“It is quite clear the procedural learning sequence that is offered as a pattern…”


“The organization of the work is structured and coordinated easily, since it is realized more or less in a guided way…”


“It highlights the inter-relation among activities…”


“The hierarchy of the process provides an image of this coordination.”


“I like having a tool that facilitates the design of collaborative work. It helps to organize and structure our teaching activity…”


“I think that the most important aspect of Collage is that it helps to structure and systematize the different types of activities… At least, Collage has helped me to structure what I already do but that I do not organize well…”


“I find the tool quite powerful, especially concerning the association of activities to groups.”


“… Since the combination of patterns is enabled, it provides flexibility… it pushes the user to “fill” the designs and thus reflecting on the different issues that should be considered…”

The design process helps to understand and determine the structure regarding the flow of activities (cf. also the first partial result of Table C.1).


“It helps to structure the envisioning of the teacher.”


273

UCA interviewees rate the help to understand and determine the hierarchy of groups with an average of 3.43 (deviation of 0.90).


The participants are asked to rate to what extent Collage helps to determine the different types of groups and the relationships between them (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to understand and determine the hierarchy of groups with an average of 4.40 (deviation of 0.49).


“The structure of the process helps to check the roles that each group plays in each moment.”

The design process helps to understand and determine the structure regarding the hierarchy of groups.


“… it helps to structure the work in groups and, thanks to the graphical representations, it helps to conceptualize the groups… these are the things that I like the most…”

Table C.5 Partial results and support data concerning the information question “does the design process support also the definition of group-size, resource distribution, computer support and the structure within

activities?”


UCA interviewees rate the help to determine the group size with an average of 3.66 (deviation of 1.11).


The participants are asked to rate to what extent Collage helps to reflect on and flexibly determine the size of groups (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the group size with an average of 3.60 (deviation of 1.74).


“It helps to determine the groups… but it depends on the number of involved students…”


“The information help provides clues and the interface of Collage allows the introduction of the desired group size limits.”

The design process helps to establish the group size, though it can be improved with automatic checkups.


“It seems to me that this is the weakest point of the tool… I do not know if the system validates the quantities specified to each group so that the whole is coherent…”

UCA interviewees rate the help to determine the distribution of resources with an average of 3.64 (deviation of 0.97).


The participants are asked to rate to what extent Collage helps to determine the distribution of resources (content and tools) needed to support each activity (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the distribution of resources with an average of 3.60 (deviation of 1.20).

UCA interviewees rate the help to determine the computer support with an average of 3.80 (deviation of 1.98).


The participants are asked to rate to what extent Collage helps to determine the computer-support of each activity (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the computer support with an average of 3.60 (deviation of 1.74).

Document ‘EMIC-GSIC-final-pedagogy-teachers’, 2 passages, 194 characters, Paragraph 104, 117 characters (in Spanish).

“It helps to structure a complex learning design and promotes times and resources planning.”

The design process helps to determine the structure regarding the resource distribution and the computer support, though it does not provide suggestions of recommended content or tools.

Document ‘UCA-quest-final-design-11b’, 3 passages, 134 characters, Paragraph 38, 92 characters (in Spanish).

“The design implies the previous determination of all these elements and their organization…”


274

Document ‘UVA-quest-final-design-11b’, 4 passages, 557 characters, Paragraph 11, 228 characters (in Spanish).

“The design of a unit of learning using Collage includes all these aspects. Since Collage guides the design process and the activities of each phase, we are obliged to reflect on each element that shapes the activity…”

Document ‘UVA-quest-final-design-11b’, 4 passages, 557 characters, Paragraph 17, 125 characters (in Spanish).

“Collage pushes to specify resources and tools, though their characteristics should be known beforehand.”

UCA interviewees rate the help to determine the structure of each activity with an average of 3.50 (deviation of 1.18).


The participants are asked to rate to what extent Collage helps to determine the structure of each activity (in the range of 1 (nothing) to 5 (a lot)).

UVA interviewees rate the help to determine the structure of each activity with an average of 4.80 (deviation of 0.40).


“This information should be clear when thinking and describing each activity…”


“We assign the objectives and work that each group will accomplish in each moment…”

The design process helps to describe each activity and its eventual (textually-defined) structure (cf. also the first partial result of Table C.3).


“In the Anatomy course we work with groups… in each of which there is a boss…”

C.3 Use of Collage

Table C.6 Partial results and support data concerning the information question “can the teachers use successfully Collage?”


8 (out of 14) UCA interviewees select Easy, 4 select Acceptable while 2 select Difficult.

4 (out of 5) UVA interviewees select Easy, and 1 selects Acceptable.


The participants are asked to value the use of Collage. They can select: Difficult, Acceptable or Easy. 4 (out of 5) GSIC-EMIC interviewees select

Easy, and 1 selects Acceptable.


The participants are asked to rate the use of Collage (in the range of 1 (not user-friendly at all) to 5 (very user-friendly))

UNFOLD interviewees rate the user-friendliness of Collage with an average of 3.86 (deviation of 0.88).

Document ‘UCA-quest-final-easy-to-use, 2 passages, 147 characters, Paragraph 23, 106 characters (in Spanish).

“I am satisfactorily astonished by the user-friendliness of the tool; I thought that it was going to be more difficult…”

Document ‘UCA-quest-final-easy-to-use, 2 passages, 147 characters, Paragraph 31, 41 characters (in Spanish).

“It is very visual and with a very logic structure.”


“It is easy to download and to install.”

Most of the participants find Collage user-friendly and intuitive.


“I think that the level of difficulty is acceptable.”


275


“Collage is intuitive, user-friendly and does not provoke an “insecurity” feeling… ”

Document ‘UVA-quest-final-easy-to-use, 2 passages, 132 characters, Paragraph 17, 95 characters (in Spanish).

“It has a user-friendly and intuitive interface. The pattern figures facilitate its use.”

Document ‘UVA-quest-design-9’, 1 passages, 74 characters, Paragraph 17, 74 characters (in Spanish).

“It also helps the functionally of informing about the empty parts…”

Document ‘UVA-observations_byBartolome’, 1 passages, 266 characters, Paragraph 18, 266 characters (in Spanish).

“… during the experience the participants affirm the user-friendliness of the tool…”

Document ‘UVA-observations_by Unes’, 1 passages, 173 characters, Paragraph 44, 173 characters (in Spanish).

“… the participants create the design without difficulty…”


All say, “Yes, yes, it is quite intuitive.”


“It is so intuitive that it encourages us not to follow strictly the steps you were indicating… and to experiment by ourselves…”

Document ‘UNFOLD_quest&observations_report2’, 4 passages, 659 characters, Paragraph 25, 124 characters (in Spanish).

“When installing Collage, there was not any problem. We could install it even in Lisa’s Mac…”

Document ‘GSIC-EMIC-engineering-teachers’, 2 passages, 142 characters, Paragraph 2, 35 characters (in Spanish).

“Collage is easy to use and quite intuitive.”

Document ‘GSIC-EMIC-pedagogy-teachers’, 2 passages, 513 characters, Paragraph 104, 165 characters (in Spanish).

“I would say that the usability is more than adequate. I think that very few problems have appeared when using Collage in this session…”

UCA interviewees rate the successfulness when creating the example with an average of 3.36 (deviation of 0.97).

UVA interviewees rate the successfulness when creating the example with an average of 3.20 (deviation of 1.02).


The participants are asked to rate the successfulness they have when creating an example during the workshop (in the range of 1 (I have not been able of doing nothing) to 5 (I have been able to complete the example without problems)). UNFOLD interviewees rate the successfulness

when creating the example with an average of 4 (deviation of 1.07).


The participants are asked to value how successfulness is the creation an example during the workshop. They can select: Not-successful (I have been not able to complete the example), Acceptable (I have some problems but I manage to complete the example) or Successful.

5 (out of 5) GSIC-EMIC interviewees select Successful.

Document ‘UCA-quest-final-problems’, 2 passages, 185 characters, Paragraph 2, 185 characters (in Spanish).

“I have not finished it completely because of a lack of time, but it has worked correctly…”


“I was able to create an IMS LD reflecting a previous CL experience without deep knowledge on IMS LD.”

The participants are able to create an almost completed example during the (short) workshops.


“I have the feeling of having done something useful and real”


276

Assessment of the scripts created in the GSIC-EMIC experiences.

CopperCore correctly validates the UoLs, and the integrated LDs largely describe the learning situation of the example (in the case of the pedagogy teachers) However, all participants fail to particularize a specific activity, i.e., they do not enter a description or associate resources to the activity. This indicates the need to clearly specify the activities which have not been completed.


The participants are asked to indicate if they have problems when using Collage. They can select: Yes or No.

5 (out of 7) UNFOLD interviewees select No, and 2 selects Yes.


“During the use of the tool, there were just minor problems. Simply, it is worth mentioning the problem when configuring the services…”

Document ‘UNFOLD_quest&observations_report2’, 4 passages, 607 characters, Paragraph 72-80, 438 characters.

“… only minor bugs that you know about…”

“navigation in the expansion window was confusing, needs time to familiarize”

“I was confused that I didn’t find an environment tab but all the environments were “resources”. This is not a problem for most users but could be indicated for LD aware users…”


“Still a lot to be filled or to be set. Maybe even possibility with predefined services…”

Document ‘GSIC-EMIC-pedagogy-teachers’, 1 passages, 110 characters, Paragraph 34-35, 110 characters (in Spanish).

“The field to define the title is hidden…”

Document ‘UVA-quest-final-problems’, 2 passages, 218 characters, Paragraph 8, 54 characters (in Spanish).

“It is not possible to delete a pattern, once it is added…”

Few problems appear.

Document ‘UCA-quest-final-improvements’, 7 passages, 737 characters, Paragraph 23, 59 characters (in Spanish).

“I would like that the number of the pyramid levels could be modified…”

Document ‘UCA-quest-final-interest-variation’, 6 passages, 997 characters, Paragraph 5, 119 characters (in Spanish).

“It is an alternative that should be applied with students…”

It is important to apply the scripts created using Collage with students. Document

‘UNFOLD_quest&observations_report2’, 3 passages, 718 characters, Paragraph 37, 201 characters (in Spanish).

“The questions at the end of the session were related to the difficulty of including level B in Collage or the importance of experiencing the execution of UoLs with real students.”

7 (out of 7) UNFOLD interviewees select Yes. Closed question of the final questionnaires.

The participants are asked to indicate if they find Collage useful. They can select: Yes or No. 5 (out of 5) GSIC-EMIC interviewees select Yes.

Closed question of the initial questionnaires.

The participants are asked to indicate if they have used LD compliant tools.

Only the UNFOLD interviewees and few of the UCA interviewees have used IMS LD compliant tools. (Mainly Reload.)


“At last, patterns in practice!”

Compared to other LD compliant tools, Collage is easier to use, specific to collaborative learning, and it is the first editor providing pattern-based templates.

Document ‘UNFOLD_quest&observations_report’, 1 passages, 386 characters, Paragraph 100-109, 478 characters.

“I am interested in using it, to show /present what kind of CL patterns in conjunction with LD can be used.”


277


“Collage seems to add some important features which will facilitate that teachers access (and use) LD authoring tools.”

“I like the graphical representation.”

Document ‘UNFOLD_quest&observations_report2’, 6 passages, 586 characters, Paragraph 93, 146 characters.

“Less complicated and more intuitive than Reload on its own…”


“Makes Reload much easier to use for specific purposes.”

Document ‘UCA-quest-final-compare, 3 passages, 207 characters, Paragraph 11, 68 characters (in Spanish).

“Reload does not implement collaborative work schemas…”

Document ‘UCA-quest-final-compare, 3 passages, 207 characters, Paragraph 41, 44 characters (in Spanish).

“Collage guides the user easily.”

Table C.7 Partial results and support data concerning the information question “how can Collage be

improved?”



“It is clear that it is better if it supports more CLFPs, level B…”

Document ‘UNFOLD-quest&observations_report’, 2 passages, 266 characters, Paragraph 134, 194 characters (in Spanish).

“Other patterns… levels B and C. But the advantage of a relative simple editor could disappear if it is extended maybe…”


“Sometimes it is too simple, it may happen that advanced users cannot specify things that they would like…”


“… when I apply the Jigsaw, I also apply “continuous evaluation” in order to make the students participate well… ”


“… a final vision of what has been design … and also the incorporation of activities that are not part of a pattern but that may be complementary – external…”


“I would suggest that the tool generates a document with the resulting design…”

Document ‘UNFOLD-quest&observations_report’, 2 passages, 266 characters, Paragraph 135, 72 characters (in Spanish).

“Make a “search and replace” that clones a UoL with different resources.”

Extensions with more CLFPs, other types of patterns and other complementary activities (i.e. more flexible design options, cf. also the last partial result of Table C.1), automatic calculation of group size (cf. also the first partial result of Table C.5), tracking information about what has been completed (cf. also the last partial result of Table C.2) and integration with the player.


“After associating a resource with several activities, if the resource is deleted, then the tool does not inform the user about that…”


278


“I think that it would be very useful to make an inverse use of the selection of patterns. In addition to that when selecting an objective, the list of patterns is limited… when selecting a pattern, the objectives and types of problems that the pattern best serves were highlighted…”


“Indicating the number of students, automatically, the number of members of each group in each activity may be calculated.”


“I would like that the tool informs me about which aspects have been completed successfully and which are not completed yet…”


“The intercommunication with the LMS.”


“The information provided by the CLFPs is sufficient concerning the design… but in order to have the complete view the assignment of participants to roles is missing…”


“The next step is to create groups with particular users…”


“I cannot execute the design created with Collage in a user-friendly environment…”


“I miss special viewers; players… the data resulting of the execution of the LD are not exported for evaluation purposes…”

Document ‘UVA-quest-initial-class-plan’, 3 passages, 454 characters, Paragraph 8, 284 characters (in Spanish).

“Before the fall semester we plan the objectives, activities and timing of the courses. However, the planning cannot be closed because many times external circumstances appear (e.g. holidays, illnesses).”

Document ‘UVA-quest-initial-class-plan’, 3 passages, 454 characters, Paragraph 38, 91 characters (in Spanish).

“I revise the plan of each session considering the real development of the course”


“Theoretically it is a good guide, but we should always have in mind that the groups evolve differently depending on their members…”

Document ‘UVA_observations_byInesRuizRequies’, 1 passages, 503 characters, Paragraph 32, 503 characters (in Spanish).

“… the participants are worried about how to make the adaptations needed when they do not know the students that will attend the classes…”


“… if some students abandon the course in the third session, then I need to re-structure…”

The integration of Collage with complementary needed tools (production and delivery systems) should support flexibility (e.g. changes on the fly).


“… the structure can be kept if the student number variation is within a range…”


279


“in the TTG course it also happens to us… we apply a Jigsaw and in some sessions expert groups do not attend the session… then we just form new groups comprising two jigsaw groups…”


“There are other cases such as NNTT, in which a Jigsaw is applied along the whole course… then the problem is again that not all the students attend all the sessions… then since the objectives are at the long-term the problem is not so important… because at the end they have to contribute, unless they abandon the course…”


“… another thing if all this works in the whole cycle, adapting on the fly is more complicated…”

Document ‘UVA-quest-final-representation2’, 1 passages, 239 characters, Paragraph 8, 239 characters (in Spanish).

“I think that this type of experiences requires a flexibility that impedes the planning of complex tasks (e.g. the whole course). It may be allowed the modifications of the activities as time goes by…”


“Since the structure is clearly planned, it may also facilitate that the students know it well and can follow it easily… so that the workload of the teacher decreases…” Other uses: the information created

with Collage is also helpful for the students.


“… we can take advantage of the tool not only from the teacher point of view but also from the evaluator and student perspective… that the students can follow their own progress…”

C.4 Potential audience characteristics

Table C.8 Partial results and support data concerning the information question “which are the characteristics and motivations of the potential audience of Collage?”



“Collage is a very interesting tool to design collaborative processes.”

Document ‘UVA-quest-final-interest-variation’, 3 passages, 402 characters, Paragraph 9, 161 characters (in Spanish).

“I have already carried out collaborative learning experiences, and this tool seems to be very useful to plan this type of activities.”


“Collage is very well designed for collaborative learning and it seems that we need to get used to it hereafter…”


“I think it should be a helpful tool to enable teachers to plan CL activities.”

Motivation and interest in designing collaborative learning processes to be used with an LMS in face-to-face, distant or blended situations or as lesson plans (without being interpreted by a player).

Document ‘UVA-quest-initial-expectatives1’, 3 passages, 435 characters, Paragraph 11, 159 characters (in Spanish).

“I am interested in knowing new methods for university education that improve the learning of my students.”


280


“I am currently modifying my way of carrying out my classes, and the use of patterns can help me.”


“I think that when we start working with the European credits, this tool can be good…”

Document ‘UCA-quest-initial-LD-valuing2’, 1 passages, 172 characters, Paragraph 11, 172 characters (in Spanish).

“It is going to be necessary to use this type of tools in our educational system. The EEES will also push to use these tools…”


“I think that the tool can be of great utility in order to apply this kind of techniques, now that it seems one of the methods to apply in our classes…”


“I am here because of my need as an university teacher … and because of the European convergence… and because we need to interchange our experiences…”


“I currently use the ICT in virtual education with individual learning and applying collaborative learning just in face-to-face situations… It is a good idea to blend both aspects, it broadens the possibilities of work and its control…”


“I was commenting that a possibility is that once we have a design, we can reproduce it in Moodle…”


The participants are asked to indicate what type of learning situation they are thinking of (see the questionnaires available in the CD-ROM, for the list of possibilities).

All of GSIC-EMIC interviewees think of a blended situation in the sense that it is partially supported by technology. 2 (out of 5) of a synchronous situation and the other 3 of a mixed synchronous and asynchronous situation. 3 think of a face-to-face situation and the other 2 of a blended situation mixing distant and face-to-face activities.


“Another thing that I like, independently that the result is later interpreted by an LMS, is the recommendation of patterns… The description of the patterns, their usefulness, etc. offer ideas to people that are not used to collaborative learning…”


“I do not find the process too constrained since it enables the combination of patterns… and I think that it is easier for a novice to have an already structured model… because if the design is finally not useful can be modified without using this tool, because it can be also applied without computer support…”

6 (out of 18) UCA interviewees selects a), 9 select b), 2 select c), and 1 selects d).

1 (out of 8) UVA interviewees selects a), 4 select b), and 3 select c).

3 (out of 7) UNFOLD interviewees select a), 2 select b) and 2 selects c).


The participants are asked to value their experience and interest in applying CL to education. They can select: a) I usually apply collaborative learning, b) I sometimes apply collaborative learning, c) I hardly apply collaborative learning but I am interested or d) I hardly apply collaborative learning and I am no interested.

4 (out of 5) GSIC-EMIC interviewees select a), and 1 selects b).

UCA interviewees value the importance of collaborative learning in their future practice with an average of 3.78 (deviation of 0.92).

(University) teachers, novice or with experience in CL practice. The design process provides ideas and foments the design creativity of the users, even promoting an increase in their interest in CL and the application of ICT.


The participants are asked to value (in the range of 1 to 5) the importance of collaborative learning in their future practice.

UVA interviewees value the importance of collaborative learning in their future practice with an average of 3.29 (deviation of 0.70).


281


“Collage is easy to use if the user is used to develop educational processes, in which the planning and definition of activities is essential…”


“… in general I was able to create the design, though I did not thought-written it on paper before… The experience was quite dynamic and offered me new design ideas…”


“The selection of patterns can be of great utility to persons that have never used a particular type of patterns. When the teacher has sufficient experience using the patterns, then he would probably value personally the use of each pattern…”


“For the novice teachers, a tool that guides users in the design process of units of learning is essential.”


“It enables the direct implementation of collaborative learning systems without a great expertise.”


“… this is what I was discussing during the coffee break, about the complexity or risk of each pattern… it is not the same to use a Jigsaw than a Pyramid, it depends on the level of student compromise, the collaborative learning experience… then the tool also suggests: if you are starting practising collaborative… and the students do not seem to be prepared to collaborate, then use the TPS, which is easier, and do not use the Jigsaw… ”


“After using Collage, I have interest in collaborative work; I will try to put it in practice…”


“My interest in the ICT has been reinforced…”

Document ‘UVA-quest-final-interest-variation’, 3 passages, 402 characters, Paragraph 17, 171 characters (in Spanish).

“I was already very interested in CL, however the workshop has broadened my perspectives… because I see that Collage facilitates the design of complex collaborative activities.”


“Collage has helped me to structure what I already do but that I have not well organized…”

Document ‘UCA-quest-final-relevance-CLFPs2, 4 passages, 259 characters, Paragraph 41, 66 characters (in Spanish).

“The patterns help me to systematize ideas that I applied before without a theoretical basis…”

Document ‘UCA-quest-final-representation2, 3 passages, 295 characters, Paragraph 35, 188 characters (in Spanish).

“… I can now systematize the activities that I usually apply in blended (semi-virtual) activities…”


“I have devised possibilities of concrete application to the type of activities I usually develop in my classes by means of active methods.”


“I have accomplished some CL experiences, but I do not consider myself an expert on this topic. That’s why I do not know if the patterns are significant. However, I have used satisfactorily some of the patterns, what leads me to think that the tool would be useful for persons with knowledge similar to mine.”


282


“I like the information help of each pattern, it facilitates a lot. Despite the fact of knowing the collaborative learning techniques, many doubts appear during the design that are easily solved thanks to this information.”


“Patterns are quite adequate and for non-experts on CL this is really an eye-opener!”

Document ‘UVA-discussion-transcription’, 5 passages, 3302 characters, Paragraph 175, 597 313 characters (in Spanish).

“… a minimum formation in patterns is necessary, for which Collage is helpful, but the user is in charge of knowing what he wants to design and how…. The tool helps the user of course, but the user is who defines…”

14 (out of 18) UCA interviewees select a) and 4 select b).


The participants are asked to value their experience and interest in applying ICT to education. They can select: a) I usually apply ICT to support my classes, b) I sometimes apply ICT to support my classes, c) I hardly apply ICT to support my classes but I am interested or d) I hardly apply ICT to support my classes and I am no interested.

1 (out of 8) UVA interviewees select a), 6 select b) and 1 selects c).

UCA interviewees value the importance of ICT in their future practice with an average of 4.50 (deviation of 0.50).


The participants are asked to value (in the range of 1 to 5) the importance that ICT will have in their future practice as teachers.

UVA interviewees value the importance of ICT in their future practice with an average of 4.00 (deviation of 0.53).

8 (out of 18) UCA interviewees select b), 9 select c) and 1 select d).

2 (out of 8) UVA interviewees select c), and 6 select d).

2 (out of 7) UNFOLD interviewees select a), and 5 select b).


The participants are asked to value their knowledge regarding IMS LD. They can select: a) I am an IMS LD expert b) I have basic IMS LD knowledge, c) I have heard something about IMS LD or d) It is the first time I see this acronym.

2 (out of 5) GSIC-EMIC interviewees select b), 2 select c) and 1 selects d).

Document ‘UVA_observations_byInes’, 2 passages, 327 characters, Paragraph 40, 146 characters (in Spanish).

“The participant with more difficulties is UVA-participant-1, who is a teacher of the Faculty of Education that has very few knowledge of Informatics…”

Document ‘UCA-quest-initial-expectations’, 11 passages, 852 characters, Paragraph 14, 118 characters (in Spanish).

“I would like to have tools that facilitate the design, implementation and evaluation of collaborative activities.”

Document ‘UCA-quest-initial-expectations’, 11 passages, 852 characters, Paragraph 32, 73 characters (in Spanish).

“I am interested in discovering tools that I could apply in my virtual courses.”

Document ‘UCA-quest-initial-experience-spec2, 12 passages, 1060 characters, Paragraph 20, 54 characters (in Spanish).

“I have just learnt superficially what IMS LD is…”

Document ‘UCA-quest-initial-experience-ICT2, 16 passages, 1995 characters, Paragraph 23, 168 characters (in Spanish).

“I use WebCT and Moodle to provide information, interact with my students using forums…”

Document ‘UVA-quest-initial-experience-ICT2, 6 passages, 1107 characters, Paragraph 14, 55 characters (in Spanish). “I use tools to show slides…”


“Collage is a big step forward for using by teacher-experts.”


“It seems useful for teachers that are already working as “e-teachers”…”

With some experience and interest in the use of ICT and also but not necessarily expert (LD) technologists (cf. also the last partial result of Table C.2 and first partial result of Table C.6).


“Make a distinction between the levels of usage within Collage. Level 1: novices, level 2: basic (as now, level 3: experts…”


283


“… Collage is an interesting tool, though I think that in order to use it a minimum training and some familiarization with the use of ICT are needed”


“I was able to create an IMS LD reflecting a previous CL experience without deep knowledge on IMS LD.”


284

APPENDIX D

SUPPORT EVALUATION DATA OF THE

NETWORK MANAGEMENT CASE STUDY

This appendix includes the support data regarding the evaluation of the Network Management case study. The data are organized in tables according to the topics and information questions forming the conceptual structure of this case. Each supporting argument has associated a coding, which identifies its data source (the raw data is available in the attached CD-ROM).


286

D.1 Meaningfulness of the CSCL script created with Collage

Table D.1 Partial results and support data concerning the information question “is the CSCL script contextualized to the actual learning situation?”



The students are asked to value the structure of the experience regarding its utility to reach the objectives directly related with the course. They can answer: the structure is negative the structure is okay, the structure is positive.

12 (out of 12) students state that “the structure of the experience is positive to reach the objectives related to the course”.

Closed question of the teacher’s questionnaire.

The teacher is asked to value the structure of the experience regarding its utility to reach the objective directly related with the course. He can answer: the structure is negative the structure is okay, the structure is positive.

The teacher states that that “the structure of the experience is positive to reach the objectives related to the course”.

Document ‘f_structure3’, 5 passages, 1641 characters. Paragraph 14, 286 characters (in Spanish).

“The structure has promoted the self-discovering of the SNMP details, what stimulates the learning.”


“I think it has been positive because the article has been broken down and we all have collaborated to extract the fundamental concepts of SNMP reflected in the text. Moreover, we have worked with a technical article, what we have never done before…”

The structure of the experience is positive to reach the objectives of the course.

Document ‘l-CC2’, 11 passages, 2755 characters. Paragraph 18, 283 characters (in Spanish).

“I think it is very positive to the definitive understanding of the main content of the different SNMP alternatives. I have acquired a good knowledge of SNMP without a great effort.”


“Since during the activity we have worked on the article once and again with the aim of reaching consensus in a group of increasing size, we deeply tackled all the details of the article. Each person provides a different vision and pays attention to diverse aspects. Besides, the different opinions are useful to clarify the aspects that have not been correctly understood.”

Document ‘focus_group_transcription_es’, 1 passages, 475 characters. Paragraph 101, 457 characters (in Spanish).

“This type of structure is merely useful to work with the document we have read (or another). But if we need to work on another type of problem then everything may be changed, and the system is no longer useful.”

The structure of the experience is adequate for this type of task.

Document ‘m_CC-3’, 9 passages, 2289 characters. Paragraph 26, 259 characters (in Spanish).

“I think it is a good practice, it has not any inconvenience when applied to this course. But it is probably not useful in other courses.”


The students are asked to grade (in the range 0-6) how difficult the task is (0 is N/A; 1, too difficult and 6, too easy).

The difficulty/ease of the experience is graded with an average of 4.33 (deviation of 0.47).


The students are asked to value (in the range 0-6) if they had sufficient time to accomplish the activities (0 is N/A; 1, I would have needed much more time and 6, I had too much time).

The students value with an average of 3.27 (deviation of 0.37) the amount of time spent to accomplish the activities.

The task is easy considering the relative duration of the experience to the rest of the course, though the time is sufficient to accomplish the proposed activities. Besides, the activities motivate the students.


“The only drawback that I find to this method is that we have spent much more time to analyze the different versions of SNMP than if the teacher explains it as in traditional classes.”

SUPPORT EVALUATION DATA OF THE NETWORK MANAGEMENT CASE STUDY

287


“We might advance more slowly than in traditional classes but I think that more ideas get fixed in our minds.”

Document ‘teacher_k_structured2’, 1 passages, 182 characters. Paragraph 5, 182 characters (in Spanish).

“The deployment might be excessive for reading an article. But it should be considered that the experience seems to have increased the motivation.”


“It motivates more than just reading the article. In this case, many of us would have either not read it, or read it too quickly, or not understood it, or not extracted so many ideas.”

Document ‘teacher_p_CC-3’, 1 passages, 237 characters. Paragraph 2-5, 237 characters (in Spanish).

“The drawback of this type of experiences is the time required to prepare all the details. The tools should provide more support.”

This type of experiences require more preparation time. Document ‘m_CC-3’, 9 passages, 2289 characters.

Paragraph 35, 599 characters (in Spanish). “The teachers need to prepare everything and the students need time to learn how to use the tools, though in our case this is not really problematic (we learn quickly…)”.

Table D.2 Partial results and support data concerning the information question “does the CSCL script guide

the learning process coordinating the students at the activity level according to the CLFPs on which is based?”



The teacher are asked to grade (in the range 0-6) to what extent the development of the experience is as designed in the script (0 is N/A; 1, it was completely different and 6, it was completely alike).

The teacher grades with 6 this question, i.e. he considers that the development of the experience reflected exactly what he intended to design in the script using Collage.

Document ‘teacher_g_desired2’, passages, characters. Paragraph , characters (in Spanish).

“The planned designed has been followed well… (except of the final TPS, but it was because I propitiated it)”

Document ‘d_structure1’, 7 passages, 3924 characters. Paragraph 8, 474 characters (in Spanish).

“… we have formed 4 groups. Each group comprised 3 members (one for each part of the article). Therefore, there are 4 experts on each part of the article).”


“The experience has been structured in 3 levels and the levels, in turn, considered different dynamic groups. Thus each person has gone along all the levels and different groups.”


After reading, the members of each group that read the same part of the article discussed what we had understood using a chat. Then, we joint our original group and explained our parts, obtaining 10 ideas and 2 questions about the article. We made comments to the results of other group, which we joined in the next session to talk about the comments and agree 8 ideas and 2 questions. Finally, we shared the ideas and questions with the other groups in a debate moderated by the teacher.”

The experience proceeds to a large extent as it is designed in the script

Document ‘summary_observations_first_session_es’, 3 passages, 422 characters. Paragraph 67, 157 characters (in Spanish).

“The teacher skipped the “Pair” phase of the TPS (because of time limitations). He proceeded directly to the discussion…”


288

Log files generated by Synergeia. The log files generated by Synergeia show that the members

assigned to a particular expert role only access to their part of the article. Similarly, the members of the successively larger groups only read (and comment) the results of the group which they are to join in the following activity.

Recorded chat conversations of the different expert groups (in Spanish). A different chat room is available to each expert group.

Students’ outcomes (in Spanish). Each group (jigsaw groups and second pyramid level groups, depending on the phase) generate a different outcome, in form of an answered questionnaire. It is possible because a different copy of the questionnaire is provided to each group (the members of the same group saw the same copy of the questionnaire).

Document ‘r_positive_negative’, 2 passages, 250 characters. Paragraph 25, 116 characters (in Spanish).

“The main positive aspect has been to always have the explanation of the steps as well as the tools needed in order to follow them.”


“We were distributed in groups and each member of the group read a part of the article.”

The script helps resource distribution.


“It was positive having access to the information in every moment (the information available in each moment) to compare ideas, clarify concepts, opine…”

Document ‘k_CC-1’, 11 passages, 2980 characters. Paragraph 23, 230 characters (in Spanish).

“The use of a script facilitates an efficient achievement of the objectives. Besides it allows the teachers to follow the progress of the students.”


“It gives you the rest of the coordination rules and how the resources are distributed. Therefore, I think it is more useful, though the freedom decreases.”


“In my (expert) group, I proposed… since I was highlighting the more important parts of the paper, I had a list of the 8 concepts I found more important. The other members agreed, so each of us summarized each concept and posted it in the chat…”


“We were basically solving doubts, discussing about the topics of our part, clarifying the concepts…”


“It is better if the coordination within activities is up to us, because each of us works differently. For example, we worked completely differently to student1’s group…“

Chat conversation of experts on topic 1 (in Spanish).

03:04:22 User: student7, “I have a doubt”

…

03:04:45 User: student7, “I don’t understand well Trap-Directed Polling”


03:03:39 User: student11, “What do you find the most important issue?”

03:04:01 User: student8, “OK, I propose a list with the most important ideas”

03:04:11 User: student8, “Then, one of us develop them and the rest make comments”


03:03:39 User: student9, “How may we start?”

03:03:56 User: student3, “I don’t know, let’s start with the chunk on SNMPv2?”

The students find effective the activity-level coordination guided by the script. They do not find it too coercive because they can collaborate freely within the activities. In fact, groups differ in their coordination within the activities.

Document ‘summary_observations_first_session_es’, 3 passages, 422 characters. Paragraph 72, 217 characters and Paragraph 74, 148 characters (in Spanish).

“The “rojo-azul” group (one of the two groups of the second pyramid level) sat in the aisle of the classroom using the chairs forming a circle. In contrast, The “verde-amarillo” group sat in front of two PC’s…”


289

Document ‘summary_observations_second_session_es’, 4 passages, 1832 characters. Paragraph 26-34, 875 characters (in Spanish).

“In general it is very different the coordination of the diverse groups…”


The students are asked to grade (in the range 0-6) the adequateness of the tools to support the activities (0 is N/A; 1, it is not adequate and 6, it is very adequate).

The students grade Synergeia with and average of 5.33, Quest with an average of 4.66 and the chat tool with an average of 4.25.

Closed questions of the teacher’s questionnaire.

The teacher is asked to grade (in the range 0-6) the adequateness of the tools to support the activities (0 is N/A; 1, it is not adequate and 6, it is very adequate).

The teacher graded Synergeia and Quest with 6 and the chat tool with 4.

Document ‘n_tools’, 5 passages, 688 characters. Paragraph 36, 122 characters (in Spanish).

“In my opinion it is essential to have available the generated documents (in the shared repository), so that they can be read at home in order to complete the study.”


“Quest’s questionnaires have been helpful to synthesize the important points…”


“The chat is quite simple… The idea of using a chat is good but a bit excessive in our case since we were face-to-face with our classmates. However, it would be a useful tool to communicate from home.”

Document ‘teacher_q_tools’, 1 passages, 143 characters. Paragraph 5, 143 characters (in Spanish).

“The chat is not very sophisticated…”

Document ‘b_mostdifficult’, 4 passages, 579 characters. Paragraph 32, 169 characters (in Spanish).

“Interacting using the chat has been the most difficult thing but we actually understood each other and organized ourselves quite good.”

The selected tools support the realization of the activities, though the use of the chat in the face-to-face activity is not perceived as effective.


“Coordination using a chat is difficult in a brief period of time.”


“It makes us work harder and commit ourselves with the work undertaken. However, if someone does not attend a session, the rest of the group suffers…”


“Unless all the experts on a topic do not attend the session, everybody may reach the understanding of everything.”

Document ‘summary_observations_second_session_es’, 5 passages, 778 characters. Paragraph 13, 116 characters (in Spanish).

“Student10 y student11 do not attend the first hour of the second session (they have another course…)”

Document ‘focus_group_transcription_es’, 4 passages, 1512 characters. Paragraph 27-29, 253 characters (in Spanish).

“It was not a problem not to attend the first hour of the second session in which the ideas of two groups were combined (second pyramid level), since we were on time to participate in the last discussion.”

Needs of flexibility emerge.

Document ‘summary_observations_first_session_es’, 3 passages, 422 characters. Paragraph 67, 157 characters (in Spanish).

“The teacher skipped the “Pair” phase of the TPS (because of time limitations). He proceeded directly to the discussion…”


290

Table D.3 Partial results and support data concerning the information question “does the CSCL script foster the desired objectives related to collaborative learning?”



The students are asked to grade (in the range 0-6) how much they had collaborated with their classmates (0 is N/A; 1, I have not collaborated and 6, I have collaborated a lot).

The students graded with an average of 5 (deviation of 0.71) how much they collaborated with their classmates.


The teacher is asked to grade (in the range 0-6) how much the students had collaborated (0 is N/A; 1, they have not collaborated and 6, they have collaborated a lot).

The teacher graded with 5 how much the students collaborated.


The students are asked to value in general (in the range 0-6) the collaboration with their classmates (0 is N/A; 1, very negative and 6, very positive).

The students valued with an average of 5.17 (deviation of 0.37) the collaboration with their classmates.


The teacher is asked to value in general (in the range 0-6) the collaboration of the students (0 is N/A; 1, very negative and 6, very positive).

The teacher valued with 5 (quite positive) the collaboration of the students.


“It fosters collaborative study and promotes the development of skills related to working in groups.”


“We have learn to work in groups, acquire knowledge working together, learn more and better, and probably in less time. It seems to me that there are many advantages…”

The experience is successful in promoting collaboration.

Log files generated by Synergeia, recorded chat conversations, and students’ outcomes.

All these sources show that all the students participated actively.


“It has been necessary an effort of synthesis and explanation of the accomplished work to the other members of the group in the way of reaching consensus.”


“We depend on the other group members’ work, so we need to trust in them…”


“The main drawback might be the different involvement of the group members if someone is not able to contribute with (s)her fair share…”


“It demands a more active participation in the activities and responsibility… because we have to explain our part to the other members…”

Document ‘focus_group_transcription_es’, 2 passages, 428 characters. Paragraph 8

5, 204 characters (in Spanish).

“It is not a problem that just one person does all the work, because each of us has (s)her task and is responsible of it…”

The script fosters positive interdependence (students need each other to succeed) and encourages individually accountability (each participant should be responsible for his/her contribution to the group work)

Document ‘summary_observations_second_session’, 2 passages, 372 characters. Paragraph 84, 215 characters (in Spanish).

“This experience requires some commitment to attend the sessions. It is possible in this course because it is optional and we are motivated.”


291


“I think the structure of the experience is quite adequate: starting from an individual study phase to fix the base ideas, and then solving doubts with the classmates involved in the same topic. The following phases devoted to sharing information within groups of increasing size in order to reach a single joint solution, have facilitated the agreement and that all the participants understand the article more easily and quickly than if we have formed a large group from the beginning.”


“Everybody participated in the discussions what improves the sharing of ideas (it helps the shy members to express themselves). A classmate explains what the others have not read, and this promotes learning. It is possible to ask doubts and everybody may answer, not only the teacher. The experience also helps to reflect in what is being doing…”


“The script has allowed the achievement of a agreed solution through discussing and raising inquietude on the different aspects of the study.”

Recorded chat conversations.

The chat-mediated discussions are much focused on the topic, intense and with a quite equally participation of the expert groups’ members that attended the first session.

All the groups discuss and reach agreement.

Students’ outcomes All the groups in the different phases formulate the maximum number of ideas (10 or 8 depending on the phase) ideas and 2 questions.

D.2 Enactment of the CSCL script using Gridcole

Table D.4 Partial results and support data concerning the information question “can the students follow successfully the CSCL script using the Gridcole system?”



The students are asked to value (in the range 0-6) if the system is useful facilitating the collaboratively realization of the activities (0 is N/A; 1, it is not useful and 6, it is very useful).

The students value with an average of 5.50 (deviation of 0.50) the utility of the system regarding the support for the collaboratively realization of the activities.


The students are asked to grade (in the range 0-6) if it is useful that the system indicated what to do and what tools should be used in each activity (0 is N/A; 1, it is not useful and 6, it is very useful).

The students grade with an average of 5.75 (deviation of 0.43) the usefulness of being indicated by the system what to do and what tools should be used in each activity.


The teacher are asked to value (in the range 0-6) if the system is useful facilitating the collaboratively realization of the activities (0 is N/A; 1, it is not useful and 6, it is very useful).

The teacher values with 5 the utility of the system regarding the support for the collaboratively realization of the activities.


The teacher is asked to grade (in the range 0-6) if it is useful that the system indicated what to do and what tools should be used in each activity (0 is N/A; 1, it is not useful and 6, it is very useful).

The teacher grades with 6 the usefulness of being indicated by the system what to do and what tools should be used in each activity.


“The structure to follow the script step by step easily and the direct access to the resources have been very useful.”

Document ‘teacher_u_positive_negative’, 1 passages, characters. Paragraph 5, 242 characters (in Spanish).

“Without the integrating system, the students would have needed to devote more attention to the tools.”

The Gridcole system is very useful supporting collaboration and indicating what to do and which tools should be used in each activity.

Document ‘o_other_tools’, 1 passages, 137 characters. Paragraph 11, 137 characters (in Spanish).

“In my opinion, the used tools satisfy the requirements of the activities because they guide us step by step in an easy way.”


292


“It was very positive that the system indicates in what phase we are and the objective of this phase as well as that it provides the access to the tools needed in each phase…”.


“The tools worked correctly and the activities have been accomplished without problems.”


“In my opinion it is essential to have available the generated documents, so that they can be read at home in order to complete the study.” The students use successfully

Gridcole in the face-to-face as well as in the distant activities. Log files generated by Synergeia. The log files generated by Synergeia show that

11 (out of 12) students read and posted comments to the results of the members of the jigsaw group that they will join in the second session. (They access Synergeia through Gridcole.)

Document ‘focus_group_transcription_es’, 2 passages, characters. Paragraph 38, 533 characters (in Spanish).

“I did not attend the first 2-hour session and, at home in half an hour I managed to find out what the rest of the classmates did in the classroom. Otherwise, I would have been lost in the other session…”

Having access to the system outside the lab provides flexibility, though more flexibility is needed. (Cf. also the partial result “Needs of flexibility emerge” of Table D.2) Document ‘teacher_u_positive_negative’, 1

passages, 75 characters. Paragraph 6, 75 characters (in Spanish).

“The system is quite rigid…”

Document ‘b_mostdifficult’, 4 passages, 579 characters. Paragraph 26, 230 characters (in Spanish).

“The most difficult aspect was learning how to use the tools… However, with the explanations of the last (familiarization) session is sufficient, and the system facilitates everything…”


“Since I did not attended the session in which how to use the system was explained, I found a bit difficult to post a comment…”

The familiarization session helps but is not crucial.


“The students need time to learn how to use the tools, though in our case this is not really problematic (we learn quickly…)”.

Table D.5 Partial results and support data concerning the information question “how can the enactment of the

CSCL script be improved?”



“The frames and the menu on the left are not very intuitive…”


“It was negative the fact that the frames cannot be resized…”

The frames of the web player are not intuitive enough.

Document ‘summary_observations_first_session’, 2 passages, 346 characters. Paragraph 26, 96 characters (in Spanish).

“The teacher explains how to open a frame in a different window…”

Document ‘focus_group_transcription_es, 1 passages, 97 characters. Paragraph 108, 97 characters (in Spanish).

“Changing groups was not intuitive in the system…”

Document ‘summary_observations_first_session’, 3 passages, 479 characters. Paragraph 57, 184 characters (in Spanish).

“The teacher reminds again that the students should change the role…”

Document ‘summary_observations_second_session’, 1 passages, 95 characters. Paragraph 88, 95 characters (in Spanish).

“It was necessary to remind the students the structure of roles/groups and the flow of activities.”

The system could be improved with awareness utilities.


“I miss an activity index indicating which role should be selected in each moment, instead of distributing it a document…”


293


“I do not understand the issue of the groups in the system. It would be easier if the system obviates the change of roles (and makes it automatically for you…)…”


“A negative aspect is the fact of having different passwords to accessing the different tools. I think it would be useful to unifying them… if we access to the system, then the direct access to the tools should be without password…”

The system could be improved with authentication utilities. Document ‘r_positive_negative’, 3 passages, 396

characters. Paragraph 31, 333 characters (in Spanish). “I had the problem that I did have the rights to access Synergeia… These types of technical reverses obstruct the work.”

Document ‘summary_observations_first_session_es’, 3 passages, 613 characters. Paragraph 14, 214 characters and Paragraph 24, 225 characters (in Spanish).

“Student5 does not attend this session. She has written an e-mail informing that she could not attend because of health problems and asking how she had to do in order not to be lost in the next session. Therefore, she did not follow the second and third phase of the Jigsaw but will need to read the whole article at home.

Student2 joint the session late (almost half an hour). (He is in the same expert group and the same second pyramid level group than student5. If he would not attend the whole first session, a solution like changing the second pyramid level groups would have been needed.)”

Document ‘summary_observations_second_session_es’, 5 passages, 778 characters. Paragraph 13, 116 characters (in Spanish).

“Student10 y student11 do not attend the first hour of the second session (they have another course…)”

Need of supporting further flexibility (cf. also the last partial result of Table D.4).

Document ‘focus_group_transcription_es’, 4 passages, 1512 characters. Paragraph 27-29, 253 characters (in Spanish).

“It was not a problem not to attend the first hour of the second session in which the ideas of two groups were combined (second pyramid level), since we were on time to participate in the last discussion.”

D.3 Educational innovation with respect to previous students’ experiences

Table D.6 Partial results and support data concerning the information question “does the enactment of the CSCL script enhance students’ previous experience in terms of structuring collaboration and use of

supporting technology?”



The students are asked how different is this experience compared to previous ones. They can answer: no differences, few differences, quite a lot of differences or a lot of differences.

9 (out of 12) students state that “they have found quite a lot of differences”.

3 (out of 12) students indicate that “they have found a lot of differences”.


“I have never read collaboratively an article as well as I have never followed this type of pyramid-structured workflow.”


“I have felt more guided because I knew what to do in each moment. In other experiences of other courses I did not have clear what to do in each moment.”

The scenario introduces many differences with respect to previous students’ experiences.


“In other courses we all need to know everything from the beginning. In contrast, in this experience we specialized at the beginning.”


294


“It is more amusing and the methodology is quite innovative, what increases the interest.”

Document ‘initial_questionnaire_a_collaboration’, 11 passages, 1180 characters. Paragraph 45, 122 characters (in Spanish).

“Actually we have not practiced collaborative learning during the university studies, except for the CA course, in which collaboration was very useful.”

Document ‘initial_questionnaire_a_collaboration’, 6 passages, 762 characters. Paragraph 6, 44 characters (in Spanish).

“I slightly prefer working individually, because in groups we have always divided the tasks in order to finish earlier or one of the members has worked more than the rest…”

Document ‘initial_questionnaire_b_structure’, 8 passages, 1094 characters. Paragraph 21, 179 characters (in Spanish).

“It is very convenient to assign the roles initially in order to make the collaboration more effective, instead that each of us “wages the war” by himself. We should be a team…”

Document ‘initial_questionnaire_b_structure’, 8 passages, 1094 characters. Paragraph 36, 136 characters (in Spanish).

“With some indications of how to work, we are more oriented to the interesting point. It may avoid that the students get lost with so many concepts…”


“This work dynamic is completely innovative. Despite in other courses working in groups is fostered to certain extent; to me the innovation of this experience is the way of working (following different phases, changing groups…)”


“For example ‘Name of the course’ also promotes collaborative activities. However, the activities are a disaster because they are not well organized. In this experience, there is a structure, a plan and collaboration is really fostered. In contrast, in ‘Name of the course’ there is not any indication and nobody works, what does nod lead to learning…”


“Working in group of increasing size has been very appropriate. In other courses we usually divided the work in parts but without contrasting information with others…”

Document ‘initial_questionnaire_c_guidance’, 8 passages, 1084 characters. Paragraph 33, 216 characters (in Spanish).

“If we guide ourselves, our creativity, maturity… may increase.”

The script provides a collaboration strategy and does not encourage totally free collaboration or mere cooperation, which does not promotes effective results. However, (partially) open scripts (self-scripts) might foster creativity and inquiry processes.

Document ‘initial_questionnaire_c_guidance’, 8 passages, 1084 characters. Paragraph 45, 127 characters (in Spanish).

“The tasks that indicates us what to do with details in order to solve them, we just do what is indicated, while with a more open script we may research additionally on our own”


“With this type of methods we learn more, work harder and better, and we take more advantage of the time. Besides “social skills” are promoted, what is so much demanded in the companies. I find the method quite useful and it should be applied in more courses. It is better that the typical classes in which we sometimes do not even know what the teachers are talking about. With this method, we are much more involved and this is positive for the acquisition of knowledge and what is learned…”


“The inconvenience is that there is not a culture of collaborating and the people are individualist… But in general I think it is very positive. I think that in our professional future we will work collaboratively. Therefore, this has been a good training experience.”

The structure of the experience promotes active learning and educational benefits useful for the professional future of the students that are not truly considered in the educational system.


“I think there should be more collaborative learning activities with large groups, since after all it is type of structure (used when working in projects) in which we will be involved in our professional future.”


295


“A difference with other courses is the use of an integrated intuitive software system.”


“It is worth mentioning that another difference is the use of telematic tools for collaborative learning (although we have used them in another courses taught by members of your group…)”

The students appreciate the use of Gridcole. They had used the employed type of tools before but not an integrating system. Document ‘m_CC-3’, 1 passages, 230

characters. Paragraph 23, 404 characters (in Spanish).

“… Sometimes the lack of efficiency makes us to waste time and hate working in groups. The realization of guided (or partly guided) activities and/or the use of tool like Gridcole, increase the efficiency and the interest in these activities.”


296

REFERENCES Adrion, W.R. (1993). Research methodology in software engineering: summary of the Dagstuhl workshop on future directions in software engineering. In Proceedings of SIGSoft Software Eng. Notes New York: ACM Press, 18(1).

Al-Holou, N., Booth, K.K., & Yaprak, E. (2000). Using computer network simulation tools as supplements to computer network curriculum. In Proceedings of 30th IEEE Frontiers in Education Conference (pp. S2C-13-16) Kansas City, Missouri: IEEE Computer Society.

Alexander, C. (1979). The timeless way of building. New York: Oxford University Press.

Alexander, C. (1999). The origins of pattern theory: The future of the theory and the generation of a living world. IEEE Software, 16(5), 71-82.

Alexander, C. (2003a). The phenomenon of life: the nature of order, book 1. Berkeley, CA: Center for Environmental Structure.

Alexander, C. (2003b). The process of creating life: the nature of order, book 2. Berkeley, CA: Center for Environmental Structure.

Alexander, C. (2004a). A vision of a living world: the nature of order, book 3. Berkeley, CA: Center for Environmental Structure.

Alexander, C. (2004b). The luminous ground: the nature of wonder, book 4. Berkeley, CA: Center for Environmental Structure.

Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., & Angel, S. (1977). A Pattern Language: Towns, Buildings, Construction. New York: Oxford University Press.

Amorim, R., Lama, M., & Sánchez, E. (2006a). Modelling and implementation of the astronomy case study with an IMS LD ontology. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1166-1167) Kerkrade, the Netherlands: IEEE Computer Society.

Amorim, R., Lama, M., & Sánchez, E. (2006b). Using ontologies to model and execute IMS Learning Design documents. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1115-1116) Kerkrade, the Netherlands: IEEE Computer Society.

Anderson, G., Boud, D., & Sampson, J. (1996). Learning contracts. A practical guide. London: Kogan Page.

Arlow, J., & Neustadt, I. (2001). UML and the Unified Process: Practical Object-Oriented Analysis and Design. Boston, MA: Addison Wesley Professional.

Aronson, E., & Patnoe, S. (1997). The jigsaw classroom: building cooperation in the classroom. (2 ed.). United States: Addison-Wesley Educational Publishers Inc.

Aronson, E., & Thibodeau, R. (1992). The Jigsaw classroom: A cooperative strategy for an educational psychology course. In J. Lynch, C. Modgil, & S. Modgil (Eds.), Cultural diversity and the schools (pp. 231-256). Washington: Palmer.

AUTC (2003). Learning designs, products of the AUTC project on ICT-based learning designs. Retrieved April 2007 from http://www.learningdesigns.uow.edu.au

Avery, K.B., Avery, C.W., & Pace, D.P. (1998). Bridging the gap: integrating video and audio cassettes into literature programs. The English Journal, 87(2), 58-62.

298

Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management systems. Educational Technology & Society, 6(2), 11-24.

Baggetun, R., Rusman, E., & Poggi, C. (2004). Design patterns for collaborative learning: From practice to theory and back. In L. Cantoni & C. McLoughlin (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 2493-2498) Lugano, Switzerland.

Bennett, S., & Lockyer, L. (2004). Becoming an online teacher: adapting to a changed environment for teaching and learning in higher education. Educational Media International, 41(3), 231-244.

Berger, A., Moretti, R., Chastonay, P., Dillenbourg, P., Bchir, A., Baddoura, R., Bengondo, C., Scherly, D., Ndubre, P., Farah, P., & Kayser, B. (2001). Teaching community health by exploiting international socio-cultural and economical differences. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), Proceedings of first European Conference on Computer-Supported Collaborative Learning Maastricht, the Netherlands.

Berlanga, A., & García, F. (2005). IMS LD reusable elements for adaptive learning designs. Journal of Interactive Media in Education, (7) Available at http://jime.open.ac.uk/2005/11/

Biggs, J. (1999). Teaching for quality learning at university: what the students does. Buckingham: Open University Press.

Bloom, B.S., & Krathwohl, D.R. (1984). Taxonomy of educational objectives, handbook I: Cognitive domain. New York: Addison-Wesley.

Bonk, C.J., Graham, C.R., Cross, J., & Moore, M.G. (2006). The handbook of blended learning: global perspectives, local designs. Indianapolis: John Wiley & Sons.

Bote-Lorenzo, M.L. (2005). Gridcole, a grid services-based tailorable system for the support of scripted collaborative learning (in Spanish). Doctoral Thesis, University of Valladolid, Valladolid, Spain.

Bote-Lorenzo, M.L., Hernández-Leo, D., Dimitriadis, Y., Asensio-Pérez, J.I., Gómez-Sánchez, E., Vega-Gorgojo, G., & Vaquero-González, L.M. (2004). Towards reusability and tailorability in collaborative learning systems using IMS-LD and Grid Services. Advanced Technology for Learning, 1(3), 129-138.

Botturi, L. (2006). E2ML: a visual language for the design of instruction. Educational Technology Research and Development, 54(3), 265-293.

Botturi, L., Derntl, M., Boot, E., & Figl, K. (2006). A classification framework for educational modeling languages in instructional design. In Kinshuk, R. Koper, P. Kommers, P. Kirschner, D. G. Sampson, & W. Didderen (Eds.), Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1216-1220) Kerkrade, The Netherlands: IEEE Computer Society.

Botturi, L., & Stubbs, T. (in press). Handbook of visual languages for instructional design: theories and practices. Hershey, PA: Idea Group Publishing.

Boyd, J.A.Jr. (1996). Floor control in synchronous groupware. Doctoral Thesis, The Ohio State University, Ohio, USA.

Brecht, J., DiGiano, C., Patton, C., Davis, K., Raj Chaudhury, K., Roschelle, J., & Tatar, D. (2006). Coordinating networked learning activities with a general purpose interface. In Proceedings of the Mlearn Conference Banff, Canada.

Brouns, F., Koper, R., Manderveld, J., van Bruggen, J., Sloep, P., van Rosmalen, P., Tattersall, C., & Vogten, H. (2005). A first exploration of an inductive analysis approach for detecting learning design patterns. Journal of Interactive Media in Education, (3) Available at http://www-jime.open.ac.uk/2005/03/

Burgos, D., & Griffiths, D. (Eds.) The UNFOLD project. Understanding and using Learning Design. Heerlen, The Netherlands: Open University of The Netherlands, 2005.

REFERENCES

299

Burgos, D., Specht, M., Kravcik, M., Dolog, P., Abbing, J., Boticario, J., Naeve, A., Zarraonandia, T., and Turker, M. A. (2006a). Presentations in ADALE (Adaptive Learning and Learning Design) workshop at Adaptive Hypermedia Conference. Retrieved April 2007a from http://dspace.ou.nl/handle/1820/668

Burgos, D., Specht, M., Kravcik, M., Dolog, P., Gasevic, D., Cristea, A., Dodero, J. M., and Naeve, A. (2006b). Presentations in ADALE (Adaptive Learning and Learning Design) workshop at International Conference on Advanced Learning Technologies. Retrieved April 2007b from http://dspace.ou.nl/handle/1820/669

Burgos, D., Tattersall, C., Dougiamas, M., Vogten, H., & Koper, R. (2006). Mapping IMS Learning Design and Moodle. A first understanding. In A. Panizo, L. Sánchez-González, B. Fernández-Manjón, & M. Llamas-Nistal (Eds.), Proceedings of 8th International Symposium on Computers in Education (pp. 79-85) León, Spain: University of León.

Burgos, D., Tattersall, C., & Koper, R. (2006). Representing adaptive eLearning strategies in IMS Learning Design. In R. Koper & K. Stefanov (Eds.), Proceedings of International Workshop on Learning Networks for Lifelong Competence Development (pp. 54-60) Sofia, Bulgaria.

Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., & Stal, M. (1996). Pattern-Oriented Software Architecture: a system of patterns. New York: John Wiley & Sons.

Buzza, D., Bean, D., Harrigan, K., & Carey. T. (2004). Learning design repositories: adapting learning design specifications for shared instructional knowlege. Canadian Journal of Learning and Technology, 30(3), 79-101.

Buzza, D., Richards, L., Bean, D., Harrigan, K., & Carey, T. (2005). LearningMapR: a prototype tool for creating IMS-LD compliant units of learning. Journal of Interactive Media in Education, (17) Available at http://jime.open.ac.uk/2005/17/

Caeiro-Rodríguez, M., Anido-Rifón, L., & Llamas-Nistal, M. (2003). A critical analysis of IMS Learning Design. In B. Wasson, L. Anido, & U. Hoppe (Eds.), Proceedings of International Conference on Computer Support for Collaborative Learning (pp. 363-367) Bergen: Kluwer Academic Publishers.

Caeiro-Rodríguez, M., Llamas-Nistal, M., & Anido-Rifon, L. (2006a). The PoEML proposal to model services in educational modelling languages. In Proceedings of 12th International Workshop on Groupware, LNCS 4154 (pp. 187-202) Medina del Campo, Spain: Springer.

Caeiro-Rodríguez, M., Llamas-Nistal, M., & Anido-Rifon, L. (2006b). A separation of concerns approach to educational modelling languages. In Proceedings of Frontiers in Education 36th Annual Conference San Diego, California: IEEE Computer Society.

Casey, J., Brosnan, K., Greller, W., Masson, A., MacNeill, A., & Murphy, C. (in press). Designing for change: visual design tools to support process change in education. In L. Boturri & T. Stubbs (Eds.), Handbook of Visual Languages for Instructional Design: Theories and Practices Hershey, PA: Idea Group.

Clarke, J. (1994). Pieces of the puzzle: the jigsaw method. In S. Sharan (Ed.), Handbook of cooperative learning methods (pp. 34-50). Westport CT: Greenwood Press.

Conole, G., & Fill, K. (2005). A learning design toolkit to create pedagogically effective learning activities. Journal of Interactive Media in Education, (8) Available at http://jime.open.ac.uk/2005/08/

Coplien, J.O., & Harrison, N.B. (2005). Organizational patterns of agile software development. USA: Pearson Prentice Hall.

CoSSICLE (2005). Computer-Supported Scripting of Interaction in Collaborative Learning Environments Kaleidoscope-NoE ERT website. Retrieved April 2007 from http://www.iwm-kmrc.de/cossicle/

300

CRAFT (2007). Manyscripts Project. Retrieved April 2007 from http://craftwww.epfl.ch/research/manyscripts/

Cronholm, S., & Melin, U. (2006). Project oriented student work: group formation and learning. In Proceedings of 23rd Information System Education Conference Dallas, Texas.

Dalziel, J. (2003). Discussion paper for learning activities and meta-data, Technical Report, Macquarie E-learning Centre of Excellence.

Dalziel, J.R. (2006a). Lessons from LAMS for IMS Learning Design. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1101-1102) Kerkrade, the Netherlands: IEEE Computer Society.

Dalziel, J.R. (2006b). Modeling a team-based astronomy task using LAMS. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1152-1153) Kerkrade, the Netherlands: IEEE Computer Society.

David, B., Delotte, O., Chalon, R., Tarpin-Bernard, F., & Saikali, K. (2003). Patterns in collaborative system design, development and use. In Proceedings of Interact 2003, 2nd workshop on Software and Usability, Cross-polination the role of usability patterns Zürich, Switzerland.

David, J.P., Lejeune, A., & Villiot-Leclercq, E. (2006). Expressing workshop scenario with Computer Independent Model. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1168-1169) Kerkrade, the Netherlands: IEEE Computer Society.

Davie, L. (1989). Facilitation techniques for the on-line tutor. In R. Mason & A. Kaye (Eds.), Mindweave: Communications, Computers and Distance Education (pp. 74-85). Oxford: Pergamon Press.

Davis, W.A. (2002). A comparison of pyramids versus brainstorming in a problem based learning environment. In focusing on the student. In Proceedings of 11th Annual Teaching Learning Forum Perth: Edith Cowan University.

de la Teja, I., Lundgren-Cayro, K., & Paquette, G. (2005). Transposing MISA Learning Scenarios into IMS Units of Learning. Journal of Interactive Media in Education, (13) Available at http;//www-jime.open.ac.uk/2005/13/

Denzin, N.K., & Lincoln, Y.S. (2005). Handbook of Qualitative Research. (3 ed.). Thousand Oaks, CA: Sage Publications.

Derntl, M., & Botturi, L. (2006). Essential Use Cases for Pedagogical Patterns. Computer Science Education, special issue on Pedagogic Patterns, 16(2), 137-156.

Derntl, M., & Motschning-Pitrik, R. (2004). Patterns for blended, person-centered learning: strategy, concepts, experiences, and evaluation. In H. Haddad, A. Omicini, R. L. Wainwright, & L. M. Liebrock (Eds.), Proceedings of 2004 ACM Symposium on Applied Computing (pp. 916-923) Nicosia, Cyprus: ACM.

Devedzic, V., & Harrer, A. (2005). Software patterns in ITS architectures. International journal of artificial intelligence in education, 15(2), 63-94.

DiGiano, C., Yarnall, L., Patton, C., Roschelle, J., Tatar, D., & Manley, M. (2003). Conceptual tools for planning the wireless classroom. Journal of Computer Assisted Learning, 19, 284-297.

Dillenbourg, P. (1999a). Collaborative learning: cognitive and computational approaches. Oxford, UK: Elsevier Science.

Dillenbourg, P. (1999b). What do you mean by "Collaborative Learning"? In P. Dillenbourg (Ed.), Collaborative Learning. Cognitive and Computational Approaches (pp. 1-19). Oxford, UK: Elsevier Science.

REFERENCES

301

Dillenbourg, P. (2002). Over-Scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Inaugural Address, Three Worlds of CSCL. Can We Support CSCL? (pp. 61-91). Heerlen: Open Universiteit Nederland.

Dillenbourg, P., & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl, & J. Haake (Eds.), Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives (pp. 277-302). New York: Springer.

Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for CSCL. Journal of Computer Assisted Learning, 23(1), 1-13.

Dimitracopoulou, A., & Petrou, A. (in press). Advanced collaborative distance learning systems for young students: Design issues and current trends on new cognitive and metacognitive tools. THEMES in Education, International Journal.

Dimitriadis, Y., Asensio-Pérez, J.I., Martínez-Monés, A., & Osuna-Gómez, C.A. (2003). Component based software engineering and CSCL: Component identification and dimensioning. Upgrade (digital journal of European Professional Informatics Societies), special issue on e-learning: Boarderless education, 4(5), 21-28.

Dimitriadis, Y., Asensio-Pérez, J.I., Hernández-Leo, D., Roschelle, J., Brecht, J., Tatar, D., Chaudhury, S.R., DiGiano, C., & Patton, C.M. (2007). From social-mediated to technology-mediated coordination: a study of design tensions using Group Scribbles. In Proceedings of Computer Supported Collaborative Learning 2007 (in press) New Jersey, USA.

Dodero, J.M., Tattersall, C., Burgos, D., & Koper, R. (2007). Transformational techniques for model-driven authoring of learning designs. In Proceedings of 6th International Conference on Web-based Learning (in press) Edinburgh, United Kingdom.

Dufresne, A. (2006a). Explor@Graph the representation power of adaptive conceptual graphs. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1111-1112) Kerkrade, the Netherlands: IEEE Computer Society.

Dufresne, A. (2006b). Explora@Graph scenarios editor - designing a collaborative task. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1162-1163) Kerkrade, the Netherlands: IEEE Computer Society.

Duval, E. (2001). Metadata standards: what, who & why. Journal of Universal Computer Science, 7(7), 591-601.

E-LANE (2004). E-LANE project website. Retrieved April 2007 from http://e-lane.org/

E-LEN (2004a). Design patterns and how to produce them. Retrieved April 2007a from http://www2.tisip.no/E-LEN/documents/ELEN-Deliverables/booklet-e-len_design_experience.pdf

E-LEN (2004b). E-LEN Project Website. Retrieved April 2007b from http://www2.tisip.no/E-LEN/

E-LEN (2005). The production of e-learning design patterns and a research road map for e-learning. Retrieved April 2007 from http://www2.tisip.no/E-LEN/documents/ELEN-Deliverables/Report_WP3_ELEN-Roadmap.pdf

EC (2006). The Bologna process: make higher education systems in Europe converge, http://europa.eu/scadplus/leg/en/cha/c11088.htm. Retrieved April 2007

El-Kharashi, M.W., Darling, G., Marykuca, B., & Shoja, G.C. (2002). Understanding and implementing computer network protocols through a lab project. IEEE Transactions on Education, 45(3), 276-284.

302

Ellis, C.A., Gibbs, G.L., & Rein, G.L. (1991). Groupware: Some Issues and Experiences. Communications of the ACM, 43(1), 39-58.

Ellis, C.A., & Wainer, J. (1994). A conceptual model of groupware. In R. Futura & C. Neuwirth (Eds.), Proceedings of 5th Conference on Computer Supported Cooperative Work (pp. 79-88) New York: ACM.

Fablusi, P. L. (2000). Role play simulation for teaching and learning. Retrieved April 2007 from http://www.roleplaysim.org

Fischer, F., Kollar, I., Haake, J., & Mandl, H. (2007). Perspectives on collaboration scripts. In F. Fischer, I. Kollar, H. Mandl, & J. Haake (Eds.), Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives (pp. 13-22). New York: Springer.

Fischer, F., Kollar, I., Mandl, H., & Haake, J. (2007). Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives. New York: Springer.

Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1995). Design Patterns, Elements of Reusable Object-Oriented Software. USA: Addison-Wesley.

Georgiakakis, P., & Retalis, S. (2006). Demystifying the asynchronous network supported collaborative learning systems. International Journal of Computer Applications in Technology, Special Issue on Patterns for Collaborative Systems, 25(2/3)

Giacomini-Pacurar, E., Trigano, P., & Alupoaie, S. (2006). Knowledge base for automatic generation of online IMS LD compliant course structures. Educational Technology & Society, 9(1), 158-175.

Gibbs, G. (1995). Teaching more students 3: discussion with more students. Headington, Oxford: The Oxford Centre for Staff Development.

Glass, R.L. (1995). A structure-based critique of contemporary computing research. Systems and Software, 28, 3-7.

Glass, R.L., Vessey, I., & Ramesh, V. (2002). Research in software engineering: an analysis of the literature. Information and Software Technology, 44, 491-506.

Goodyear, P. (2005). Educational design and networked learning: Patterns, pattern languages and design practice. Australasian Journal of Educational Technology, 21(1), 82-101.

Goodyear, P., Avgeriou, P., Baggetun, R., Bartoluzzi, S., Retalis, S., Ronteltap, F., & Rusman, E. (2004). Towards a pattern language for networked learning. In S. Banks, P. Goodyear, V. Hodgson, C. Jones, V. Lally, D. McConnell, & C. Steeples (Eds.), Proceedings of Networked learning 2004 (pp. 449-455) Lancaster: Lancaster University.

Gorissen, P., & Tattersall, C. (2005). A Learning Design worked example. In R. Koper & C. Tattersall (Eds.), Learning Design: a handbook on modelling and delivering networked education and training (pp. 3-20). Berlin-Heidelberg: Springer Verlag.

Goubil-Gambrel, P. (1992). A practitioner's guide to research methods. Technical Communications, 39(4), 582-591.

Griffiths, D. (2005). From primitives to patterns: a discussion paper, UNFOLD project. Retrieved April 2007 from http://www.unfold-project.net/providers_folder/papers/terms/primitives-patterns.pdf

Griffiths, D., & Blat, J. (2005). The role of teachers in editing and authoring Units of Learning using IMS Learning Design. Advanced Technology for Learning, Special issue on Designing Learning Activities: From Content-based to Context-based Learning Services, 2(4), 243-251.

REFERENCES

303

Griffiths, D., Blat, J., García, R., Vogten, H., & Kwong, K.L. (2005). Learning Design tools. In R. Koper & C. Tattersall (Eds.), Learning Design, a handbook on modelling and delivering networked education and training (pp. 109-135). Heidelberg: Springer-Verlag.

Grudin, J. (1992). Utility and usability: research issues and development context. Interacting with Computers, 4(2), 209-217.

Grundin, J. (1992). CSCW: history and focus. Computer, 29(6), 27-35.

GSIC/EMIC (1994). Intelligent & Cooperative Systems Research Group/Education, Media, Informatics and Culture research group website. Retrieved April 2007 from http://gsic.tel.uva.es/

Guba, E.G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. Educational Communication and Technology: A Journal of Theory, Research, and Development, 29(2), 75-91.

Guba, E.G. (1985). Trustworthiness criteria in naturalistic research (in Spanish). In J. Gimeno & A. Pérez (Eds.), Education, Theory and Practice (pp. 148-165). Madrid: Akal.

Gutwin, C., & Greenberg, S. (1999). The effects of workspace awareness support on the usability of real-time distributed groupware. ACM Transactions on Computer-Human Interaction, 6(3), 243-281.

Guy, E.S. (2003). Patterns as artefacts in user-developer collaborative design. In Proceedings of Workshop Applying Activity Theory to CSCW Research and Practice, 8th European Computer Supported Cooperative Work Helsinki, Finland.

Gómez, E., Rubia, B., Dimitriadis, Y., & Martínez, A. (2002). Quest, A Telematic Tool for Automatic Management of Student Questionnaires in Educational Research. In Proceedings of 2nd European Conference on Technology, Information, Education Citizenship, Barcelona, Spain.

Haake, J., & Pfister, H.R. (2007). Flexible scripting in net-based learning groups. In F. Fischer, I. Kollar, H. Mandl, & J. Haake (Eds.), Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives (pp. 161-181). New York: Springer.

Häkkinen, P. (2002). Challenges for design of computer-based learning environments. British Journal of Educational Technology, 33(4), 461-469.

Harrer, A. (2006). An approach to organize re-usability of learning designs and collaboration scripts of various granularities. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies Kerkrade, the Netherlands: IEEE Computer Society.

Harrer, A., Hernández-Leo, D., & Dimitriadis, Y. (2005). Languages for Modelling of Collaborative Learning Process - Formalization, Practical Uses and Tools. In Proceedings of Computer Supported Collaborative Learning 2005, Workshop proposals Taipei, Taiwan.

Harrer, A., & Malzah, N. (2006). Bridging the gap, towards a graphical modelling language for learning designs and collaboration scripts of various granularities. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 296-300) Kerkrade, the Netherlands: IEEE Computer Society.

Hedberg, J., Oliver, R., Harper, B., Wills, S., & Agostinho, S. (2002). Implementing generic learning designs based upon quality ICT exemplars. In Proceedings of International Conference on Computers in Education (pp. 1011-1016) Auckland, New Zealand: IEEE Computer Society.

Helic, D. (2006). Technology-supported management of collaborative learningprocesses. International Journal of Learning and Change, 1(3), 285-295.

304

Hermans, H., Manderveld, J.M., & Vogten, H. (2004). Educational modelling language. In W. Jochems, J. van Merriënboer, & R. Koper (Eds.), Integrated E-learning (pp. 80-99). London: RoutledgeFalmer.

Hernández-Leo, D., Burgos, D., Tattersall, C., & Koper, R. (submitted). Representing Computer-Supported Collaborative Learning macro-scripts using IMS LD. Draft available at http://dspace.ou.nl/handle/1820/784.

Hernández-Leo, D., Asensio-Pérez, J.I., & Dimitriadis, Y. (2004). IMS Learning Design support for the formalization of Collaborative Learning Patterns. In Kinshuk, C. Looi, E. Sutinen, D. Sampson, I. Aedo, L. Uden, & E. Kähkönen (Eds.), Proceedings of 4th IEEE International Conference on Advanced Learning Technologies (pp. 350-354) Joensuu, Finland: IEEE Computer Society.

Hernández-Leo, D., Asensio-Pérez, J.I., & Dimitriadis, Y. (2005). Computational representation of Collaborative Learning Flow Patterns using IMS Learning Design. Educational Technology & Society, 8(3), 75-89.

Hernández-Leo, D., Asensio-Pérez, J.I., & Dimitriadis, Y. (2006). Collaborative learning strategies and scenario-based activities for understanding network protocols. In S. Lord & D. T. Hayhurst (Eds.), Proceedings of Frontiers in Education 36th Annual Conference (pp. S2F-19-24) San Diego, California: IEEE Computer Society.

Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y., Bote-Lorenzo, M.L., Jorrín-Abellán, I.M., & Villasclaras-Fernández, E.D. (2005a). Reusing IMS-LD formalized best practices in collaborative learning structuring. Advanced Technology for Learning, 2(4), 223-232.

Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y., Bote-Lorenzo, M.L., Jorrín-Abellán, I.M., & Villasclaras-Fernández, E.D. (2005b). Describing effective collaborative learning flows using IMS Learning Design. In Proceedings of 4th IASTED International Conference on Web-Based Education (pp. 273-278) Grindelwald, Switzerland: ACTA Press.

Hernández-Leo, D., Bote-Lorenzo, M.L., Asensio-Pérez, J.I., Gómez-Sánchez, E., Villasclaras-Fernández, E.D., Jorrín-Abellán, I.M., & Dimitriadis, Y. (in press). Free- and open source software for a course on network management: authoring and enactment of scripts based on collaborative learning strategies. IEEE Transactions on Education.

Hernández-Leo, D., Harrer, A., Dodero, J.M., Asensio-Pérez, J.I., & Burgos, D. (2006a). Creating by reusing learning design solutions. In Proceedings of the 8th International Symposium on Computers in Education (pp. 417-424) León, Spain.

Hernández-Leo, D., Villasclaras-Fernández, E.D., Asensio-Pérez, J.I., & Dimitriadis, Y. (in pressa). Diagrams of learning flow patterns' solutions as visual representations of refinable IMS Learning Design templates. In L. Boturri & T. Stubbs (Eds.), Handbook of Visual Languages for Instructional Design: Theories and Practices Hershey, PA: Idea Group.

Hernández-Leo, D., Villasclaras-Fernández, E.D., Asensio-Pérez, J.I., Dimitriadis, Y., Bote-Lorenzo, M.L., & Marcos-García, J.A. (2006b). Tuning IMS LD for implementing a collaborative lifelong learning scenario. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1160-1161) Kerkrade, the Netherlands: IEEE Computer Society.

Hernández-Leo, D., Villasclaras-Fernández, E.D., Asensio-Pérez, J.I., Dimitriadis, Y., Bote-Lorenzo, M.L., & Rubia-Avi, B. (2006c). Linking collaborative learning practice with IMS LD and service-oriented technologies: an approach based on collaborative learning flow patterns. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1109-1110) Kerkrade, the Netherlands: IEEE Computer Society.

Hernández-Leo, D., Villasclaras-Fernández, E.D., Asensio-Pérez, J.I., Dimitriadis, Y., & Retalis, S. (2006d). CSCL scripting patterns: hierarchical relationships and applicability. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 388-392) Kerkrade, the Netherlands: IEEE Computer Society.

REFERENCES

305

Hernández-Leo, D., Villasclaras-Fernández, E.D., Jorrín-Abellán, I.M., Asensio-Pérez, J.I., Dimitriadis, Y., Ruiz-Requies, I., & Rubia-Avi, B. (2006e). COLLAGE, a collaborative learning design editor based on patterns. Educational Technology and Society, 9(1), 58-71.

Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research & Development, 48(3), 23-48.

Hodgings, W. (2000). IEEE LTSC Learning Objects Metadata. Retrieved April 2007 from http://ieeeltsc.org/

Holkeboer, K. (1987). Patterns for theatrical costumes: garments, trims, and accessories from ancient Egypt to 1915. New York: Prentice-Hall.

Holland, S., Griffiths, R., & Woodman, M. (1997). Avoiding object misconceptions. In Proceedings of 28th SIGCSE Technical Sysmposium on Computer Science Education (pp. 131-134) New York, USA: ACM Press.

Häkkinen, P., & Mäkitalo-Siegl, K. (2007). Discussion: educational perspectives on scripting CSCL. In F. Fischer, I. Kollar , H. Mandl, & J. Haake (Eds.), Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives (pp. 267-303). New York: Springer.

IMS (2001). IMS Global Learning Consortium, Inc. Retrieved April 2007 from www.imsglobal.org

IMS (2002). IMS Resource Meta-data specification. Retrieved April 2007 from http://www.imsglobal.org/metadata/

IMS (2003a). IMS Learning Design Best Practice and Implementation Guide. Retrieved April 2007a from http://www.imsglobal.org/learningdesign/ldv1p0/imsld_bestv1p0.html

IMS (2003b). IMS Learning Design specification. Retrieved April 2007b from http://www.imsglobal.org/learningdesign/

IMS (2004a). IMS Content Packaging specification. Retrieved April 2007a from http://www.imsglobal.org/content/packaging/

IMS (2004b). IMS Enterprise Services Specification. Retrieved April 2007b from http://www.imsglobal.org/es/

IMS (2006). IMS Question & Test Interoperability specification. Retrieved April 2007 from http://www.imsglobal.org/question/

ISI (2007). The network simulator ns-2, http://nsnam.isi.edu/nsnam/. Retrieved April 2007

ISO/IET (2002). JTC1 SC36/WG2: Collaborative Technology. Retrieved 12 July 2006 from http://collab-tech.jtc1sc36.org/

ITCOLE (2005). Synergeia Website. Retrieved May 2005 from http://bscl.gmd.de

Jermann, P., & Dillenbourg, P. (2002). Elaborating new arguments through a CSCL script. In J. Andriessen, M. Baker, & D. Suthers (Eds.), Arguing to Learn: Confronting Cognitions in Computer-Supported Collaborative Learning environments (pp. 1-6). The Netherlands: Kluwer Academic Publishers.

Jermann, P., Soller, A., & Lesgold, A. (2004). Computer software support for CSCL. In J. W. Strijbos, P. A. Kirschner, & R. L. Martens (Eds.), What we know about CSCL and implementing it in higher education (pp. 141-166). Norwell, MA, USA: Kluwer Academic Publishers.

Jochems, W., van Merriënboer, J., & Koper R. (2004). Integrated e-Learning: implications for pedagogy, technology and organisations. London: Routledge & Falmer.

306

Johnson, D.W., & Johnson, R.T. (1999). Learning together and alone: cooperative, competitive, and individualistic learning. (5 ed.). Needham Heights, MA, USA: Allyn and Bacon.

Jones, R. (2004). Designing adaptable learning resources with learning objects patterns. Journal of Digital Information, 6(1), Article No. 305.

Jorrín-Abellán, I., Rubia-Avi, B., Anguita-Martínez, R., Gómez-Sánchez, E., & Martínez-Monés, A. (in press). Bouncing between the dark and the bright sides: can technology help in qualitative research? Qualitative Inquiry.

Jorrín-Abellán, I.M. (2006). Formative portrayals emerging from CSCL situations (in Spanish). Doctoral Thesis, University of Valladolid, Valladolid, Spain.

Jorrín-Abellán, I.M., Dimitriadis, Y., Rubia-Avi, B., Anguita-Martínez, R., & Ruíz-Requies, I. (2006). A new formative pedagogical model emerged from the experience applicable to engineering courses based on CSCL. In S. Lord & D. T. Hayhurst (Eds.), Proceedings of Frontiers in Education 36th Annual Conference (pp. T2C-7-12) San Diego, California: IEEE Computer Society.

Karagiorgi, Y., & Symeou, L. (2005). Translating constructivism into instructional design: potential and limitations. Educational Technology & Society, 8(1), 17-27.

Kenny, R.F., Zhang, Z., Scwier, R.A., & Campbell, K. (2005). A review of what instructional designers do: question answered and question not asked. Canadian Journal of Learning and Technology, 31(1), 9-26.

Kindley, R.W. (2002). Scenario-Based E-learning: A Step Beyond Traditional E-Learning. Learning Circuits, 3(5).

King, A. (2007). Scripting collaborative learning processes: a cognitive perspective. In F. Fischer, I. Kollar, H. Mandl, & J. Haake (Eds.), Scripting computer-supported collaborative learning. Cognitive, computational, and educational perspectives (pp. 24-48). New York: Springer.

Kirschner, P.A. (2002). Can we support CSCL? Educational, social and technological affordances for learning. In P. A. Kirschner (Ed.), Inaugural Address, Three Worlds of CSCL. Can We Support CSCL? (pp. 7-47). Heerlen: Open Universiteit Nederland.

Kirschner, P.A., Carr, C., van Merriënboer, J., & Sloep, P. (2002). How experts designers design. Performance Improvement Quarterly, 15(4), 86-104.

Kirschner, P.A., Strijbos, J., Kreijns, K., & Beers, P.J. (2004). Designing electronic collaborative learning environments. Educational Technology Research and Development, 52(3), 47-66.

Kitchenham, B.A. (1996). Evaluating software engineering methods and tool—part 2: selecting an appropriate evaluation method - technical criteria. ACM SIGSOFT Software Engineering Notes, 21(2), 11-15.

Knight, C., Grasevic, D., & Richards, G. (2006). An ontology-based framework for bridging learning design and learning content. Educational Technology and Society, 9(1), 23-37.

Knight, P. (1995). Assessment for learning in higher education. London: Kogan Page.

Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., & Fischer, F. (submitted). Specifying collaboration scripts. International Journal of Computer-Supported Collaborative Learning,

Kollar, I., Fischer, F., & Hesse, F.W. (2003). Cooperation scripts for Computer-Supported Collaborative Learning. In B. Wasson, R. Baggetun, & U. Hoppe (Eds.), Proceedings of Computer Supported Collaborative Learning 2003, Community Events - Communication and Interaction (pp. 59-61) Bergen, Norway: InterMedia.

REFERENCES

307

Kollar, I., Fischer, F., & Slotta, J.D. (2005). Internal and external collaboration scripts in web-based science learning at schools. In T. Koschmann, D. Suthers, & T. W. Chan (Eds.), Proceedings of Computer Supported Collaborative Learning 2005: The Next 10 Years! (pp. 331-340) Mahwah, NJ: Lawrence Erlbaum Associates.

Koper, R. (Ed.) Special Issue on "Current Research in Learning Design", Educational Technology & Society 9(1), 2006.

Koper, R., & Burgos, D. (2005). Developing advanced Units of Learning using IMS Learning Design level B. Advanced Technology for Learning, 2(4), 252-259.

Koper, R., & Olivier, B. (2004). Representing the Learning Design of Units of Learning. Educational Technology & Society, 7(3), 97-111.

Koper, R., & Tattersall, C. (Eds.) Learning Design, a handbook on modelling and delivering networked education and training. Heidelberg: Springer-Verlag, 2005.

Koschmann, T. (1996). Paradigm shift and instructional technology. In T. Koshmann (Ed.), CSCL: Theory and Practice of an emerging paradigm (pp. 1-23). New Jersey: Lawrence Erlbaum.

LAMS (2006). LAMS, Learning activity management system. Retrieved April 2007 from http://www.lamsfoundation.org/

Learning Patterns (2005). Learning patterns for the design and deployment of mathematical games Kaleidoscope-NoE JEIRP website. Retrieved April 2007 from http://lp.noe-kaleidoscope.org/

Leclerc, D. and Poumay, M. (2005). The 8 learning events model, Release 2005.1. Retrieved April 2007 from http://www.labset.net/media/prod/8LEM.pdf

Littlejohn, A.H. (2005). Key issues in the design and delivery of learning and teaching. In P. Levy & S. Roberts (Eds.), Developing the new learning environment: the changing role of the academic librarian London: Routledge.

LN4LD (2005). Example IMS LD packages, Open University of The Netherlands. Retrieved April 2007 from http://dspace.learningnetworks.org/handle/1820/257

Lukosch, S., & Schümmer, T. (2006). Groupware development support with technology patterns. International Journal of Human-Computer Studies, 64(7), 599-610.

Lundgren-Cayrol, K., Marino, O., Paquette, G., Léonard, M., & de la Teja, I. (2006). Implementation and deployment process of IMS Learning Design: findings from the Canadian IDLD research project. In Kinshuk, R. Koper, P. Kommers, P. Kirschner, D. G. Sampson, & W. Didderen (Eds.), Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 581-585) Kerkrade, The Netherlands: IEEE Computer Society.

Martel, C., Vignollet, L., & Ferraris, C. (2006). Modeling the case study with LDL and implementing it with LDI. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1158-1159) Kerkrade, the Netherlands: IEEE Computer Society.

Martel, C., Vignollet, L., Ferraris, C., David, J.P., & Lejeune, A. (2006a). LDL: an alternative EML. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1107-1108) Kerkrade, the Netherlands: IEEE Computer Society.

Martel, C., Vignollet, L., Ferraris, C., David, J.P., & Lejeune, A. (2006b). Modelling collaborative learning activities on e-learning platforms. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 707-709) Kerkrade, the Netherlands: IEEE Computer Society.

308

Martens, H., & Vogten, H. (2005). A reference implementation of a Learning Design engine. In K. Rob & C. Tattersall (Eds.), Learning Design, a handbook on modelling and delivering networked education and training (pp. 91-108). Heidelberg: Springer-Verlag.

Martínez-Monés, A. (2003). Method and model to computationally support evaluation in CSCL (in Spanish). Doctoral Thesis, University of Valladolid, Valladolid, Spain.

Martínez-Monés, A., Dimitriadis, Y., Gómez-Sánchez, E., Jorrín-Abellán, I.M., Rubia-Avi, B., & Marcos-García, J.A. (2006). Studying participation networks in collaboration using mixed methods. International Journal on Computer-Supported Collaborative Learning, 1(3), 383-408.

Martínez-Monés, A., Dimitriadis, Y., Rubia-Avi, B., Gómez-Sánchez, E., & de la Fuente-Redondo, P. (2003). Combining qualitative evaluation and social network analysis for the study of classroom social interactions. Computers and Education, 41(4), 353-368.

Martínez-Monés, A., Gómez-Sánchez, E., Dimitriadis, Y., Jorrín-Abellán, I.M., Rubia-Avi, B., & Vega-Gorgojo, G. (2005). Multiple case studies to enhance project-based learning in a computer architecture course. IEEE Transactions on Education, 48(3), 482-489.

McAndrew, P., Goodyear, P., & Dalziel, J. (2005). Patterns, designs and activities: unifying descriptions of learning structures. International Journal of Learning Technology, 36(1) http://kn.open.ac.uk/public/getfile.cfm?documentfileid=6000

McConnell, D. (2000). Implementing computer supported collaborative learning. (2 ed.). London: Kogan Page.

Merril, M.D. (2002). First principles of instruction. Educational Technology Research & Development, 50(3), 43-59.

Miao, Y. (2005). CoSMoS: facilitating learning designers to author units of learning using IMS LD. In Proceedings of 13th International Conference on Computers in Education (pp. 275-282) Singapore: IOS Press.

Miao, Y., Hoeksema, K., Hoppe, H.U., & Harrer, A. (2005). CSCL scripts: Modelling features and potential use. In T. Koschmann, D. Suthers, & T. W. Chan (Eds.), Proceedings of Computer Supported Collaborative Learning 2005 (pp. 423-432) Mahwah, NJ: Lawrence Erlbaum Associates.

Milligan, C.D., Beauvoir, P., & Sharples, P. (2005). The Reload Learning Design tools. Journal of Interactive Media in Education, (7) Available at http://jime.open.ac.uk/2005/06/

Millis, B.J., & Cottell, P.G. (1998). Cooperative learning for higher education faculty. Phoenix, AZ: The Oryx Press.

Muller, M., & Kuhn, S. (1993). Participatory Design. Communications of the ACM, 36(4), 25-28.

Muller, M.J., Wildman, D.M., & White, E.A. (1993). Taxonomy of PD practices: a brief practitioner's guide. Communications of the ACM, 36(4), 26-27.

Muńoz-Merino, P.J., Delgado-Kloos, C., Seepold, R., & Crespo-García, R.M. (2006). Rating the importance of different LMS functionalities. In S. Lord & D. T. Hayhurst (Eds.), Proceedings of Frontiers in Education 36th Annual Conference (pp. T1C-13-18) San Diego, California: IEEE Computer Society.

Mäkitalo, K., Weinberger, A., Häkkinen, P., Järvelä, S., & Fischer, F. (2005). Epistemic cooperation scripts in online learning environments: fostering learning by reducing uncertainty in discourse? Computers in Human Behavior, 21(4), 603-622.

Mäkitalo-Siegl, K., Kaplan, F., Zottmann, J., Dillenbourg, P., & Fischer, F. (2007). The classroom of the future: orchestrating collaborative learning spaces, Technical Report, Kaleidoscope Alpine Rendez-Vous.

REFERENCES

309

NISE (1997). Doing CL: CL Structures. Retrieved September 2006 from http://www.wcer.wisc.edu/archive/cl1/CL/

Nodenot, T., & Laforcade, P. (2006a). CPM: a UML profile to design cooperative PBL situations at didactical level. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1113-1114) Kerkrade, the Netherlands: IEEE Computer Society.

Nodenot, T., & Laforcade, P. (2006b). Learning from a planets game: elements of a didactical transposition described with the CPM language. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1164-1165) Kerkrade, the Netherlands: IEEE Computer Society.

O'Donnel, A.M., & Dansereau, D.F. (1992). Scripted cooperation in student dyads: A method for analyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups: The theoretical anatomy of group learning (pp. 120-141). London: Cambridge University Press.

Oliver, R., Harper, B., Hedberg, J., Wills, S., & Agostinho, S. (2002). Formalising the description of learning designs. In A. Goody, J. Herrington, & M. Northcote (Eds.), Quality conversations: research and development in higher education (pp. 496-504). Jamison, ACT: HERDSA.

OrbiTeam (2007). Basic Support for Cooperative Work, http://www.bscw.de/. Retrieved April 2007

Osuna, C., & Dimitriadis, Y. (1999). A framework for the development of educational collaborative applications based on social constructivism. In Proceedings of the International Workshop on Groupware (pp. 254-261), Cancun, Mexico.

OTEC (2006). Educational technology expertise centre at the Open University of the Netherlands. Retrieved April 2007 from http://www.ou.nl/eCache/DEF/22/853.html

Palomino-Ramirez, L., Martínez-Monés, A., Bote-Lorenzo, M.L., Asensio-Pérez, J.I., & Dimitriadis, Y. (2007). Data flow between tools: towards a composition-based solution for learning design. In Proceedings of 7th IEEE International Conference on Advanced Learning Technologies (in press) Niigata, Japan: IEEE Computer Society Press.

Paquette, G. (2006). Discussion of collaborative Learning Designs, languages, models and tools - LICEF-Télé-université's contribution. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1105-1106) Kerkrade, the Netherlands: IEEE Computer Society.

Paquette, G., & Léonard, M. (2006). The educational modelling of a collaborative game using MOT+LD. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1156-1157) Kerkrade, the Netherlands: IEEE Computer Society.

Paulsen, M. F. (1995). The online report on pedagogical techniques for computer-mediated communication. Retrieved April 2007 from http://www.nettskolen.com/forskning/19/cmcped.html

Pilkington, R.M., & Walker, S.A. (2003) Facilitating debate in networked learning: reflecting on online synchronous discussion in higher education. Instructional Science, 31(1-2), 41-63.

PoInter (2001). Patterns of interaction: a pattern language for CSCW. Retrieved August 2005 from http://www.comp.lancs.ac.uk/computing/research/cseg/projects/pointer/pointer.html

PPP (2005). The pedagogical patterns project. Retrieved May 2005 from http://www.pedagogicalpatterns.org/

Rada, R., & Hu, K. (2002). Patterns in Student-Student Commenting. IEEE Transactions on Education, 45(3), 262-268.

310

Rawlings, A., van Rosmalen, P., Koper, R., Rodríguez-Artacho, M., & Lefrere, P. (2002). Survey of Educational Modelling Languages (EMLs), Technical Report of the CEN/ISSS Learning Technology Workshop.

Reigeluth, C.M. (1999). What is Instructional-Design theory and how is it changing? In C. M. Reigeluth (Ed.), Instructional-Design theories and models: a new paradigm of instructional theory (pp. 5-29). Mahwah: Lawrence Erlbaum.

Renkl, A., Mandl, H., & Gruber, H. (1996). Inert knowledge: analyzes and remedies. Educational Psychologist, 31(2), 115-221.

Renzulli, J. S. & Reis, S. M. (2005). Schoolwide enrichment model. Retrieved April 2007 from http://www.gifted.uconn.edu/sem/semart.html

Retalis, S., Georgiakakis, P., & Dimitriadis, Y. (2006). Eliciting design patterns for e-learning systems. Computer Science Education, special issue on Pedagogic Patterns, 16(2), 105-118.

Riehle, d.Z.H. (1996). Understanding and using patterns in software development. Theory and Practice of Object Systems, 2(1), 3-13.

Rodríguez-Artacho, M., & Verdejo-Maíllo, M.F. (2004). Modeling educational content: the cognitive approach of the PALO language. Educationa Technology & Society, 7(3), 124-137.

Rodríguez-Estévez, J., Caeiro-Rodríguez, M., & Santos-Gago, J.M. (2003). Standarization in computer based learning. Upgrade (European Journal for the Informatics Professional), special issue on e-Learning: Boarderless education, 4(5), 8-15.

Rogers, P.L. (2002). Designing instruction for Technology-Enhanced Learning. Hershey, PA: Idea Group Publishing.

Ronteltap, F., Goodyear, P., & Bartoluzzi, S. (2004). A pattern language as an instrument in designing for productive learning conversations. In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 4271-4276) Lugano, Switzerland.

Roschelle, J., & Teasley, S. (1995). The construction of shared knowledge in collaborative problem solving. In C. O'Malley (Ed.), Computer-supported collaborative learning (pp. 69-197). Berlin, Germany: Springer Verlag.

Rosengard, J.M., & Ursu, M.F. (2004). Ontological representations of software patterns. In R. J. Howlett, M. Negoita, & L. C. Jain (Eds.), Proceedings of 8th International Conference on Knowledge-Based Intelligent Information & Engineering Systems. Lecture Notes on Computer Science (pp. 31-37) Wellington, New Zealand: Springer-Verlag.

Prins, F.J., Sluijsmans, D.M.A., Kirschner, P.A., & Strijbos, J.W. (2005). Formative peer assessment in a CSCL environment: a case study. Assessment and evaluation in higher education, 30(4), 417-444.

Ruiz-Requies, I., Anguita-Martínez, R., & Jorrín-Abellán, I.M. (2006). Un estudio de casos basado en el análisis de competencias para el nuevo maestro/a experto en nuevas tecnologías aplicadas a la educación. Revista Latinoamericana de Tecnología Educativa, 5(2), 357-368.

Salingaros, N.A. (2000). The structure of pattern languages. Architectural Research Quarterly, 4, 149-161. http://www.math.utsa.edu/sphere/salingar/StructurePattern.html

Sampson, D.G., Karampiperis, P., & Zervas, P. (2005). ASK-LDT: a web-based learning scenarios authoring environment based on IMS Learning Design. Advanced Technology for Learning, 2(4), 207-215.

Santoro, F.M., Borges, M.R.S., & Santos, N. (2004). Planning the collaboration process: one-way to make it happen. In Proceedings of 8th International Conference on Computer-Supported Collaborative Work in Design Xiamen, China.

REFERENCES

311

Schümmer, T. (2003). GAMA, a pattern language for computer supported dynamic collaboration. In Proceedings of European Conference on Pattern Languages of Programs Irsee, Germany.

Schümmer, T., & Lukosch, S. (in press). Patterns for computer- mediated interaction. Wiley.

Shuell, T.J. (1992). Designing instructional computing systems for meaningful learning. In M. Jones & P. Winnie (Eds.), Adaptive Learning Environments (pp. 19-54). New York: Springer-Verlag.

Sicilia, M.A. (2006). Semantic learning designs: recording assumptions and guidelines. British Journal of Educational Technology, 37(3), 331-350.

Simon, H.A. (1982). Models of bounded rationality. Cambridge: The MIT Press.

Slavin, R.E. (1995). Cooperative learning: theory, research and practice. (2 ed.). Boston: Allyn & Bacon.

Sloep, P., Hummel, H., & Manderveld, J. (2005). Basic design procedures for e-learning courses. In R. Koper & C. Tattersall (Eds.), Learning Design, a handbook on modelling and delivering networked education and training (pp. 139-160). Heidelberg: Springer-Verlag.

Soller, A., Martínez-Monés, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: a review of state of the art technology for supporting collaborative learning. International Journal on Artificial Intelligence in Education, 15(4), 261-290.

SQR (1997). Nud*IST. Software for qualitative data analysis. Thousand Oaks, CA: Scolari.

Stahl, G. (2006). Essays on technology, interaction and cognition. In G. Stahl (Ed.), Group Cognition: computer support for building collaborative knowledge (pp. 1-24). Cambridge, MA: MIT Press.

Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: a historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences Cambridge, UK: Cambridge University Press.

Stake, R.E. (1995). The art of case study research. London: Sage Publications.

Stake, R.E. (2005). Multiple case study analysis. New York: The Guilford Press.

Stallings, W. (1998). SNMP and SNMP v2: the infrastructure for network management. IEEE Communications Magazine, 36(3), 37-43.

Stevens, W.R. (1998). Unix Network Programming vol. 1: Networking APIs: Sockets and XTI. (2 ed.). Prentice-Hall.

Stevens, W.R. (1995). TCP/IP Illustrated vol. 1: the protocols. Reading, Massachusetts: Addison-Wesley.

Strijbos, J.W., Martens, R.L., & Jochems, W.M.G. (2004). Designing for interaction: Six steps to designing computer-supported group-based learning. Computers & Education, 42(4), 403-424.

Surma, D.R. (2003). Lab exercises and learning activities for courses in computer networks. In Proceedings of 33rd IEEE Frontiers in Education Conference (pp. T2C-21-25)

Tatar, D. (in press). The design tensions framework. Journal of Computer-Human Interaction.

Tattersall, C. (2006a). Comparing educational modelling languages on a case study: an approach using IMS Learning Design. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1154-1155) Kerkrade, the Netherlands: IEEE Computer Society.

312

Tattersall, C. (2006b). Using IMS Learning Design to model collaborative learning activities. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1103-1104) Kerkrade, the Netherlands: IEEE Computer Society.

Tattersall, C. & Koper, R. (2005). Special Issue on "Advances in Learning Design", Journal of Interactive Media in Education.

Tattersall, C., Vogten, H., Brouns, F., Koper, R., van Rosmalen, P., Sloep, P., & van Bruggen, J. (2005). How to create flexible runtime delivery of distance learning courses. Educational Technology and Society, 8(3), 226-236.

Tcpdump (2006). www.tcpdump.org. Retrieved April 2007

TELL (2005a). Design patterns for teachers and educational (system) designers. Retrieved April 2007a from http://cosy.ted.unipi.gr/TELL/media/TELL_pattern_book.pdf

TELL (2005b). TELL project website. Retrieved April 2007b from http://cosy.ted.unipi.gr/tell/

Tesanovic, A. (2001). What is a pattern? In Proceedings of 4th International Symposium in Intelligent Data Analysis Lisbon, Portugal.

Torres, J., Dodero, J.M., Aedo, I., & Zarraonandia, T. (2006). An architectural framework for composition and execution of complex learning processes. In Proceedings of 5th IEEE International Conference on Advanced Learning Technologies (pp. 143-147) Kaohsiung, Taiwan: IEEE Computer Society Press.

UNFOLD (2004). Understanding New Frameworks Of Learning Design project website. Retrieved April 2007 from http://www.unfold-project.net

Universanté (2002). Universanté Project Website, http://www.universante.org. Retrieved April 2007 from http://www.universante.org

University of Bolton (2004). Reload project website. Retrieved April 2007 from http://www.reload.ac.uk/

van der Vegt, W. (2005). CopperAuthor project website. Retrieved 29 March 2005 from http://www.copperauthor.org/

van Es, R., & Koper, R. (2006). Testing the pedagogical expressiveness of IMS LD. Educational Technolgy & Society, 9(1), 229-249.

Vega-Gorgojo, G., Bote-Lorenzo, M.L., Gómez-Sánchez, E., Asensio-Pérez, J.I., Dimitriadis, Y., & Jorrín-Abellán, I.M. (2006). Ontoolcole: an ontology for the semantic search of CSCL services. In Y. Dimitriadis, I. Zigurs, & E. Gómez-Sánchez (Eds.), Proceedings of 12th International Workshop, LNCS 4154 (pp. 310-325) Heidelberg: Springer-Verlag.

Vega-Gorgojo, G., Bote-Lorenzo, M.L., Gómez-Sánchez, E., Dimitriadis, Y., & Asensio-Pérez, J.I. (2005). Semantic search of learning services in a grid-based collaborative system. In Proceedings of the 5th IEEE/ACM International Symposium on Cluster Computing and the Grid, Workshop on Collaborative and Learning Applications of Grid Technology and Grid Education (pp. 19-26) Cardiff, UK.

Vesseur, A., Lutgens, G., Broek, A., Koehorst, A., & Ronteltap, F. (2005). A pattern language for the design, implementation, assessment and evaluation of network supported collaborative learning courses, Technical Report, Learning Lab, University of Maastricht, Maastricht, The Netherlands.

Vignollet, L., David, J.P., Ferraris, C., Martel, C., & Lejeune, A. (2006). Comparing educational modelling languages on a case study. In Kinshuk, R. Koper, P. Kommers, P. Kirschner, D. G. Sampson, & W. Didderen (Eds.), Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 1149-1150) Kerkrade, The Netherlands: IEEE Computer Society.

REFERENCES

313

Villasclaras-Fernández, E.D. (2005). Development of a tool for authoring Learning Designs based on Collaborative Learning Flow Patterns, Master Thesis, University of Valladolid, Valladolid, Spain.

Villasclaras-Fernández, E.D., Hernández-Leo, D., Asensio-Pérez, J.I., & Dimitriadis, Y. (accepted). Towards pattern-based design of embedded assessment in CSCL settings. Computer Supported Collaborative Learning 2007, workshop on "Linking CSCL design patterns to authentic educational case studies" New Jersey, USA.

Vogten, H., Martens, H., Nadolski, R., Tattersall, C., van Rosmalen, P., & Koper, R. (2006). CopperCore Service Integration - Integrating IMS Learning Design and IMS Question and Test Interoperability. In Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies (pp. 378-382) Kerkrade, the Netherlands: IEEE Computer Society.

W3C (1996). Extensible Markup Language (XML). Retrieved April 2007 from http://www.w3.org/XML/

Waters, S.H., & Gibbons, A.S. (2004). Design languages, notation systems and instructional technology: a case study. Educational Technology Research and Development, 52(2), 57-68.

Weinberger, A., Fischer, F., & Stegmann, K. (2005). Computer-Supported Collaborative Learning in higher education: scripts for argumentative knowledge construction in distributed groups. In T. Koschmann, D. Suthers, & T. W. Chan (Eds.), Proceedings of Computer Supported Collaborative Learning 2005: The Next 10 Years! (pp. 717-726) Mahwah, NJ: Lawrence Erlbaum Associates.

Williams, D., Coles, L., Wilson, K., Richardson, A., & Tuson, J. (2000). Teachers and ICT: current use and future needs. British Journal of Educational Technology, 31(4), 307-320.

Wilson, S. (2005). Architectures to support authoring and content management with Learning Design. In R. Koper & C. Tattersall (Eds.), Learning Design, a handbook on modelling and delivering networked education and training (pp. 40-62). Heidelberg: Springer-Verlag.

Zarraonandia, T., Dodero, J.M., & Fernández, C. (2006). Crosscutting runtime adaptations of LD execution. Educational Technology and Society, 9(1), 123-137.

Zelkowitz, M.V., & Wallace, D.R. (1998). Experimental models for validating technology. IEEE Computer, 31(5), 23-31.