1. Assessment Portfolios: An Integrated Model-Based Approach Supporting The Needs And Scenarios Across Users
Jacques Raynauld, HEC Montral and MATI Montral
Christian Martel, Universit de Savoie and Pentila

2. Key Words
Flexible
Comprehensive
Epic 2011 Conference London, UK July 11-13
2

3. Overview
Our portfolio vision
Twofundamental portfolio challenges
Our solution : integration of twocomplementarytools
OpenSyllabus
EmaEvaL
Demo of proof of concept
Future work and Road-map
Epic 2011 Conference London, UK July 11-13
3

4. Context
MATI Montral is a research center for HEC Montral, cole polytechnique and University of Montral
Started a strategic initiative on guided- assessment portfolios in 2008 manyschoolsinvolved
4
Epic 2011 Conference London, UK July 11-13

5. Context
Universit de Savoie group of researchers worked on the project since 2007
Pentila : spin-off on an assessment solution
EvalComp Consortium: community of French universities using the system since 2009
5
Epic 2011 Conference London, UK July 11-13

6. Our portfolio vision
The assessment or evaluation process is the key driving force for individuals to create artefacts that will populate their portfolios
Epic 2011 Conference London, UK July 11-13
6

7. Two fundamental challenges
Many actors or activities involved in the creation of artefacts
Course related activities (instructors)
Internships (outside supervisors)
Co-curricular activities (peer)
Others
Epic 2011 Conference London, UK July 11-13
7

8. Two fundamental challenges
Need to keep track of all the results in light of skills or competencies framework
Different scales
Different rules
Over time
Need for a very flexible tool or set of tools
Epic 2011 Conference London, UK July 11-13
8

9. Our solution integration of two complementary tools
A flexible and semantic authoring environment to create and sequence learning activities
Open Syllabus : used since 2003 at HEC Montral now a Sakai 2.8 tool (10 000 students)
A comprehensive tool to manage any evaluation process
EmaEval : used in numerous French universities since 2009 (6 000 students)
Epic 2011 Conference London, UK July 11-13
9

10. Open Syllabus
An open source model-based approach to organize the content of any pedagogical structure (skills framework, program, course, portfolio)
1. Resources + 2. structure + 3. role + 4. workflow
Epic 2011 Conference London, UK July 11-13
10

11. Open Syllabus
1. Resources
Text
Files (Power Point, PDF, Doc, videos, etc.)
URL
Comments
11
Epic 2011 Conference London, UK July 11-13

12. Open Syllabus
Learning outcomes/competencies
Civic knowledge & engagement (local & global)
Creativethinking
Criticalthinking
Ethicalreasoning & action
Foundations & skills for lifelonglearning
Information literacy
Inquiry & analysis
Integrativelearning
Interculturalknowledge & competence
Oral communication
Problemsolving
.
Association of American Colleges and Universities
http://www.aacu.org/
12
Epic 2011 Conference London, UK July 11-13

13. 1. Resources
Rubrics (forms)
Association of American Colleges and Universities
http://www.aacu.org/
13
Epic 2011 Conference London, UK July 11-13

14. Open Syllabus
Organizing the resources according to a model approach
Structure
Unit
Context
Resources
Root
Structure
Unit
Context
Resource
14
Epic 2011 Conference London, UK July 11-13

15. Open Syllabus
Root
Structure
Competency 1
Porfolio
Unit
Context
Instructions
Text file (instructor)
Year1
Resource
Competency 1
Competency 2
Submit form
Form to submit the artefact (student)
Year 2
Competency 1
Competency 2
Evaluation
Grading rubric form (tutor)
4. Workflow
3. Role
15
Epic 2011 Conference London, UK July 11-13

16. EmaEval
A fully integrated management tool
Integration of all the functionalities related to the evaluation, validation and certification of knowledge and competencies for the education sector
16
Epic 2011 Conference London, UK July 11-13

17. EmaEval Functionalities
Definition of training programs (role distribution, institutional rules, etc.)
Definition of the competency framework
Choice of the evaluation scenario (quiz, session paper, validation of non-formal and informal learning, auto-evaluation, peer evaluation, )
Choice of evaluation methods (criteria, scales, ..)
Follow-up of individual of collective progress
Committeeand certificates emission
Link to administrative systems (results, etc.)
Epic 2011 Conference London, UK July 11-13
17

18. EmaEval
18
Epic 2011 Conference London, UK July 11-13

19. EmaEval
19
Epic 2011 Conference London, UK July 11-13

20. Open Syllabus EmaEval
Google WebToolKit
BonitaWorkflow
Jack-Rabbit
Google WebToolKitjavascriptwidgets
Sling REST API
Jack-Rabbit
Use of external open source tools (ORBEON, EMaEval)
20
Epic 2011 Conference London, UK July 11-13

21. Demo of proof of concept
OpenSyllabus
Set the competencyframework
Create a set of activities
Specify the roles
Identify the evaluationactivities , theirsequences and the criteriaused
EmaEval
Synchronizemanually the comp. framework
Synchronize the roles
Synchronize the activities and the criteriaused
Set of Web services thatlink the differenttools to play the assessementactivities

22. July 2009
All these entries are activities to beexecutedsequentially by the actors
10th Sakai Conference - Boston, MA, U.S.A.
22

23. 23
Epic 2011 Conference London, UK July 11-13

24. Integration and demo
Additional scenarios
Course work : different scales by different instructors
Workshop : auto-evaluation
Internship : same skills are graded by numerous evaluators
Co-curricular activities : the students choose the skills + peer evaluation
Epic 2011 Conference London, UK July 11-13
24

25. Road-map
Proof of concept isworking fine weunderstand the wholechain very flexible
Small Pilot in Septemberwith cole polytechnique de Montral
Automation of the links between the twoplatforms
More stable and robust application nextyear
Epic 2011 Conference London, UK July 11-13
25

26. Road-map
Vimoscreencast
Paper on the conference site
Jacques.raynauld@hec.ca
Christian.Martel@univ-savoie.fr
Questions?
Epic 2011 Conference London, UK July 11-13
26