An Open Source LMS for a Mission Critical Enterprise-Level Application:

Are we there yet?Kevin Kelly, Wen Hao Chuang

Academic Technology,

San Francisco State University

Copyright © [Wen Hao Chuang and Kevin Kelly] [2008]. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors.

2

Session topics (outline)

• Background: Learning Management System (LMS) landscape at SF State

• Lessons learned about making an open source LMS enterprise-ready

O Development / Systems PerspectiveO Teaching and Learning PerspectiveO Organizational Administration Perspective

3

Introduction: Review of LMS at SF State

F 05S 05F 04S 04F 03S 03F 02S 02

1. Reasonably steady state with BbO 1997 to 2002

2. Period of problems with BbO Jan 2003 to present

3. Faculty focus groups to determine need to investigate alt. LMSO Dec 2003

4. Investigation of LMS alternativesO Spring 2004

5. iLearn alpha test (5 instructors, 400 students)O Fall 2004

6. iLearn beta test (25 instructors, 1500 students)O Spring 2005

7. iLearn large scale deployment test (150+ instructors, 9,000+ students)O Fall 2005 to Spring 2006

S 06

4

Usage statistics

Number of students using iLearn (Semester courses only)

3001346

5171

7556

14590

19000

23500

0

5000

10000

15000

20000

25000

Fall2004

Spring2005

Fall2005

Spring2006

Fall2006

Spring2007

Fall2007

Term

# s

tud

en

ts

Number of students using iLearn(Semester courses only)

5

Usage statistics

Number of instructors/Number of course sections

0

200

400

600

800

1000

1200

1400

1600

1800

Fall2004

Spring2005

Fall2005

Spring2006

Fall2006

Spring2007

Fall2007

Term

Cou

nt Number of active courses

(Semester courses only)

Number of instructors usingiLearn (Semester courses only)

6

Looking at the LMS from 3 perspectives

7

Looking at the LMS from 3 perspectives

8

Technology Management (outline)

• System Architecture

• Development & QA

• Lessons learned and Recommendations

9

System architecture (before)

10

System architecture (now)

11

System setup (now)

• 5 Sun Fire X4100 ServersO 2 dual-core 64-bit AMD processors (for each server)O 1 Load Balancer (running squid)O 2 Application ServersO 1 Database Server (running MySQL)O 1 Admin and Backup Server

• LAMP (Linux, Apache, MySQL, PHP) StackO Red Hat Enterprise Linux 4O Apache 2.2.xO MySQL 5.0.22O PHP 5O Others

Zend Optimizer (PHP accelerator) -> eAccelerator Squid, clamav, MRTG, awstats, Nagios, Splunk(x), Ganglia, etc.

12

Data and storage solutions

• Backup solutions

• For a large database like ours, the moodle default site-wide backup codes do not work.

• MySQL dump and nightly rsync to disk storage, then to tape.

• Database storage engines

• MyISAM (default)

• InnoDB

• Moodle’s “course-centered” design

• Solution: Diva? (diva.sfsu.edu)

13

Problems with current architecture

• During mid-term and final exam periods, the two application servers get really loaded

O Also related to how teachers are using it

• DatabaseO MySQL vs. Postgres vs. OracleO MyISAM vs. InnoDB (storage engines)

• Other possible optimization solutionsO Squid vs. varnishO Apache vs. lighttpdO LAMP tuning and optimization

14

Problems with Data and Storage

• You probably need a DBA! (for larger sites)O Some tuning tips: http://day32.com/MySQL/

• Backup issuesO MyISAM vs. InnoDB

• Database issues:O Current database size (MySQL dump) over 11G!O Avg 200 queries per secondO In 30 days avg 151,675,735 queriesO Peak connections between 400-500 concurrent

connectionsO Table indexO Optimization and table maintenance

15

Future hardware plans (1)

• Pillar NAS and SAN storage unit

O Switch from xRaid to dedicated terabyte NAS system.

O Higher efficiency for storing data.O Allows for snapshotting of data.

• Introduce BAD (Bad Ass Database)

O High-end 64 gigabyte database server

O Repurpose current database system as another application server (3 APP servers behind squid).

16

Future hardware plans (2)

• VirtualizationO Moving to VMWare ESX

(possibly this summer)O Dynamic allocation of

server resourcesO Enhance the flexibility

and scalability of enterprise level application

O Reduce possible downtime (VMotion)

17

Programming & Development

• Small development team: 2 full-time developers (including project management)

• Version control system

• SVN: (internal)

• CVS: (external with Moodle core team)

• Bug tracking system: Trac (open source)

• For bug reports

• For project management purposes

• Build-in Wiki (for internal documentation)

18

Typical Build Process

• Developers’ codes got committed into their personal branch (SVN commit)

• “Buildmaster” merges all the codes before code freeze date

• Deploy a testing build on ilearntest (DEMO)

• If tested without any problem, code releases are “svn switched” to a “deployed” branch by system admin team (avoid conflict of interest).

19

20

Development tools

• PHP

• NuSphere PhpED (great debugger/IDE)

• Eclipse (with PHPeclipse plugin) (free)

• PSPad (Windows) and vi/vim (Linux) (free)

• Other tools

• JMeter (simulating traffic)

• Perl (script for performance profiling)

• XML Editors (BB to moodle conversion tool)

21

QA Testing

• Small QA testing team

• 3 part-time student assistants

• Supervised by DEV staff members

• Tools that we are using

• Selenium IDE / RC

• TestNG / JUnit

• JMeter (stress testing)

• Tools that we are evaluating

• QA Wizard Pro (Seapine software inc.)

• Quick Test Pro (HP)

• Silktest (Borland)

22

Problems with QA

• Training student assistants

• Time, resource, and testing scopeO After all, it’s open-source!

• Budget, select the right tool, and manual testing

• To automate or not to automate?

23

Lessons learned and recommendations

• Make a good projection about your LMS traffic and usage, plan your server architecture accordingly.

• Plan your workflow / processes in advance.

• Start with a fresh DB every year

• Schedule regular down time to repair/maintain tables

• QA testing!• Knowledge: * [Promote] HSU “Introduction to Moodle Programming”

course

24

Looking at the LMS from 3 perspectives

25

Teaching and Learning:Engaging the Community• Outreach

O Training and support team visits faculty department meetings

O Conduct needs assessment and formative evaluation focus groups

O Advertise training and support options

• InputO Advisory groups help inform decisionsO Exemplary instructors share best practices

26

Teaching and Learning:Engaging the Community - Best Practices

27

Work Flow:Support for Faculty, Staff, & Students

28

Work Flow:Support for Faculty, Staff, & Students

29

Teaching and Learning: Training and support• Constant evolution of training options

O Held frequent workshops throughout Fall and Spring semesters with extra offerings and open labs at the beginning

O Added week-long summer and winter workshops (with lunch)

O Modularized week-long workshops

• Big change for supportO Standardized format for "QuickGuides"O Switch to database-driven QuickGuides

Dynamic approach will update multiple QuickGuides at once

30

Looking at the LMS from 3 perspectives

31

Organizational Administration:Policy issuesAcademic policies (Academic Senate)

Intellectual property

Faculty workload

Recognition for Retention/Tenure/Promotion purposes

Copyright/TEACH Act

Accessibility

Articulation

Privacy

Contact hours for students

Assessment and Teacher Evaluations

Quality of online courses

Minimum requirements for online/hybrid courses

Communication with students (virtual office hours) Student code of conduct Academic integrity

32

Organizational Administration:Policy issues

Operational policies (Service Level Agreement)

• Technology ManagementO Storage limitsO Archive practices

• Organizational AdministrationO Incentives for training

Also see article:

Waterhouse, S. & Rogers, R.O. (2004). The importance of policies in e-learning instruction. Educause Quarterly 3, 28-39.

33

Closing

Q & A

Contact information

• Wen Hao ChuangO wchuang@sfsu.eduO 415.405.3543

• Kevin KellyO kkelly@sfsu.eduO 415.338.6455