From Onsite to Online Learning Contexts:

Pedagogical and Practical Insights into Online Course Design

Dr. Anandi Nagarajan

GSE, Rutgers University

November 27, 2012

Background• Teaching

▫ Psychology of Learning▫ Problem-based Learning▫ Program Evaluation▫ Developmental Educational Psychology

• Research focus ▫ Use of video-cases as a learning context▫ Computer-supported collaborative learning▫ PBL online and facilitation in e-learning contexts▫ Cost-effectiveness and evaluation of online learning environments (Mellon

foundation grant)▫ Leaning in a simulated laboratory

• Current Interests▫ Developing pedagogically effective and practically efficient online courses▫ Incorporating and facilitating collaborative dialogue and in both hybrid and

online courses by creating a cognitive, social and teaching presence in an online course.

▫ Exploring and evaluating multiple options to bring some synchronous video chat features to my current online courses.

QUALITY ONtheLINE

• The Institute for Higher Education Policy reviewed existing benchmarks for distance education in 2000▫ Institutional support ▫Course development▫Teaching/learning▫Course structure▫Student support▫Faculty support▫Evaluation and Assessment

• Surveys, debates, and research has since brought new ideas and instructional design models to the forefront

Going the DistanceOnline Education in the United States, 2011Babson Survey Research Group

Recent Survey Research (2011)based on responses from 2500 universities and colleges

• Over 6.1 million students were taking at least one online course during the fall 2010 term; an increase of 560,000 students over the number reported the previous year. 

• The 10% growth rate for online enrollments far exceeds the less than 1% growth of the overall higher education student population. 

• 31% of all higher education students now take at least one course online.

• In the first report of this series in 2003, 57%of academic leaders rated the learning outcomes in online education as the same or superior to those in face-to-face.  That number is now 67%, a small but noteworthy increase. 

• 1/3rd of all academic leaders continue to believe that the learning outcomes for online education are inferior to those of face-to-face instruction. 

Challenges facing online learning

• SRI International reported:▫ Found that blended learning conditions resulted in slightly higher outcome measures 

than face to face. Differences between purely online versus face-to-face could not be computed due to few number of studies.

▫ So basically, the face-to-face component in a hybrid course may add an advantage and one has to consider how to incorporate that into a purely online course.

▫ Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies, 2010

• Issues with accreditation of online courses and programs▫ Chronicle of Higher Education

• Competition from MOOCs (massive open online courses) that are offered free▫ National Program on Technology Enhanced Learning (IIT, India), OpenCourseware 

(MIT), Coursera, Khan Academy …

• Clearly a need exists to look into improved online course design models that address:▫ instructional quality, appropriate level of content, efficient & context-appropriate

pedagogical/communication methods, and effective assessment procedures

Some questions we need to address• How do we design online courses that enhance

student learning, are efficient, and cost-effective?

• What are some best practices for planning, implementing, and evaluating an online course?

• What are some ways to incorporate both synchronous and asynchronous modes of communication?

• What are effective means of formative and summative assessment and feedback?

Plan for today’s talk•My personal journey from onsite to online

•Exemplary online course design models and rubrics for evaluation

•A look at my e-courses

•Considerations for the online masters program at Rutgers

•Quality, Sustainability & Cost-effectiveness

From on-site to online

Virginia Commonwealth UniversityCenter for Teaching Excellence, 2009

From on-site to online

Virginia Commonwealth UniversityCenter for Teaching Excellence, 2009

On-Site to Hybrid to Online• Mostly intuition versus following a model design• Experience with socio-constructivist methods

and online learning▫PBL▫CSCL

• A direct-mapping approach with some tweaking where appropriate

• Areas for consideration▫Course structure and content▫Course materials ▫Instructional methods ▫Time commitment▫Group work and collaboration contexts▫Assessment and feedback

On-site to Online

• Some relatively easy transitions▫Course structure, reading assignments, lecture

slides, individual case analysis and assignments

• Some challenges faced▫Collaborative discussion on readings▫Problem-based learning scenarios▫ Individual versus group accountability online▫Assessment of participation and engagement▫Feedback practices – frequency, content, and

depth▫Written versus verbal medium* important to note these as potential issues faced by new instructors

Insights• Transition is more than a cut and paste approach• Requires additional sets of skills• Needs advanced planning and time allocation for

the different tasks▫Deciding course structure and content,▫Providing additional resources and modes of

communication as needed,▫Initiating, sustaining and facilitating discussion,▫Providing feedback on all aspects of participation

and assignments, and▫Learning technical skills needed to support the

instructional goals and affordances of e-college.

TPCK

TPCK:Essential skills for a successful online course

Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A new framework for teacher knowledge . Teachers College Record 108 (6), 1017-1054.

Knowledge about the

subject/domain

Ideas and techniques on how to teach the content

Technical skills on developing tools and course content in an online context

A combination of content,

pedagogical, and technological

knowledge and skills

Some Examples:

•Idaho State University•Northern Illinois University•Oklahoma State University•Pace University•University of Central Florida

Analyze Design Develop Implement EvaluatePre-planning; thinking about the course

Design your course on paper

Develop course materials & assemble the course

Begin teaching Look at the course outcomes with a critical eye

•Audience•Goal•Objectives•Identify content•Identify delivery  and environment.•Instructional strategies•Assessment strategies•Formative evaluation•Constraints

•Name the learning units of Instruction•Identify content and strategies for an individual unit of instruction•Write instructions for the learning unit•Name the menu items for a learning module•Consider different assessments

•Based on design phase•Build content, assignments, assessments•Build course structure•Upload content

•Overview of course•Expectations•Initiate instruction•Interaction•Ask for feedback early on (formative evaluation)

•Did the students achieve expected learning outcomes?•What have you learned?•How can you make the course better?

ADDIE model of instructional design

Important to align ANALYZE and EVALUATE aspects with the department and school goals

Self-evaluation•Some Exemplary Online Course Design Models and Resources

California State University, Chico Virginia Commonwealth University University of Illinois, Illinois Online Network University of Southern Mississippi, Learning

Enhancement Center Quality Matters, Maryland Online Michigan State University Stanford Research Institute

Rubric for Online Learning

•Developed by California State University•6 parameters

▫Learner Support & Resources▫Online Organization &  Design▫Instructional Design & Delivery  ▫Assessment & Evaluation of Student▫Innovative Teaching with Technology  ▫Faculty Use of Student Feedback

Learner Support & ResourcesBaseline Effective Exemplary

A.Course contains limited information for online learner support and links to campus resources. B.Course provides limited course-specific resources, limited contact information for instructor, department and/or program. C.Course offers access to a limited number of resources supporting course content.

A.Course contains some information for online learner support and links to campus resources. B.Course provides some course-specific resources, some contact information for instructor, department and program. C.Course offers access to some resources supporting course content.

A.Course contains extensive information about being an online learner and links to campus resources. B.Course provides a variety of course-specific resources, contact information for instructor, department and program. C.Course offers access to a wide range of resources supporting course content.

Online Organization & DesignBaseline Effective Exemplary

A.Much of the course is under construction, with some key components identified such as the syllabus. B.Course syllabus is unclear about what is expected of students.

C.Aesthetic design does not present and communicate course information clearly. D.Web pages are inconsistent both visually and functionally. E.Accessibility issues are not addressed.

A.Course is organized and navigable. Students can understand the components and structure of the course. B.Course syllabus identifies and delineates the role the online environment will play in the course. C.Aesthetic design presents and communicates course information clearly. D.Most web pages are visually and functionally consistent. E.Accessibility issues are briefly addressed.

A.Course is well-organized and easy to navigate. Students can understand all components and structure of the course. B.Course syllabus identifies and clearly delineates the role the online environment will play in the total course. C.Aesthetic design presents and communicates course information clearly . D.All web pages are visually and functionally consistent . E.Accessibility issues are addressed throughout

Instructional Design & DeliveryBaseline Effective Exemplary

A.Course offers limited opportunity for interaction and communication S-S, S-I, S-C. B.Course goals are not clearly defined and do not align to learning objectives. C.Learning objectives are vague or incomplete and learning activities are absent or unclear. D.Course provides few visual, textual, kinesthetic and/or auditory activities to enhance student learning. E.Course provides limited or no activities to help students develop critical thinking and/or problem solving skills

A.Course offers some opportunities for interaction & communication S-S, S-I, S-C. B.Course goals are defined but may not align to learning objectives. C.Learning objectives are identified and learning activities are implied. D.Course provides some visual, textual, kinesthetic and/or auditory activities to enhance student learning. E.Course provides some activities to help students develop critical thinking and/ or problem-solving skills.

A.Course offers ample opportunities for interaction and communication S-S, S-I, S-C. B.Course goals are clearly defined and aligned to learning objectives.

C.Learning objectives are identified and learning activities are clearly integrated. D.Course provides multiple visual, textual, kinesthetic and/or auditory activities to enhance student learning. E.Course provides multiple activities that help students develop critical thinking and /or problem-solving skills.

Assessment & Evaluation of Student Learning

Baseline Effective ExemplaryA.Course has limited activities to assess student readiness for course content and mode of delivery. B.Learning objectives, instructional and assessment activities are not aligned. C.Assessment strategies are used to measure content knowledge, attitudes and skills. D.Feedback about own performance is infrequent and sporadic. E.Students' self-assessments and/peer feedback opportunities are limited or do not exist.

A.Course has some activities to assess student readiness for course content and mode of delivery. B.Learning objectives, instructional and assessment activities are somewhat aligned. C.Ongoing strategies are used to measure content knowledge, attitudes and skills. D.Feedback about performance is provided. E.Students' self-assessments and peer feedback opportunities exist.

A.Course has multiple timely and appropriate activities to assess student readiness for course content and mode of delivery. B.Learning objectives, instructional and assessment activities are closely aligned. C.Ongoing multiple assessment strategies are used to measure knowledge, attitudes and skills. D.Regular feedback about student performance is provided in a timely manner. E.Students' self-assessments and peer feedback opportunities exist throughout the course.

Innovative Teaching with Technology

Baseline Effective ExemplaryA.Course uses limited technology tools to facilitate communication and learning.

B.New teaching methods are applied to enhance student learning. C.Multimedia elements and/or learning objects are limited or non-existent.

D.Course uses Internet access and engages students in the learning process.

A.Course uses some technology tools to facilitate communication and learning. B.New teaching methods are applied to innovatively enhance student learning. C.Multimedia elements and/or learning objects are used and are relevant to student learning. D.Course optimizes Internet access and effectively engages students in the learning process.

A.Course uses a variety of technology tools to appropriately facilitate communication and learning. B.New teaching methods are applied and innovatively enhance learning, and engage students. C.A variety of multimedia elements and/or learning objects are used and are relevant to student learning throughout the course. D.Course optimizes Internet access and effectively engages students in the learning process in a variety of ways throughout the course.

Available Tools: An example

Faculty Use of Student Feedback

Baseline Effective ExemplaryA.Instructor offers limited opportunity for students to give feedback to faculty on course content. B.Instructor offers limited opportunity for students to give feedback on ease of online technology in course. C.Instructor uses student feedback at the end of the semester to help plan instruction and assessment of student learning for the next semester.

A.Instructor offers some opportunities for students to give feedback on course content. B.Instructor offers some opportunities for students to give feedback on ease of online technology in course. C.Instructor requests and uses student feedback a couple times during the semester to help plan instruction and assessment of student learning for the rest of the semester.

A.Instructor offers multiple opportunities for students to give feedback on course content. B.Instructor offers multiple opportunities for students to give feedback on ease of online technology in course. C.Instructor uses formal and informal student feedback on an ongoing basis to help plan instruction and assessment of student learning throughout the semester.

 

Needs assessment and self-evaluation

•Other Exemplary Online Course Design Models

University of Illinois, Illinois Online Network University of Southern Mississippi, Learning

Enhancement Center Quality Matters, Maryland Online Michigan State University

•Other Resources Virginia Commonwealth University Stanford Research Institute

A closer look at my teaching

• Learning Contexts▫Face-to-face, Web-supported, Hybrid or blended,

Completely online• Learning Portals

▫Ecollege @ Rutgers Psychology of Learning

▫Sakai @ Rutgers Problem-based Learning Program Evaluation (Fall’07, Fall’08) Psychology of Learning

▫Blackboard @ Rider Developmental Educational Psychology Learning & Memory

E-college sample course 1- Spring 2010

Review the ecourse Spring 2010•Whole-class discussion•Small-group case analysis•Reflections•Artifact analysis•PBL online•Individual versus group products•Doc-Sharing

Self-evaluationPsychology of Learning – Spring 2010: 1 face to face meeting

What worked What needed change

• Communication with students via email, phone, weekly reflections and online discussion (private and whole-class)

• Small-group and whole-class interaction on asynchronous discussion board

• Synchronous chat was attempted

• Facilitation of weekly discussions shared among instructor and students

• Multiple, frequent assessments• Individual and group

accountability

• Course structured in themes was confusing▫ Changed it to weekly modules

• Students were not natural facilitators▫ Needed scaffolding and training on

asking effective questions• Synchronous chat was not

manageable▫ Selectively picked fewer tools

• Redundant assignments were eliminated

• Too many written assignments▫ Impacted contribution to the

discussion• Sample papers were uploaded to

clarify expectations• Downloadable resources were added

to DocSharing.

E-college sample course 2- Spring 2011

Self-evaluationPsychology of Learning – Spring 2011: Completely online

What worked

• Weekly modules improved course structure and student understanding.

• Gradual access to course content made it manageable for students

• Orientation to facilitation and scaffolded training on asking effective questions was helpful.

• Sample papers made expectations clear.• Time management in facilitating online discussion and grading • Bi-weekly email check with each student and a phone call as

needed added the personal interaction component.• Personalizing assignments based on student background and

professional interest increased engagement & motivation.

Continued effortsNeed to:

▫Provide a visual model for the tasks and discussion cycle during the week

▫Evaluate level of participation and engagement▫ Improve facilitation skills in students▫ Incorporate some synchronous video chat – either

individually or small-group efficiently and effectively▫Combine textbook readings and journal articles▫Continue to locate resources and case studies that apply

to the diverse range of students in my courses Counseling, nursing, special education, dance

▫ Include some testing option that can assess conceptual knowledge gained.

▫ (All my current assignments are open-ended and application-oriented)

Technological Resources

•OIRT Faculty training▫Workshops, seminars and one-on-one help

•Online tools▫http://oirt.rutgers.edu/instruc/tools/▫Elluminate documents for ecollege support▫http://www.vcu.edu/cte/resources/OnlineTo

ols/index.html

•24/7 helpdesk and resource person at the Department of Continuing Education, GSE

Online masters program

•Motivation to enroll in the program▫Convenience of online courses▫Access not constrained by geographical location▫Adds qualifications/degree for work-related

promotions and pay-raises especially amongst teachers

▫Acts as a stepping stone towards the doctoral program

Considerations for theOnline Master’s Program at the GSE

• Recruitment▫ Who are we looking to recruit?▫ How do we go about doing it?▫ What makes the online program appealing as opposed to all the

MOOCs that are available for free?• Job prospects on completion

▫ Advancement in current placement▫ Increased teaching qualification▫ Entry-level research positions at ETS, Mathematica…▫ Preparation for doctoral work▫ Others?? Need to actively identify potential job prospects.

• Course offerings as listed in the LCD-Online brochure▫ Are all courses being offered online?▫ Frequency?

Important to consider these issues in developing the program

Other Critical Elements

•Learning, Instruction, & Design ▫Quality of Course Design▫Consistency across Courses▫Faculty and Student Resources

•Bigger Picture▫Accreditation of Online Masters Program▫Sustainability▫Cost-effectiveness

• Quality of Course Design▫ Need for closer look at current practices

▫ Compare current practices with best practices and exemplary design models

▫ Develop guidelines and a rubric for online course design to share with instructors to ensure quality as accreditation of the online program is essential to make it sustainable.

▫ Let the learning objectives lead the technology use rather than the other way around For this to happen, we need to define department and program

objectives clearly and have instructors design and align their courses accordingly

Considerations for theOnline Master’s Program at the GSE

Learning, Instruction & Design

•Online Course Design & Implementation Issues▫ADDIE: Analyze, Design, Develop, Implement,

Evaluate▫Align all courses to a common set of

benchmarks/guidelines▫Certain degree of consistency across courses in

terms of: Structure and organization Collaboration/communication practices, Optimal use of available tools, Frequent and pertinent feedback , Rubrics/Sample work to set clear expectations, Multiple assessments

• Instructor Resources

▫ Benchmarks or rubrics that can serve as guide to course design AND as an evaluation tool

▫ Instructor collaboration and idea sharing with respect to online courses Teaching online can be an isolated experience and it is

critical that we provide a community/forum for idea generation, synergy, motivation, and creativity

▫ Monthly or bi-weekly (virtual) meetings to share ideas/issues/resources. Similar to a teachers’ forum.

▫ A course development day before the semester begins▫ Customized faculty training workshops that address

course design issues – already available

Learning, Instruction & Design

Bigger picture•Quality and Accreditation

▫ The Middle States Commission on Higher Education have developed Interregional Guidelines for the Evaluation of Distance Education (Online Learning)

▫ Critical to examine our current course offerings against these guidelines and develop rubrics aligned to these guidelines

▫ Research and review the courses and programs offered at other universities

•Sustainability▫ Start-up costs to develop courses (effort, time, technical and

cognitive resources)▫ Number of students per course per semester▫ Optimal and efficient use of technological tools, activities, &

assessments to keep it sustainable and manageable

•Cost-effectiveness

Questions?