Design, Development and Implementation of Online Programme on Evaluation of Training

1 | P a g e

Project Report

Needs Analysis, Design, Development, Implementation and Evaluation

of

Online Course

on ‘EVALUATION OF TRAINING’

by

S K Pulist

Enr No. PGDEL-131812560

(Submitted in partial fulfillment of the requirements of the course MDEP-005 of the PG Diploma in eLearning Programme)

to

IGNOU,

Maidan Garhi, New Delhi 110 068.

2013

2 | P a g e

Online Course on

‘EVALUATION OF TRAINING’

3 | P a g e

Table of Contents

Certificate of Originality .............................................................................................. 7

Acknowledgement ....................................................................................................... 8

Chapter 1: Background and Introduction ................................................................. 9

1.1 Introduction ............................................................................................... 9

1.2 Rationale of the Course ........................................................................... 9

1.3 Needs analysis of the Target Audience ............................................... 10

1.4 Assumptions ........................................................................................... 10

1.5 Methods .................................................................................................... 11

1.6 Supports Received ................................................................................. 11

1.7 Limitations ............................................................................................... 11

Chapter 2: Design of the Course.............................................................................. 13

2.1 Title of the Course: ................................................................................. 13

2.2 Instructional Design ............................................................................... 13

2.3 Framework of the Course ...................................................................... 16

2.4 Software Used ......................................................................................... 17

2.5 Self-learning Material .............................................................................. 17

2.6 Teaching and Learning Process ........................................................... 18

2.7 Assessment Strategy ............................................................................. 18

2.8 Programme Evaluation ........................................................................... 19

Chapter 3: The Course .............................................................................................. 20

3.1 Chapter Scheme ...................................................................................... 20

3.2 Features of the Course ........................................................................... 23

3.2.1 General Introduction ................................................................... 23

3.2.2 Introduction Section ................................................................... 24

3.2.3 Learning Style Index ................................................................... 25

3.2.4 Meta-cognitive Thinking Skills Scale ........................................ 25

3.2.5 Schedule Finaliser ...................................................................... 26

3.2.6 Introductory Class on WizIQ ...................................................... 26

3.2.7 Suggested Reading Material ...................................................... 27

3.2.8 Learning Outcomes and Reflective Questions ........................ 28

3.2.9 Discussion Forum ....................................................................... 29

4 | P a g e

3.2.10 General Content ........................................................................... 30

Chapter 4: Learning Experiences ............................................................................ 37

4.1 Implementation Plan ............................................................................... 37

4.2 Course Activities ..................................................................................... 37

4.3 Self Assessment ..................................................................................... 39

4.4 Formative Assessment .......................................................................... 39

4.5 Summative Assessment ........................................................................ 40

4.6 Course Evaluation .................................................................................. 40

4.7 Findings ................................................................................................... 41

Chapter 5: Lessons Learnt and Recommendations .............................................. 43

5.1 Lessons Learnt........................................................................................ 43

5.2 Recommendations .................................................................................. 45

5.3 Conclusion ............................................................................................... 47

References .................................................................................................................. 48

Appendix-1 .................................................................................................................. 49

Appendix-2 .................................................................................................................. 52

Appendix-3 .................................................................................................................. 56

5 | P a g e

List of Figures

Figure 1: Concept Map of Programme in Training and Development ................. 13

Figure 2: A ‘Brief’ on the Course ............................................................................. 23

Figure 3: Components of the ‘Brief’ on Course Page ........................................... 23

Figure 4: Introduction Section ................................................................................. 24

Figure 5: Course Introduction by Voki .................................................................... 24

Figure 6: Learning Style Index ................................................................................. 25

Figure 7: Metacognitive Thinking Skills Scale ....................................................... 25

Figure 8: Virtual Class Scheduler ............................................................................ 26

Figure 9: WizIQ platform ........................................................................................... 27

Figure 10: List of Suggested Reading Material ...................................................... 28

Figure 11: Learning Outcomes and Reflective Questions .................................... 28

Figure 12: Discussion Forum ................................................................................... 29

Figure 13 GoogleGroup created as ‘Community of Practice’ .............................. 30

Figure 14: Audio programme ................................................................................... 30

Figure 15: Video programme .................................................................................... 31

Figure 16: Blog providing the Reading Content .................................................... 32

Figure 17: Virtual Class on WizIQ in progress ....................................................... 32

Figure 18: Tutor Marked Assignment ...................................................................... 33

Figure 19: Computer Marked Assignment .............................................................. 33

Figure 20: Tutor Marked Assignment ...................................................................... 34

Figure 21: Grade Report of the Learners ................................................................ 35

Figure 22: Questionnaire Survey launched on Qualtrics.com ............................. 35

Figure 23: Analysis of the Responses .................................................................... 40

Figure 24: Overall Rating of the Course by the students ..................................... 42

6 | P a g e

List of Tables Table 1: Instructional Design ................................................................................... 14

Table 2: Chapterisation Scheme .............................................................................. 20

7 | P a g e

Certificate of Originality

This is to certify that this Project is my original work which has been submitted

as partial fulfillment of the requirements of the Post-Graduate Diploma in eLearning

Programme of Staff Training and Research Institute of Distance Education, Indira

Gandhi National Open University. This work has not been plagiarized in any way. It

has also not been submitted to any other agency in any kind in part or full. The

sources used in the Report have been duly acknowledged and referenced.

S K Pulist PGDEL-131812560

8 | P a g e

Acknowledgement

The Project Report speaks about the process of needs analysis, design,

development, implementation and evaluation of an online course on ‘Evaluation of

Training’. The Report narrates the different steps involved in the journey. It also

includes the lessons learnt in the process and shares recommendations for design,

development and launch of a similar online programme.

I would like to thank IGNOU for allowing me admission to PGDEL Programme

and giving me the opportunity to work on this Project. I humbly acknowledge the

guidance and support extended from time to time by the Programme Coordinators

Prof. Santosh Panda and Ms Mythili Gowtham. Their progressive scaffolding helped

me in developing myself from a learner to an online programme administrator and

then an eTeacher. The timely technical help extended by Ms Mythili made my

journey on the Moodle smooth. My thanks are due to the Technicians of eGyankosh

who worked hard for ensuring uninterrupted virtual connectivity. The Virtual

Classroom launched through AdobeConnect gave me an exclusive experience and

exposure. I got an opportunity to exchange my experiences with my fellow learners

while in the Workshop and even thereafter. I thank the Coordinators for making the

Workshop a part of the instructional design of the programme and my fellow learners

for becoming part of my learning experience.

Last but not the least, I am thankful to my family for standing by me while I

was experimenting and documenting this Project.

S K Pulist

9 | P a g e

Chapter 1: Background and Introduction

1.1 Introduction

The training is an important component of human resource development

activities. While it helps an individual in bridging the gap between his actual and

expected performance for accomplishment of a task, the organisation benefits from

the improved knowledge, skills and attitudes of that individual. The component of

training helps in converting an individual into an asset for the organisation. The

concept of training and development has gained more focus in the wake of

awareness of human resource development activities in the organisations.

1.2 Rationale of the Course

The people engaged in training and development activities are not normally

trained. They learn different tools and techniques for carrying out different training

related activities during the course of their job only. With the development of

management techniques and organisations becoming sensitive to the requirements

of the human resource, more money is now spent on capacity building and staff

development activities. The awareness has also increased among the senior

management of the organisations with the commitment of National Policy on Skill

Development that at least 2.5% of the salary budget should be spent on training and

other continuous skill enhancement activities for the staff. However, how to assess

the impact of such activities on the organisational objectives is again an arduous

activity which most of the organisations though interested, do not perform due to

extra cost.

In view of the above, it was thought appropriate to develop an online

programme which could help the training functionaries to evaluate different

components of the training programmes and establish the actual worth not only for

the individual but for the organisation as well.

10 | P a g e

1.3 Needs analysis of the Target Audience

The people working in Training and Development Units of different

organisations were contacted. The idea of the development of an online programme

on the theme was appreciated. The needs of these people with regard to training

and development activities in their respective organisations were discussed and their

comments and suggestions on the nomenclature of the programme were sought. The

needs analysis proved that the personnel involved with training of human resource

development in the organisations are normally not trained in this area of

specialization. They just learn by doing with the passage of time on the job through

hit and trial method. It could have been ideal if sufficient opportunities were made

available to such people without letting them leave their job to learn professionally

and excel in this area. ‘How to evaluate the worth of a training programme

conducted by them’ was an area which they required to be attended to with special

skills and knowledge. With this background, the programme on evaluation of training

was proposed to be conducted through online mode.

The content of the programme was chosen to suit the requirements of the

personnel involved with training and development activities in public and private

organisations.

1.4 Assumptions

The programme on ‘Evaluation of Training’ is a specialised programme which

is designed for a specific target group. The programme expects a person having full

knowledge of tools and techniques being implemented and used in conduct of

training and development programmes across the industries and organisations. It

was expected that the persons joining this programme would fully understand the

importance and need of evaluation of a training programme. With this

understanding, the client group is supposed to be working in training development

units/organisations handling training and development and capacity building

activities. Since the programme was launched online, therefore, the entrants were

expected to be computer literate and be able to use different online and web-based

tools meant for programme delivery.

11 | P a g e

1.5 Methods

The course has been launched completely online and therefore, the learners

were made to carry out all activities online. However, in order to know their learning

style a Learning Style Index was provided to them the feedback on which was used

for developing the audio/video programmes and other content for the course. The

Metacognitive Thinking Skills Scale was provided to the learners to enable them to

know their strengths and weaknesses and capitalise on the former. The reading

content was provided to the learners with Moodle. Audio and Video programmes

were uploaded as part of the learning material. A list of suggestive reading material

was also provided to enable them to go for additional material if they so desired. The

virtual classroom was organised with the help of WizIQ (an online educational

platform). In order to enable the learners use the Moodle and WizIQ comfortably, an

orientation session was kept in the beginning of the course on how to use these

platforms.

1.6 Supports Received

The Moodle platform for launching of the course was provided by the

University. The eGyankosh provided the technical support for running the

programme on Moodle. The Programme Coordinators of the PGDEL Programme

provided all the required help. They were prompt in removing the doubts on conduct

of any activity.

1.7 Limitations

The programme was launched online, therefore, it was thought appropriate to

provide full content online. Since the e-content as desired was not fully available

freely online, some of the core content had to be developed by the author. The

Moodle was slow due to connectivity and took longer time in uploading the files.

Therefore, more patience was required for working on this platform. The students

selected for the programme were busy in their own activities and therefore, could not

spare time conveniently. The programme was of 2 credits (60 study hours) and was

to be completed in a month’s time, therefore, it required at least 2 hours of

commitment daily from the students. In view of this, out of more than 15 students

12 | P a g e

enrolled for the course, only 5-6 were found active. Since this was an experiment, the

interest of the students was also limited in the course. However, the learners were

time and again persuaded to complete different activities which they could not do

many times in spite of their will.

13 | P a g e

Chapter 2: Design of the Course

The course has been designed following the ADDIE model of Instructional

Design (Merrill, 1994). The course under reference is a part of the Certificate

Programme in Training and Development which is a 16–Credit Programme

containing 4 courses of 4 credits each. The concept map of the programme is

presented in Figure 1.

Figure 1: Concept Map of Programme in Training and Development

2.1 Title of the Course:

“Evaluation of Training”

The Course is of 2 Credits containing 8 Chapters.

2.2 Instructional Design

The course has been designed keeping in view the needs and requirements

of the individuals working in the area of human resource development in different

organisations. The content of the course has been specially designed so as to

provide the learners with full working knowledge of evaluating of training

programmes. The Instructional Design of the Course is presented in Table 1.

14 | P a g e

Table 1: Instructional Design

Major Content Area/ Topics

Specific Objectives Media to be used

Interactive tools

Assessment tools

Course-3 Learner will be able to:

Explain the concept, definition and principles of evaluation of training

Explain and justify the need for evaluation of training

Identify and mark various stages of evaluation

Explain and apply various methods of evaluation

Choose an appropriate method of evaluation of training in a given context.

Audio, video, text

Discussion forum, Virtual classroom

Quizzes, CMA, TMA,

Chapter 1:

Concept,

Definition

and Need

for

Evaluation

Learner will be able to:

Explain the concept of evaluation of training

Define the evaluation of training

Explain and establish the need of training

Audio, video, text


Quizzes, CMA, TMA,

Chapter 2:

Principles of

Evaluation


Explain the various principles of evaluation of training

Define the role of the evaluators

Report the results on the basis of available data

Make clear assumptions on the basis of which

Audio, video, text


Quizzes, CMA, TMA,

15 | P a g e

evaluation design will be based

Chapter 3

and 4:

Typologies

of

Evaluation


Explain the various types of evaluation

Explain various models of evaluation

Audio, video, text


Quizzes, CMA, TMA,

Chapter 5:

Stages of

Evaluation


Explain the various stages of evaluation

Compare various models of evaluation

Set different stages of evaluation in a given situation

Audio, video, text


Quizzes, CMA, TMA,

Chapter 6,

7 and 8:

Evaluation

Methods


Explain the various methods of evaluation of training

Explain various pre-requisites for undertaking evaluation work at different stages

Undertake different activities for evaluation at pre-training stage

Apply evaluation mechanism at during training stage

Set the mechanism for evaluation at post training stage

Use various devices and tools at different stages of evaluation.

Audio, video, text


Quizzes, CMA, TMA,

16 | P a g e

2.3 Framework of the Course

The Course on ‘Evaluation of Training’ was launched online and therefore,

different software and online tools were used to provide academic and administrative

support to the learners. Broadly, the following methods were used for different

activities of the programme:

Moodle - the Mainframe: The Moodle was used for providing overall framework of

the programme and its different chapters. The learners were supposed

to login on the Moodle which further directed them to the concerned

site/content as per choice.

Conduct of Survey: It was thought appropriate to know about the learning styles of

the learners so that different learning objects could be designed and

created keeping in view their styles. At the same time, it was also

thought necessary to acquaint the learners with their metacognitive

thinking skills. In order to launch the questionnaires on both the above,

GoogleGroup was used.

Introduction Section: An introduction section was created for the benefit of the

learners which carried the content related to introduction to the course

and the facilitator. The ‘Voki’ ‘Avtar’ – the online narrator with modulated

voice was used for this purpose.

News Forum: The Moodle News Forum was used to provide relevant information

about the course to the learners.

Collaboration tool: GoogleGroup was used to provide an informal collaborative

platform to the learners where they could share their views with peers.

Dissemination of course content: The digital reading content was provided with

the help of ‘Blog’ which was updated from time to time with the required

content for the Chapters. The Videos from YouTube relevant to the topics

were also provided through the Blog.

17 | P a g e

Audio/Video Programmes: The Audio and Video programmes were launched

through the ‘ScreenCast’ website which was linked to the Moodle.

Quizes: The Quizzes and Computer Marked Assignments (CMAs) were created and

launched on the Moodle.

Programme Evaluation: A questionnaire survey was designed for administering on

the learners for programme evaluation purposes which was launched on

‘qualtrics.com’ and linked to the Moodle.

Virtual Classroom: In order to provide online interaction on the programme and

conduct online sessions, WizIQ platform was used.

Session Scheduler: In order to decide the schedule for virtual

interaction/classroom, programme scheduler ‘Doodle’ was used which

provided for marking the available time for facilitating a common available

time with the participants. The link of this scheduler was emailed to the

participants with a request to mark the available time for the virtual

classes.

2.4 Software Used

Launching an online course is a sophisticated activity. Keeping in view the

requirements, the following software were used to handle different components of the

course:

MS-Word : to create the reading e-content (text) MS-PowerPoint : to create presentation slides Audacity : to create audio programmes Camtasia : to create video programmes TipCam : to capture the screen Moodle : to launch the programme WizIQ : to conduct virtual classes FreeMind : for concept mapping AllVideoConverter : to convert the audios / videos in different formats

2.5 Self-learning Material

The content used for the course has been developed following a mixed

approach i.e. some of the content has been developed by the author himself and

18 | P a g e

remaining part of the content has been made available from the OERs. The course

content consisted of the text, audio/video files and power point presentations. While

content developed by the author was made available through LMS (with link to blog),

the external content was made available through web links. The audio/ video

programmes and presentations were suitably integrated with the main content

keeping in view the learning objectives of the chapters.

2.6 Teaching and Learning Process

The learners took self-study of the content. While audio/video programmes

including presentations provided learning support to the students and helped in

clearing various concepts, the discussion forum provided through GoogleGroup gave

them an opportunity to share their views and discuss the topics of common interest

with peers. The virtual classroom activities were organised through WizIQ twice a

week for two hours as per convenience of the students.

2.7 Assessment Strategy

The multiple-channel assessment strategy was adopted for this course. In

the first instance, the learners were tested on their understanding of different

concepts with the help of reflective questions. These questions were based on the

objectives of the concerned chapter and were provided to them before they started

the course. They could try to find out the answers to those questions and construct

their knowledge. This activity was supposed to keep them focused and prevent from

getting distracted through attraction of other irrelevant but interesting web links.

During the course, the learners were evaluated on the basis of their

interaction in the discussion forum, group discussion initiation/interaction and

moderation (these were used for feedback only with no weightage), computer

marked assignments and tutor marked assignments.

At the term-end level, the learners were given tutor marked assignment and

they were required to submit brief answers to the questions given. These were

evaluated as part of the summative assessment. The following methodology was

adopted for learner’s performance evaluation:

19 | P a g e

Evaluation Methodology

Reflective Questions : Internal feedback Self-Check exercises : Internal feedback Discussion Forum : feedback only Tutor Marked Assignments : 70% weightage Computer Marked Assignments : 30% weightage Total Marks : 100

2.8 Programme Evaluation

The Programme evaluation is an important activity in order to know the quality

of a programme, its effectiveness and impact upon the learners. The systematic

approach was adopted for evaluation of this course also. The learners were

administered an online questionnaire survey at the end of the course to provide

feedback on different components of the course. They were requested to provide

their free and frank feedback. The feedback so received was analysed and the

findings have been presented in Chapter 4 of this Report.

20 | P a g e

Chapter 3: The Course

The course has been presented using a multiple-media approach. The

different components of the Instructional Design have been media-mixed in a way

that suits the learning styles of the individual learners. While the lessons have been

provided through text files, the audio and video lectures have been used to

supplement the course. The PowerPoint presentations have been included wherever

necessary.

3.1 Chapter Scheme

The course on ‘Evaluation of Training’ has been divided into 8 Chapters.

Each chapter has been systematically presented for the benefit of the learners. The

chapterisation scheme of the course is given in Table 2.

Table 2: Chapterisation Scheme

Chapteri-sation/

Description Design Phase Development Phase (VC: Virtual Class)

BLOCK 1

Unit 0: Rationale and Background of the Course

Chapter 1: Concept, Definition and Need for Evaluation

1.1 Learning Outcomes ppt, audio PPT, audacity

1.2 Reflections Text html/text

1.3 Introduction Text, video html/text, VC

1.4 Concept of Evaluation Text, video html/text, VC

1.5 Definition Text, video html/text, VC

1.6 Need for Evaluation Text, video html/text, VC

1.6.1 Why to evaluate? Text, video html/text, VC

1.6.2 When to evaluate? Text, video html/text, VC

1.6.3 What to evaluate? Text, video html/text, VC

1.6.4 How to evaluate? Text, video html/text, VC

1.7 Summing up Text, video html/text, VC

1.8 Self-Check Exercises Text html/text

1.9 Bibliography and further readings

Text html/text

Chapter 2: Principles of Evaluation




21 | P a g e

Chapteri-sation/


2.4 Principles of Evaluation Text, video html/text, VC

2.4.1 Clarity Text, video html/text, VC

2.4.2 Objectivity Text, video html/text, VC

2.4.3 Reliability Text, video html/text, VC

2.4.4 Feasibility Text, video html/text, VC

2.4.5 Evaluation: A process Text, video html/text, VC

2.4.6 Evaluation Design Text, video html/text, VC

2.5 Role of the Evaluator Text, video html/text, VC

2.6 Reporting of Evaluation Data Text, video html/text, VC

2.7 Level of Training Techniques Text, video html/text, VC




Text html/text

Chapter 3: Typologies of Evaluation-1




3.4 Typologies of Evaluation Text, video html/text, VC

3.4.1 Hamblin’s Model Text, graph, video html/text, VC

3.4.2 Kirkpatric’s Design Text, graph, video html/text, VC

3.4.3 Warr’s Framework Text, graph, video html/text, VC




Text html/text

Chapter 4: Typologies of Evaluation-2




4.4.1 Virmani and Premila’s Model Text, graph, video html/text, VC

4.4.2 Peter Bramley’s Model Text, graph, video html/text, VC

4.4.3 David Reay’s Approach Text, graph, video html/text, VC




Text html/text

BLOCK 2

Chapter 5: Stages of Evaluation




5.4 Stages of Evaluation Text, video html/text, VC

5.4.1 Pre-Training Text, video html/text, VC

5.4.2 During Training Text, video html/text, VC

5.4.3 Post-Training Text, video html/text, VC


5.6 To-Do Activity Text html/text

5.7 Bibliography and further Text html/text

22 | P a g e

Chapteri-sation/


readings

Chapter 6: Evaluation Methods-1




6.4 Pre-Training Evaluation Text, video html/text, VC

6.4.1 Identification of Training Needs Text, video html/text, VC

6.4.2 Evaluation of Performance Standards

Text, video html/text, VC

6.4.3 Evaluation of Training Objectives


6.4.4 Evaluation of Trainee’s Profile Text, video html/text, VC

6.4.5 Input Evaluation Text, graph, video html/text, VC




Text html/text





7.4 During Training Evaluation Text, video html/text, VC

7.5 Precautions for Mid Course Review





Text html/text





8.4 Post Training Evaluation Text, video html/text, VC

8.4.1 Reaction Evaluation Text, video html/text, VC

8.4.2 Learning Evaluation Text, video html/text, VC

8.5 Job Behaviour Evaluation Text, video html/text, VC

8.5.1 Width Techniques Text, video html/text, VC

8.5.2 Depth Techniques Text, video html/text, VC

8.6 Job Improvement Plan Text, video html/text, VC

8.7 Ultimate Value Evaluation Text, video html/text, VC

8.8 Follow Up of Results Text, video html/text, VC




Text html/text

23 | P a g e

3.2 Features of the Course

The course has been developed as a comprehensive learning package so

that it is able to support the learners as per their learning styles. The salient features

of the course are briefly discussed below:

3.2.1 General Introduction

This section provides know-how about the course and its expectations from

the learners. The snapshot from Moodle is presented in Figure-2.

Figure 2: A ‘Brief’ on the Course

This Section provides for different items like concept map, objectives of the

course, course outline, instructional design, teaching learning process and

assessment plan as given in Figure 3.

Figure 3: Components of the ‘Brief’ on Course Page

24 | P a g e

3.2.2 Introduction Section

This section welcomes the learners to the course and provides introduction to

the Facilitator and the course through video programmes. A GoogleGroup created for

interaction among the learners was also linked to this area The snapshot from

Moodle is presented in Figure 4.

Figure 4: Introduction Section

Use of Voki for the introduction section was interesting. It helped in catching

the attention of the learners. A snapshot of the Voki programme is presented in

Figure 5.

Figure 5: Course Introduction by Voki

25 | P a g e

The Introduction Section provides for the following important components:

3.2.3 Learning Style Index

The Learning Style Index was developed with a view to know the learning

styles of the learners (Appendix 1). It was hosted on GoogleDrive and was linked to

the Moodle in the course area. The feedback received was used for developing

instructional components for the course. The Index prepared by Solomon & Felder

(n.d.) was taken as reference for preparing this Index. The Snapshot of the Index is


Figure 6: Learning Style Index

3.2.4 Meta-cognitive Thinking Skills Scale

The Metacognitive Thinking Skills Scale was designed with a view to helping

the learners know their different skills (Appendix 2). The scale was expected to help

the learners know their week areas so that they could work on improving these areas

and capitalize on the strong areas. The Scale designed by Tuncer & Kaysi (2013)

was taken as reference for preparing this Scale. The snapshot of the scale is


Figure 7: Metacognitive Thinking Skills Scale

26 | P a g e

3.2.5 Schedule Finaliser

Since the learners were fulltime working people, there was a limitation of

availability of common free time with them all in order to enable them to attend the

virtual classes. Therefore, in order to know their common free time, help of Doodle

was taken and a scheduler was prepared on which the learners were asked to

provide the information about time availability. Efforts were made to host the virtual

class when maximum number of learners was available. The Snapshot of the

Doodle is presented in Figure 8.

Figure 8: Virtual Class Scheduler

3.2.6 Introductory Class on WizIQ

Since the Moodle was to be used for offering the course and virtual

classes were to be conducted through WizIQ, it was thought appropriate to

given the learners an introduction to both the Platforms so that they could

work on them comfortably. Therefore, an introductory class for Moodle and

WizIQ explaining various features which the learners would be using for

undergoing this course was conducted. The snapshot of the WizIQ course

area is presented in Figure 9.

27 | P a g e

Figure 9: WizIQ platform

3.2.7 Suggested Reading Material

This section provides a list of suggestive reading material which the learners

could refer in addition to the content provided. The Snapshot is presented in Figure

10.

28 | P a g e

Figure 10: List of Suggested Reading Material

3.2.8 Learning Outcomes and Reflective Questions

Each chapter has been provided with Learning Outcomes and Reflective

Questions. The learning outcomes put forth the expectations from the learners after

having gone through the chapter.

Figure 11: Learning Outcomes and Reflective Questions

29 | P a g e

On the other hand the reflective questions provided the learners with an

opportunity to assess their knowledge about the topic before they really go through

the content. The learners were expected to match the possible answers pre and post

reading of the chapter. The Moodle snapshot of the same is presented in Figure 11.

3.2.9 Discussion Forum

A discussion Forum was created on the Moodle for the learners. The

learners were supposed to discuss the topic given by the coordinator/facilitator.

Once all the learners had commented on the topic, a collective review of the

comments was to be taken which was to be moderated by the coordinator. The

snapshot of the discussion forum is presented in Figure 12.

Figure 12: Discussion Forum

A GoogleGroup on ‘Evaluation of Training’ was created for the course with

membership to all the learners. This group provided the learners a common informal

platform where they could discuss about their problems related to different topics of

the course and also the progress made by them. The Group helped in building a

‘community of practice’ and motivated the learners to help each other, learn from

each other and appreciate each other’s works. The snapshot of the GoogleGroup is

given in Figure 13.

30 | P a g e

Figure 13 GoogleGroup created as ‘Community of Practice’

3.2.10 General Content

The learning content for each of the chapters based on the course curriculum

was provided in the following different forms keeping in view the learning styles of the

learners:

Audio programme

The audio content of each chapter was provided in the form of Audio

Programme uploaded on screencast.com and linked to Moodle to enable the link to

open as pop-up. The snapshot of the programme is presented in Figure 14.

Figure 14: Audio programme

31 | P a g e

Video programme

The recorded video lectures on each chapter were provided on the Moodle

the same way as the audio programmes were provided. The video programmes also

were able to open as pop-up. It facilitated the learners not to leave the Moodle

platform while viewing the lecture. They could put the pop-up off, the moment lecture

was over. The Moodle Snapshot is presented in Figure 15.

Figure 15: Video programme

Reading Content

The reading content was provided to the learners through Blog. The material

was uploaded on the Blog and was linked to the Moodle. The learners were able to

view the content without losing the contact of the LMS. This was arranged through a

pop-up window. The snapshot of the Blog presenting the reading content is


32 | P a g e

Figure 16: Blog providing the Reading Content

PowerPoint presentation

The PowerPoint presentations used for lectures were also uploaded on the

relevant chapter section so that the learners could make their own explanations while

viewing the slides. The same slides were used for preparing video presentations and

virtual classroom.

Virtual classroom

The WizIQ was used to provide virtual classes to the learners. The lectures

were organised twice a week after taking availability of time with the learners through

‘Doodle’. The snapshot of the virtual class in progress in presented in Figure 17.

Figure 17: Virtual Class on WizIQ in progress

33 | P a g e

Tutor Marked Assignment Tutor Marked Assignments (TMAs) were part of the formative and summative

assessment of learner’s performance as per the assessment plan for the course.

The TMAs were provided through the Moodle. The snapshot of the TMA provided on

the Moodle is presented in Figure 18.

Figure 18: Tutor Marked Assignment

Computer Marked Assignment

The Computer Marked Assignments (CMAs) were made part of the formative

assessment of the learner’s performance in the course. They were designed and

developed in the Moodle itself. The snapshot of the same is presented in Figure 19.

Figure 19: Computer Marked Assignment

34 | P a g e

Term End Examination

The Term End Examination for the course was conducted through Tutor

Marked Assignment (TMA). The learners were given topics with word limit and were

asked to upload their answers on the LMS. The same were evaluated offline. The

snapshot of the TMA meant for the Term End Examination is presented in Figure 20.

Figure 20: Tutor Marked Assignment

Grade Report

As mentioned earlier the evaluation methodology for the course included

formative and summative evaluation through TMA and CMA. The Grade Report

reflected the grades /percentile marks obtained by each of the learners who

appeared for the respective items. The snapshot of the Grade Report is presented in

Figure 21

35 | P a g e

Figure 21: Grade Report of the Learners

Programme Evaluation

The Project Guidelines for the MDEP-5 envisaged for evaluation of the course

developed and launched by the learners as part of their Project based on the

learners’ feedback. In view of this, with a view to having the feedback of the learners

who participated in the course, a questionnaire survey was designed and uploaded

on the Qualtrics.com. In all 18 items on different aspects of the course were kept in

the tool. The learners were requested to give their free and frank feedback about the

course through this survey. The responses received were analysed and the results

are presented in Chapter 4.The snapshot of the survey is presented in Figure 22.

The complete questionnaire is appended as Appendix 3.

Figure 22: Questionnaire Survey launched on Qualtrics.com

36 | P a g e

The full course has been presented as a self-sustained package for the

learners. It was integrated to provide access to all the relevant resources spread

across the Web on single platform on a click-away distance.

37 | P a g e

Chapter 4: Learning Experiences

The course was implemented online through the Learning Management

System (Moodle) provided by IGNOU. The Course was delivered to the registered

learners as per the plan mentioned in the forthcoming paragraphs.

4.1 Implementation Plan

The course on ‘Evaluation of Training’ contained 8 Chapters. The following

timeline was followed for implementation of the course:

The course was rolled on 28th October, 2013 for four weeks’ time. Since the

course was of 2 credits, 60 hours of student study was mandatory. Therefore, 2

chapters were taken every week for study. This was extra workload on the students.

While all other activities were kept as per the plan, there were two virtual classes

every week. The students were requested to go through the content provide to them

through the Blog1 linked to Moodle with the help of pop-up window. Quizzes and

computer marked assignments were provided to the students as per requirement of

the course instructional design. The tutor marked assignments were also provided

as per evaluation methodology of the course. The students were requested to

adhere to the timeline for submission of these components.

4.2 Course Activities

The first activity for the course was to register the interested students to the

course. In the beginning 10 students were identified and registered to the course.

However, due to their pre-occupation, they could not spare time and showed inability

to follow the timeline. But since the course was to be completed in a time-bound

manner, therefore, another lot of 6 students was registered. They were requested to

spare time from their busy schedule and undertake this course seriously. They were

1 http://evaluationoftraining.blogspot.in/

http://evaluationoftraining.blogspot.in/

38 | P a g e

assured of all help. It was also assured that the virtual classes would be held only on

the time convenient to majority of the students.

The course was launched as per the schedule discussed above. The students

were briefed about the course, its requirements and expectations from them. In order

to know their learning style, a learning style index2 was administered on the learners.

At the same time a Metacognitive thinking skills scale3 was also introduced to enable

the learners know their skills so that they could improve upon the desired areas. A

special session of virtual class was arranged on the WizIQ. The Moodle platform and

WizIQ were introduced to the students through this class. Different aspects of the

platform and tools available to them for use were thoroughly explained. They were

demonstrated as to how to perform different student activities on both the above

platforms so that their experience becomes enriching rather a burden on them.

The learners were requested to upload their photo and complete the profile

on the Moodle. They were introduced to the course and the course facilitator with the

help of the Voki audio/video files4. They were requested to make maximum use of

the GoogleGroup5 specially created for the course to share their views on different

topics. The study material for each chapter was provided through the Blog. The

students were requested to go through the audio and video presentations provided

with each of the chapters. The PowerPoint slides were also provided with each of

the chapters in order to help them to build their own presentation on the topic.

Each Chapter was introduced starting with the learning outcomes for that

Chapter and the reflective questions based on the content to be undertaken by the

students for that forthcoming course. They were requested to build their answers

before they really went through the content so that they could test their prior

knowledge about the course.

2

https://docs.google.com/a/ignou.ac.in/forms/d/1KwRJPIakhl9coXtWm8JsX5IMcQQwdEawEAGUxY7hzKA/viewform 3

https://docs.google.com/a/ignou.ac.in/forms/d/1uWofNeom8UK0UueJPUq_tyw9x2A8ZvX8I6m5VRza49g/viewform 4 http://www.voki.com/pickup.php?scid=8413485&height=267&width=200

5 https://groups.google.com/forum/#!forum/evaluationoftraining

https://docs.google.com/a/ignou.ac.in/forms/d/1KwRJPIakhl9coXtWm8JsX5IMcQQwdEawEAGUxY7hzKA/viewform

https://docs.google.com/a/ignou.ac.in/forms/d/1KwRJPIakhl9coXtWm8JsX5IMcQQwdEawEAGUxY7hzKA/viewform

https://docs.google.com/a/ignou.ac.in/forms/d/1uWofNeom8UK0UueJPUq_tyw9x2A8ZvX8I6m5VRza49g/viewform

https://docs.google.com/a/ignou.ac.in/forms/d/1uWofNeom8UK0UueJPUq_tyw9x2A8ZvX8I6m5VRza49g/viewform

http://www.voki.com/pickup.php?scid=8413485&height=267&width=200

https://groups.google.com/forum/#!forum/evaluationoftraining

39 | P a g e

As committed by the course facilitator, Doodle6 was used to decide the

common time for organizing virtual classroom. The chapters were then covered

through virtual classes organized twice a week through WizIQ. A small introduction

of the course was provided on the WizIQ before the class was started. The slides

were uploaded on the platform and then discussed with the students. They were

suggested to ask questions anytime during the lecture so that they do not waste their

time in framing and re-framing the same question till such time they were not heard.

The News Forum on Moodle was used from time to time to disseminate

important information on the course for the benefit of the registered students. The

Moodle Discussion Form was used to discuss pre-decided topics and receive

comments from the learners so that they could be then commented upon by the

peers and discussed as a group.

4.3 Self Assessment

The self assessment activities were provided through Moodle. The reflective

questions were provided with each chapter to enable the students to assess their

prior knowledge. This was a mechanism followed to provide feedback to the students

which was instant in this case since the student would immediately know as to what

was known to him/her and get focused primarily on those topics where he/she was

lacking in knowledge.

4.4 Formative Assessment

The formative assessment was used for continuous monitoring of the

progress of the students in the course. Quizzes and computer marked assignments

(CMA) formed part of the formative assessment methodology. The students were

asked to perform and attempt these quizzes and CMAs carefully since they were to

be evaluated and their grades/marks were to be added to the final Grade Report of

each learner.

6 http://doodle.com/skpulist

http://doodle.com/skpulist

40 | P a g e

4.5 Summative Assessment

The Tutor Marked Assignments (TMAs) were used as the tool for summative

assessment of the performance of the students. The students were requested to

attempt the question as provide in the TMA made available on the Moodle and

upload the file containing their answers in the specified assignment uploading area of

Moodle.

4.6 Course Evaluation

The Course evaluation was a compulsory component of the project report.

Therefore, a questionnaire containing 18 items was designed covering different

aspects of the course related to design and delivery. This questionnaire was

uploaded on the Qualtrics.com7. The students were asked to provide their feedback

freely. Since there were only 6 students active in the course, the responses were

received only from them. A snapshot of the responses received from the students is


Figure 23: Analysis of the Responses

7 https://qtrial.qualtrics.com/SE/?SID=SV_2bLfEppUMotiAXb

https://qtrial.qualtrics.com/SE/?SID=SV_2bLfEppUMotiAXb

41 | P a g e

4.7 Findings

The feedback received from the students through course evaluation survey

was analysed in the light of the objectives of course evaluation. The findings are

presented as follows:

4.7.1. The objectives drawn for the course were found appropriate by the learners.

4.7.2. Instructional Design of the course was appropriate and was able to

accommodate the learning styles of the learners.

4.7.3. The 'Metacognitive Thinking Skills Scale' helped the learners in

understanding their own Metacognitive Thinking skills.

4.7.4. The instructional components (reading content, audio/video programmes,

blog, quizzes, group discussion forum and virtual classroom) were relevant

and appropriate, and their quality was good. Their presentation style was also

found appropriate by the learners.

4.7.5. The students liked the idea of linking the external resources containing the e-

content of the course to the Moodle.

4.7.6. The video programmes were the first preference of the students followed by

virtual classroom as the second and audio programmes & quizzes as the third

preference.

4.7.7. The students were comfortable ‘to some extent’ (not fully) in use of Moodle

and WizIQ platforms.

4.7.8. The course was able to meet the expectations in case of majority of the

students and they were benefited from the course.

4.7.9. The major problems highlighted by the students were: Moodle took longer

time in opening and internet connectivity was problematic.

4.7.10. The idea of providing metacognitive thinking skills scale and learning styles

index was appreciated by the students. They also liked the idea of providing

a separate section on ‘introduction’.

42 | P a g e

4.7.11. The overall rating for the course as give by the students was Excellent (50%)

and Very Good (50%) (Figure-24).

Figure 24: Overall Rating of the Course by the students

43 | P a g e

Chapter 5: Lessons Learnt and Recommendations

While going through the programme, I experienced different situations which

need mention and sharing these experiences may help the students undergoing this

programme.

5.1 Lessons Learnt

The PG Diploma in eLearning is a programme par-excellence. It maintains its

uniqueness in many ways including the following:

5.1.1 It covers an upcoming area of specialisation (eLearning/online learning) of

open and distance learning. It has its own place even in face to face mode

of education.

5.1.2 It deals with theory of eLearning and other related areas and enables the

learners to put their theoretical knowledge to practice as an inbuilt

requirement of the programme.

5.1.3 The programme covers the whole experience of making of an eTeacher and

provides a learner the opportunity to act as a student, as a system

administrator and then as an eModerator/ eFacilitator/ eTeacher at the same

time during different phases of the programme.

To enroll in this programme as a student and successfully complete is an

amazing experience for anybody without exception and the same is true in my case.

It has been an excellent learning experience for me. The dedication and commitment

of high order is required to complete the programme. I am happy that I have been

able to keep myself composed and focused while undergoing this programme. Of

course it could have not been possible without the incessant support and help

extended by our two Programme Coordinators. While I have been able to sail

through the programme smoothly, I would like to share my experiences of launching

the course on ‘Evaluation of Training’ as under:

44 | P a g e

5.1.4 A student, though enthusiastically, enters into a distance education

programme, however, is not able to sustain the same pace throughout the

programme and with the result either he would become a drop out adding to

the attrition rate or would be an inactive learner slowly meeting the

unsuccessful dead-end. The same is true in my case also. Initially I

registered 10 students for the course with their consent to spare the required

time for the course. However with their behaviour, it took me not long to

realize that my experience with them may fail given the current situation lasts

for long. Therefore, I registered another lot of 6 students who were not so

busy as the earlier ones. I chased them constantly with the timeline for the

course and helped and persuaded them constantly for completing the given

activities within the timeframe.

5.1.5 In order to build a ‘community of practice’ I introduced the GoogleGroup and

requested the students to use it as an informal discussion forum not to be

evaluated for the course. But the students were not able to use this forum to

its capacity probably due to the following reasons:

5.1.5.1 their services for me were voluntary,

5.1.5.2 participation in the group was not compulsory for the course,

and

5.1.5.3 they were busy with their activities and could not participate in

all the activities except the core and compulsory activities for

the course.

I realised that maintaining the interest of the learners in an online

programme and keeping them actively engaged throughout, is a great

challenge.

5.1.6 Lack of tech-savvy nature of the learners comes in the way of their

successfully handling different activities in the eLearning environment. In my

case the students were not comfortable in using the Moodle and WizIQ

platforms. In order to enhance their comfort zone and enable them to at least

use the platforms, I organised an introductory session for them and that really

worked.

45 | P a g e

5.1.7 It requires more alertness and activeness for the learners to handle eLearning

activities. In order to help them do this, they have to be kept busy in one or

the other activity where they should demonstrate not only before the facilitator

but also the peers.

5.1.8 In order to reduce the load on the eLearning platform, alternative resources

for storage of content and then linking (integration) should be used.

5.1.9 The learning style index and metacognitive thinking skills scale aroused the

interest of the learners in the course as a component not seen before by them

which indirectly paid me by sustaining their interest in different activities of the

course.

5.1.10 Since the students were busy in their life, it was difficult to find a common

convenient time for them all for organizing live activities. If they agreed, even

then all of them would not be able to do at the very point in time.

5.2 Recommendations

Having gone through the whole process of design, development,

implementation and evaluation of an online programme, I would like to make the

following recommendations which can make the process enjoyable and smooth:

5.2.1. We should be cautious while enrolling the students for such an experiment.

Only the students who can easily spare time for different activities for this

experiment should be enrolled.

5.2.2. Interest of a student plays an important role in enabling him/her to complete

different activities as required and participate actively in the experiment.

Therefore it should be ensured that the student enrolled is really interested in

carrying out this programme and that it will help him/her in enhancing his/her

skills in some way or the other which would be useful to him/her in life/career.

5.2.3. The pre-requisites for the programme should also be carefully checked with

the desirous students failing which it will be difficult to bring them at par with

the expectation of the programme and extra effort would be required to do so

on the part of the experimenter.

46 | P a g e

5.2.4. The learning style of the learner should be ensured in the beginning and

teaching/learning components should be designed keeping in view these

styles. This would save lot of time of the facilitator which could be utilized in

other constructive activities.

5.2.5. The students should be orientation for the platform they are going to use for

the eLearning programme. This is the time when support of the facilitator is

needed the most. Once the learner gets used to of the environment, he/she

would start exploring many things at his/her own out of interest in the

experience.

5.2.6. A constant follow up on the part of the facilitator is required for the online

learners. Therefore, they should not be left unmonitored and constant

contact/touch should be ensured.

5.2.7. The facilitator should help the learners in forming small peer groups which is

a great help for the novice who are new to the system. It has been seen that

the students are friendlier with their peers than the facilitator and share their

problems with them unhesitatingly.

5.2.8. Though ‘KEEP IT SIMPLE, STUPID’ dictum is best for online learning

programme as far as use of technology is concerned, the design of the

audio/video programmes and their presentation style should be attractive

which will help in maintaining interest of the students in the programme.

5.2.9. Use of scheduler for fixing an online activity e.g. virtual classroom is a noble

idea. If the learners chose time for an activity at their own, it is more likely

that they will ensure attending the activity in spite of all odds.

5.2.10. The online learners should not be put to too much writings for evaluation

purposes, rather they should be made to demonstrate their skills practically.

They should be asked to produce something practically which could be

evaluated on account of their performance in the programme e.g. in case of a

programme on accounting, they can be asked to prepare a small balance

sheet with different components instead of asking them to narrate the

different components of the balance sheet.

5.2.11. Constant and timely feedback acts as lifeline for the online learners. They feel

that somebody is there to help them anytime just at a mouse-click away.

47 | P a g e

5.2.12. Timely technical support also plays an important role in sustaining the interest

of the students in the programme. Therefore, it is must for the

institution/facilitator to ensure timely technical support to the learners. An

honest confession would build confidence of the learners in the

facilitator/institution. Here also the informally formed peer groups can help

the learners among themselves. A student having little bit knowledge of

technical aspects can help other students in his/her group.

5.2.13. The assignment grades of different evaluative activities (quizzes, CMAs,

TMAs, Group Discussion, presentation etc.) should be informed to the

students/uploaded on the environment /platform without loss of much time

since in the absence of this, the students feel restless and get demotivated

feeling that their facilitator is not serious about these activities.

5.3 Conclusion

The Course on ‘Evaluation of Training’ was designed, developed and

implemented by me as part of the requirement of the PGDEL programme. However,

this experiment made me learn the intricacies and complicacies of launching such a

programme. Traversing through this experiment, I developed many skills which would

help me while I would be launching an online programme of any nature. During this

process first I learnt the theory of eLearning and then put it into practice by delivering

an online programme.

48 | P a g e

References

Cassidy, S. (2004). Learning styles: An overview of theories, models and measures.

Educational Psychology, 24(4), 419-444.

Cottrell, S. (2005). Critical thinking skills. New York: Palgrave MacMillan.

Gould, J. (2012). Learning theory and classroom practice in the lifelong learning sector. New Delhi: SAGE.

Hogan, K. & Pressley, M. (1997). Scaffolding student learning: Instructional approaches and issues. Cambridge, MA: Brookline Books.

Honey, P. & Mumford, A.(1986). Using Your Learning Style. Maidenhead: Honey.

Mayer, R. E. (1998). Cognitive, meta-cognitive and motivational aspects of problem solving. Instructional Science, 26(1-2), 49-63.

Merrill, M. D. (1994). Instructional Design Theory. Englewood Cliffs: Educational Technology Publications.

Mishra, S., Ed. (2009). E-Learning. New Delhi: IGNOU (STRIDE Handbook 8).

Oakley, L. (2004). Cognitive Development. Hove: Routledge.

Rheingold, H. (1995). The vitual community. London: Minerva.

Salmon, G. (2000). E-moderating: The key to teaching and Learning online. London: Kogan Page.

Siemens, G. (2005). A learning theory for the digital age. Instructional Technology and Distance Education, 2(1), 3-10.

Solomon, B. A. & Felder, R. M. (n.d.). Inventory of Learning Styles [http://www.engr.ncsu.edu/learningstyles/ilsweb.html]

Tan, L. (1999). The faceless facilitator: An impossible learning approach? Proceedings of online Educa, Berlin.

Tiffin, J. & Rajasingham, L. (1997). In search of the virtual class: Education in an information society. London: Routledge.

Tuncer, M. & Kaysi, F. (2013). The Development of the Metacognitive Thinking Skills Scale, International Journal of Learning & Development, 3(2), 70-76. Doi:10.5296/ijld.v3i2.3449.

http://www.google.com/url?q=http%3A%2F%2Fwww.engr.ncsu.edu%2Flearningstyles%2Filsweb.html&sa=D&sntz=1&usg=AFQjCNHZzCP7RWLcHuag8t9EMJg8Q11DyQ

49 | P a g e

Appendix-1

50 | P a g e

51 | P a g e

52 | P a g e

Appendix-2

53 | P a g e

54 | P a g e

55 | P a g e

56 | P a g e

Appendix-3

Questionnaire Survey for Programme Evaluation of the

Course on ‘Evaluation of Training’

Dear Friends,

This is the Questionnaire meant for evaluation of the Course on 'Evaluation of

Training' which has been designed, developed and launched as part of the fulfilment

of the Project requirements of the PGDEL Programme which I am pursuing with

IGNOU. You have undergone the course as learner and must have experienced

some good and bad moments while undergoing this course. You might have liked

some aspects of the Course while would like to criticize/appreciate some others

given an opportunity. I request you to share your experience and valuable comments

on different aspect of the course. Please respond to this survey sparing 5 minute's

time from your busy schedule. Your feedback will not only help me in building a

public view on the course but also will guide me in improving its delivery.

I wholeheartedly thank you for sparing time and participating in the survey. Cheers!!!

Dr S K Pulist Course Facilitator

Q.1 Please mention your name: ______________________________________ Q.2 The OBJECTIVES drawn for the Course were appropriate.

a) Yes Fully b) Yes to some extent c) Not at all

Q.3 The 'Meta-Cognitive Thinking Skills Scale' helped you in understanding your Meta-Cognitive Thinking Skills.


Q.4 Instructional Design of the Course was APPROPRIATE.

a) Yes b) No

57 | P a g e

Q.5 Instructional Design of the Course was able to ACCOMMODATE your learning style.


Q.6 Please give your feedback on the RELEVANCE of the instructional

components

Very Good

Good Fair Poor Very Poor

Reading content

Audio Programmes

Video Programmes

Blog

Quizzes

Google Group

Virtual Classroom

Q.7 Please give your feedback on the APPROPRIATENESS of the instructional

components

Very Good


Reading content

Audio Programmes

Video Programmes

Blog

Quizzes

Google Group

Virtual Classroom

Q.8 Please give your feedback on the QUALITY of the instructional components

Very Good


Reading content

Audio Programmes

Video Programmes

Blog

Quizzes

Google Group

Virtual Classroom

58 | P a g e

Q.9 You liked the idea of hosting the content on other sites and LINKING it to

Moodle Platform.


Q.10 The presentation STYLE of different instructional components was

Very Good


Reading content

Audio Programmes

Video Programmes

Blog

Quizzes

Google Group

Virtual Classroom

Q.11 Please give your PREFERENCE for different instructional components

______ Reading content

______ Audio Programmes

______ Video Programmes

______ Blog

______ Quizzes

______ Google Group

______ Virtual Classroom Q.12 You were COMFORTABLE in using the 'Moodle' Platform.


Q.13 You were COMFORTABLE in using the ‘WizIQ’ as Virtual Classroom

Platform.


Q.14 The Course was able to meet your EXPECTATIONS.


59 | P a g e

Q.15 You got BENEFTTED from the programme.

a) Yes immensely b) Yes to some extent c) Not at all

Q.16 Please mention the PROBLEMS, if any, faced by you during pursuing this

programme. Q.17 Please give your REMARKS and SUGGESTIONS for improvement of the

programme. Q.18 How would you RATE the Course as a whole?

a) Excellent b) Very good c) Good d) Satisfactory e) Poor f) Very Poor g) Unacceptable

-o-o-o-