Embed Size (px)
Citation preview
Page | 1
INDEX
Lesson No.
Topic Page No
Background: Medical educational Research 1
Objectives of the workshop 3
Contributors and Acknowledgements 5
1 Introduction to Educational Research
6
2 Educational Techniques: Trends, Utility & Effectiveness
18
3 Research Methodology: Outline of Qualitative, Quantitative & Mixed Research Designs & Methods
23
4 Quantitative Methods: Data Collection, Questionnaire Preparation
46
5 Qualitative & Mixed methods in Educational Research
70
6 Focus Group Discussions: Participatory and Non-Participatory Techniques of Qualitative Data Collection in Medical Education
78
7 Descriptive & Inferential Statistics
83
8 Areas of Research in Medical Education & Ethics
123
9 Qualitative Techniques and Computer aided Analysis
127
10 Research Project Proposals Prepared by Participants
133
List of Participants 140
Teaching & Learning Methods for the Revised MBBS Curriculum
142
Page | 2
Report Workshop
Medical Educational Research: Concepts and Methodologies
Background: The basic issue is not whether our students and our medical colleges are
better than those of a generation ago, but whether the quality of today‟s
education is sufficient to meet tomorrow‟s demands, which will be infinitely
more complex than those of the past or the present. In other words, it is not
a matter of whether, through research, we can prove that our medical
institutions are better, but whether, through research and implementation,
we can make them good enough.
Medical Education has been conventionally taught and learnt in an
inductive way and has been considered as difficult to both impart and
imbibe. Already, there have been some major breakthroughs in education
towards accommodation of individual differences, improved learning theory
and practice, better tests and measurements, more effective counselling and
guidance, use of new media, team teaching, evaluation of teacher
effectiveness, and in other areas. Still, many areas have gone with out
investigation and many “good” teachers have been so busy with the twin
problems rising from combined pressures of the explosion of knowledge and
rapid increases in enrolment of medical students.
Basically we need to know a great more deal about how people learn. If we
are to attract and prepare the best possible teachers, we must learn what
kind of person makes a good teacher, what his motivations, attitudes, and
values are likely to be. Stated in more general terms, we need to know how
to maintain quality and enhance it, how to guarantee that the educational
programmes of the future will stimulate the fullest development of every
individual in spite of the raising costs of providing for longer periods of
duration.
At present we have laboratories, Classrooms, and teachers and patients to
initiate research in education. The addition of teaching machines
(computers), multimedia and television, self instructional material etc lead
to experiment the teaching and learning procedures ultimately to find out
Page | 3
the most effective processes. This means we move rapidly and judiciously in
trying out new technologies and constantly evaluating them. By persistent
refinement of the process and tools, we should consistently improve the
product. This calls for research, not guess work.
The total research effort requires adequate planning, prompts
communication and should cover all levels of educational curriculum. The
present medical research has the following setbacks.
1 It has focused too much on what happens to cohort of students and
less on the individual cognitive, emotional and attitudinal changes that
occur during the course and how these affect learning.
2 There were no attempts to be systematic in the efforts to rate the two
styles of learning. In this golden age of evidence-based medicine, where were
the calls for active comparator trials of teaching methods? There are no
trials of the use of placebo teaching, let alone "sham" lectures. Could not the
lecturers/tutors be blinded? What about random allocation of academic low
performers to different teaching methods? As to long-term follow-up, has
anyone given consideration to the comparison of patient satisfaction when
treated by doctors trained by the different styles of teaching?
3 There is no evidence available to either refute or support the major
curricular reforms embarked on by Many Indian Medical Institutions
4 The suggested curricular reform should be carefully researched and
evaluated. A major barrier to this is lack of funding available for such
research despite the possible „consequences for the future of our profession
and our patients'.
Research efforts are of little value unless the educational enterprise is able
to put the findings into practice. Many Medical Institutions in Maharashtra
are devoid of sufficient resources for current programmes and are unable to
implement the recommendations
Page | 4
Topic Objectives
Introduction to Educational Research
1. Sensitize the issue on Medical educational Research 2. Understand the types of Research 3. Delineate the necessity of Documenting the
evidence on effectiveness of techniques
Educational Techniques: trends, Utility & Effectiveness
1. Describe the various techniques of Teaching and learning
2. Understand the limitations of each method 3. Outline the basic learning concepts in adults
Tea Break
Research Methodology: outline of Qualitative & quantitative and Mixed Research Designs & Methods
1. Describe the educational research proposal preparation
2. Enumerate the General research designs adopted for qualitative and quantitative data.
3. Able to outline the Experimental research 4. Understand the importance of quasi-experimental
and single-case designs in educational research
Quantitative Methods: Data Collection, Questionnaire preparation
1. Describe the various categories of data(Nominal, ordinal, interval & Ratio)
2. Understand the sampling procedures used in educational research.
3. Devise a questionnaire for data collection and data quality checks
Lunch Break
Qualitative & Mixed Methods in Education:
1. Understand the types of qualitative research (phenomenology, ethnography, grounded theory and case study)
2. Describe the various qualities (SWOT) of qualitative research
3. Understand the data collection tools and techniques for qualitative data(questionnaire, Interview, Focus group, Observation ).
4. Devise a suitable tool for educational data collection
Tea Break
Focus Group Discussions and Non participatory and Participatory techniques of qualitative data collection in education
1. Describe the merits and demerits of Focus group discussions
2. Understand the method of conducting Focus group and small group Discussions.
3. Understand the method of Participatory Learning Appraisal(PLA)
4. Describe the concepts of non participatory educational techniques like Objective structured examinations/assessments
Descriptive & Inferential Statistics:
1. Understand the concepts in Frequency distribution, Measures of central tendency, measures of variability.
2. Describe the various sampling distributions and Procedures used in educational research ( purposive, opportunistic, critical case)
3. Able to conduct Hypothesis testing. 4. Understand the concept of t-test, Analysis of
Variance and Chi-square tests.
Page | 5
Areas of Research in Medical Education & Methods
1. Understand the areas in Education and Learning requiring research
2. Delineate the Priority areas of educational research in India
3. Describe the Ethical problems involved in Educational Research
Analysis of qualitative data: Use of computers & Software
1. Familiar with the basic operational and data handling techniques with SPSS software
2. Understand the techniques used in qualitative data analysis software
3. Able to analyse data collected through interviews and focus group discussions
Tea Break
Group Work: Preparation of Research Projects in 1)Innovative Teaching techniques 2) Academic Assessment 3) Impact evaluation 4) Economic assessment
Dr Amol Dongre Dr S P Rao Dr Pradeep Borle Dr J V Dixit
Lunch Break
Presentation and Discussion of Research Projects
All Faculty
Tea Break
POST TEST and Concluding Session
Page | 6
Contributors:
1 Amol Dongre M. G. Institute of Medical Sciences, Sevagram
2 Jagannath Dixit Government Medical College, Aurangabad
3 Pradeep Borle BJ Medical College, Pune
4 Surya Prakasa Rao SBH Government Medical College, Dhule
The authors wish to acknowledge the following experts for their invaluable support
Dr Hemant Apte, Anthropological Society on India
Dr Payal Bansal, Maharashtra University of Health Sciences
Dr Avinash Supe, KEM Medical College
Dr Biranjan JR, Government Medical College, Dhule
Dr Patil , ACPM medical College, Dhule
Dr Haribhai Patel, Ahmedabad
The Authors acknowledge the generous financial support rendered by the National
Academy of Medical Sciences, New Delhi
Page | 7
Lesson I: Introduction to Educational Research
Objectives:
1 Sensitize the issue on Medical educational Research
2 Understand the types of Educational Research 3 Delineate the necessity of Documenting the evidence on effectiveness of
techniques
Lesson Outline:
1 Existing Medical Colleges and The Role of Medical Educational Units
2 Scientific Methods: Inductive and Deductive. Qualitative, Quantitative & Mixed Research Designs. Why Educational Research
in Medicine is not taking off. Why to study educational Research: Newer methods of teaching & learning; newer methods of Evaluation/Assessment. Research Wheel
3 Objectives of Educational Research. Enumerate the Various learning
techniques (Adult Learning, Student Autonomy, Computer Assisted
Learning, Web based Learning) and the assessment methods ( OSCE, OSLER, Mini CEX, Case Based Discussion, Portfolio, Mini Source
Feedback, 360 degrees and client satisfaction
By separating teaching from learning, we have teachers who do not listen and students who do not talk"
Based on Palmer P (The Courage to Teach. Jossey Bass, 1998) Remembering the famous Rudyard Kipling‟s five brave men, in medical
education, the Curriculum, Content conveniently classified as Must Know
and Desirable fall into the category of What to teach. In the When to teach
category, identify the subjects to be taught in the pre, para and clinical
years of medical education. In the traditional compartmental approach,
subjects like Anatomy, Physiology and Biochemistry are imparted in the pre
clinical years; Pathology, Microbiology, Pharmacology and Forensic Medicine
in the Para clinical years and the Medicine, Paediatrics, Surgery,
Orthopaedics, Obstetrics & Gynaecology, ENT, Ophthalmology and
Community Medicine in the clinical years. The teaching of the above
subjects will be at the hospital, Out Patient department/ In patient
Page | 8
department, Class rooms, practical halls and in the community form the
Where to teach category. In the How to teach category, there are various
methods of teaching and learning including the Lecture Discussion,
practicals, tutorials, seminars, Problem Based Learning, Small group
teaching, Projects, puzzles and case based learning etc.
The 300 odd medical schools in India are fortunately equipped with medical
education Units (MEU). Thanks to the untiring efforts of Medical Council of
India and their inspectors. Mere existence of these units would not be able
to usher in required sea changes in medical education. Because most
medical schools in India, currently are experiencing difficulties in providing
the right quality and quantity of educational experiences as the curricula
have failed to respond to the needs of the community and country. The
pedagogic shift from traditional approach to a need-based approach requires
a fundamental change of the roles and commitments of educators, planners
and policymakers. Teachers of health professional education are to be well-
informed of the trends and innovations and utilize these to increase
relevance and quality of education to produce competent human resources.
The main functions of Medical Educational unit are as follows
1. The MEU should create a culture of educational research. 2. It should keep the faculty aware of the ongoing research in the field
3. It should generate publications and resources in medical education 4. It should identify and facilitate the teaching learning needs of the
students 5. It should provide instructional design 6. It should focus on newer learning technologies such as simulation and
e-learning 7. It should develop guidelines for student evaluation and curriculum
development
8. It should emphasize Faculty Development
In its document on graduate regulations Medical Council of India
emphasized that Medical Education Units/ Departments be established in
all medical colleges for faculty development and providing learning resource
material to teachers. "The Edinburgh Declaration" of World Federation for
Medical Education (WFME) and "Tomorrow's Doctors" of General Medical
Page | 9
Council (GMC) of UK outlined a number of specific strategies to guide
reforms and bring need-based changes in medical education. The Edinburgh
Declaration, now translated into all major languages, has been very widely
adopted as basis for reform of medical education. Most of the medical
schools in India have traditional, teacher-centred and hospital-based
training with a few exceptions only. Educational innovations and
experiments are not quite evident in India.
Newer methods of Learning and teaching are being introduced in the
west and the Asian countries are eager to globalise these methods. These
include PBL, Student centred teaching, Modular teaching.
………………….etc. Changing learning styles such as application of adult
learning principles, student autonomy, self learning, experimental
learning, reflective learning, computer assisted learning, distance
learning, e- and web based learning, use of skill learning laboratories
Innovative curriculum models such as problem based curriculum,
integrated curriculum, competency based curriculum and hybrid
curriculum
New evaluation methods such as Objective structured long examination
record (OSLAR), Objective structured clinical examination (OSCE) &
Objective structured practical examination (OSPE), Case evaluation
exercise (Mini CEX), Case based discussion (CbD), Portfolio, Multi source
feedback, 360 degrees, Videoing consultation, Patient satisfaction
questionnaire.
Even the assessment methods are revolutionised. The age old systems are
being replaced by objective assessments leading to transparency and
improving the quantity in terms of passed out graduates. However, all these
techniques need rigorous scrutiny and it is necessary to provide evidence of
their superior utility over the traditional methods. Unravelling the truths
about adult learning lead to more specific insights into the student learning.
Medical colleges as basic Educational Institutions should be able to cater
the following basic functions.
1 Administration
Page | 10
2 Clinical Care
3 Research and
4 Education
The function of Education is the foremost and should dominate all other
functions. In practice, medical colleges are obsessed and preoccupied with
functions pertaining to administration and clinical care. The function of
research has taken a back seat. Education reforms and novel educational
techniques are never attempted to be implemented.
Why Research in Medical education?
To this day, tradition and intuition continue to be the prime guiding
principles of education. However, just like medicine, education should be
grounded as much as possible in the best evidence we can find. We
acknowledge the parallel between evidence-based medicine and the
importance of best evidence in medical education. Education should use
research methods that are geared to the idiosyncrasies of the domain of
education and, unlike much of medical research; the education research
neither can nor should always use controlled experimentation as the method
of preference. Such research in education can use both quantitative and
qualitative research methods and combinations thereof.
In recent years, political systems, epidemiological and demographic
patterns, micro-economic strategies, technology, and health care systems
have undergone profound changes. To cope with these changes, educational
institutions around the world have been increasingly confronted with the
challenge of making their curricula more meaningful and relevant to the
needs of the time to produce doctors oriented to the real needs of the
community. Many authorities highlighted the need for reorientation of
medical education and suggested strategies for direction of such changes.
In summary, the reasons for conducting research in medical education are
as follows:
• Become Research Literate • Medicine is a high stake education • Education in age of accountability
Page | 11
• Evidence based approach to education • Improve critical thinking skills
• Read & Critically evaluate published research • Research in medical education is feasible
• Learn to design and conduct research
However, in India, medical education research has not taken off. The apathy
towards this area can be mainly due to the paucity of funds & lack of
resources like absence of Teaching Development Grants, long career in
medical education (almost 10 years to become eligible to teach), not so
attractive option for the clinical scientists ( at present there are only handful
of academicians engaged in educational research), Ivory Tower approach
(feeling and following the age old traditional teaching and learning methods
as the best), Lack of awareness/ apathy towards learning & education
among teaching faculty, medical education research not so attractive for
both basic or clinical scientists and low impact factor for the journals in
medical education.
Impact Factor: It is, devised by Eugene Garfield, a measure of the citations
to science and social science journals. It is frequently used as a proxy for
the relative importance of a journal within its field. Impact factors are
calculated each year for those journals which it indexes, and the factors and
indices are published in the Journal Citation Reports. The impact factors for
various medical journals calculated for 2007 are as follows.
• Medical Education 2.1 • Academic Medicine 1.9
• Medical Teacher .8 • BMJ 7.0 • Lancet 22.0
There are now 18 International Medical Education Journals and the three
major journals in medical education are Medical Education, Academic
Medicine and Medical Teacher.
Is it true that a physician must be a good teacher?
Yes. Doctors have to teach their patients how to get well. They have a
responsibility to teach and educate the members of community how to stay
well. They have additional burden to teach their colleagues all that they
Page | 12
learnt. And if they choose to become medical teacher, they have the
responsibility for medical students.
Although we would never allow a patient to be treated by an untrained
doctor or nurse, we often tolerate professional training being delivered by
untrained teachers. Traditionally students were expected to absorb most of
their medical education by attending timetabled lectures and ward-rounds,
moving rapidly from one subject to the next in a crowded curriculum. Our
junior doctors learnt by watching their seniors in between endless menial
tasks. In recent years the importance of active, self directed learning in
higher education has been recognised. Outcome led structured programmes
for trainees are being developed in the face of reduced working hours for
both the learners and teachers. These all constitute new challenges for
teachers in medicine of all levels of seniority.
Throughout the world there is great interest in developing a set of
qualifications for medical teachers, both at the elementary “teaching the
teacher” level and as part of progressive modular programmes leading to
formal certification. In addition to acquiring new qualifications and
standards, teachers also need access to literature resources that describe
essential components in medical education and supply tips and ideas for
teaching.
What is expected from MBBS doctor now?
Medical Council of India expects that Graduate students to undertake the
responsibilities of a physician of first contact who is capable of looking
after the preventive, promotive, curative & rehabilitative aspect of
medicine. In addition to clinical competencies, students must develop
generic competencies or transferable personal skills essential to their roles
as health professionals, which include bio-ethics and communication skills,
interpersonal skills, problem-solving ability, decision-making capability,
management and organization skills, working in team, IT skills and doctor-
patient relationship.
Why to change the present system of teaching and learning?
The trends in present day medical education are as follows
1 Education for Capability
Page | 13
2 Community Oriented Medical Education
3 Self directed/ Learner centered Learning
4 Problem Based Learning (PBL) and Task Based Learning (TBL)
5 Integration and Early Clinical Contact
6 Continuing Professional Development
7 Unity Between Education and Practice
8 Evidence Based Medical Education/Best Evidence Medical
Education(BEME)
9 Communication and Information Technology
What is the situation of Medical Education Research in India?
More than a dozen peer reviewed journals are available for research in
medical education. However, Indian authors‟ contribution to these journals
is miniscule. Hence it is necessary to inculcate the methodology of
educational research among the faculty of medical educators and promote
the evidence based teaching and learning methods. Until now medical
teachers are confined to the domain of clinical care and devoted less
importance to medical education research
Unfortunately, it has been reported that the majority of published studies
and dissertations on medical education are seriously flawed, containing
analytical and interpretational errors. Some of these flaws have arisen from
ill conceived statistical concepts, inappropriate research methodology both
qualitative and quantitative, deep rooted beliefs of various erroneous
"mythologies" about the nature of research and from a failure,
unwillingness, or even refusal to recognize that analytical and
interpretational techniques that were popular in previous decades no longer
reflect best practices and, moreover, may now be deemed inappropriate,
invalid, or obsolete.
The present understanding in medical education is such that the
standard/traditional methods on instruction/ teaching are ineffective. It has
been emphatically proved that learning among medical students is passive
rather than active. There is now enough evidence to prove that Traditional
Methods do not stimulate critical thinking, Creative thinking and
Page | 14
Collaborative Problem Solving. The explorative knowledge in Cognitive
Psychology proved beyond doubt these facts. The emphasis should be on
Andragogy rather than Pedagogy. It is to be kept in mind that the present
teaching is in an environment of Internet expansion/explosion.
Government of India and Medical Council of India revamped the curriculum
for MBBS course in a draft circulated to all medical colleges in India during
2007. The draft has rightly incorporated the various newer teaching
methods and encouraged to use newer assessment methods. The teaching
methods suggested are as follows:
Teaching Methods: • Lectures • Structured interactive sessions
• Small group discussion a) Demonstrations. b) Tutorials.
c) Seminars. d) Problem Based Learning.
• Focus group discussion (FGD)
• Projects • Participatory learning appraisal (PLA) • Video clips
• Written case scenario • Self learning tools Interactive learning
• e-modules • Skills Labs • Preparation of scientific article
Assessment Methods:
• MCQ • SAQ • OSCE
• OSLER • MiniCEX • Case Based Discussion
• Multi Source Feedback 360 Degrees • Client Satisfaction
It has been observed that medical scientific research is obsessed with
Quantitative methods. The emphasis on Observatory & Explanatory
(Interpretive) studies is minimal. It is time now to adopt Research Approach
exploring Cultures & Subjectivities.
Objectives of Educational Research:
Page | 15
1. Exploration: This is done to generate ideas about something.
2. Description: This is done to describe the characteristics of something
or some phenomenon.
3. Explanation: This is done to show how and why (causality)a
phenomenon operates as it does.
4. Prediction: The advanced sciences make much more accurate
predictions than the newer social and behavioral sciences.
5. Influence: The application of research results to impact the world.
Although we would never allow a patient to be treated by an untrained
doctor or nurse, we often tolerate professional training being delivered by
untrained teachers. Traditionally students were expected to absorb most of
their medical education by attending timetabled lectures and ward-rounds,
moving rapidly from one subject to the next in a crowded curriculum. Our
junior doctors learnt by watching their seniors in between endless menial
tasks. In recent years the importance of active, self directed learning in
higher education has been recognised. Outcome led structured programmes
for trainees are being developed in the face of reduced working hours for
both the learners and teachers. These constitute present new challenges for
teachers in medicine of all levels of seniority.
Types of Research:
1 Basic & Applied Research: • Basic Research: generate fundamental knowledge & theoretical
understanding basic human and other natural processes
eg. Process of cognitive priming • Applied Research: Answer practical questions to provide immediate
solutions eg. Effectiveness of two approaches to counselling
BASIC…………………………MIXED………………………….APPLIED
2 Evaluation Research: • Formative: for programme improvement
• Summative: programme summary judgments & decision to continue It can be further subdivided into the following components
Page | 16
• Needs assessment, which ask this question: Is there a need for this type of program?
• Theory assessment, which asks this question: Is this program
conceptualized in a way that it should work? • Implementation assessment, which asks: Was this program
implemented properly and according to the program plan?
• Impact assessment, which asks: Did this program have an impact on its intended targets?
• Efficiency assessment, which asks: Is this program cost effective? 3 Action Research: It focuses on the solving practitioner‟s local
problem. Through action research, investigators will be able to constantly
observe students for patterns and think about ways to improve instruction,
classroom management etc.
4 Orientational Research: It is mainly based on the critical theory. The
focus is on some form of inequality, discrimination, or stratification in
society. Some areas in which inequality manifests itself are large differences
in income, wealth, access to high quality education, power, and occupation.
A good researcher‟s basic quality is the ability to reason. There are two kinds
of reasoning namely deductive and inductive.
Deductive reasoning (i.e., the process of drawing a specific conclusion from
a set of premises). In this approach of formal logic, a conclusion from
deductive reasoning will necessarily be true if the argument form is valid
and if the premises are true.
Inductive reasoning (i.e., reasoning from the particular to the general). The
conclusion from inductive reasoning is probabilistic. It is based on the
assumption that the future might not resemble the present.
Common Assumptions in Medical Education Research: • World out there that can be studied
• World is unique. Some of it is regular and predictable. But most of it is Dynamic and complex
• Researchers can examine/ study the unique, regular and complex world
• Researchers follow agreed norms/ practices
• It is possible to distinguish between good & poor research • Science can not provide answers to All questions.
Page | 17
Scientific Methods: There are many scientific methods. The two major
methods are the inductive method and the deductive method.
• The deductive method( Quantitative technique) involves the following
three steps:
1. State the hypothesis (based on theory or research literature).
2. Collect data to test the hypothesis.
3. Make decision to accept or reject the hypothesis.
• The inductive method(Qualitative technique) also involves three steps:
1. Observe the world.
2. Search for a pattern in what is observed.
3. Make a generalization about what is occurring.
Diagramatically, these two methods are represented as below.
Qualitative Vs Quantitative Design*: *= Based on Assumption that people have meaningful experiences that can be interpreted
Qualitative Research
Collect
More Data Tighter
Specification of Question
Page | 18
Quantitative Research: Based on Assumption that Random Events are
Predictable. Any application of science includes the use of both the
deductive and the inductive approaches to the scientific method either in a
single study or over time. The inductive method is as “bottom up” method
that is especially useful for generating theories and hypotheses; the
deductive method is a “top down” method that is especially useful for testing
theories and hypotheses. This is called “Research Wheel”.
Page | 19
Lesson II: Educational Techniques: trends, Utility & Effectiveness
Objectives:
1 Describe the various techniques of Teaching and learning 2 Understand the limitations of each method 3 Outline the basic learning concepts in adults
Lesson Outline:
1 The traditional methods of Teaching & Learning. Newer Methods of Learning: Self Directed Learning; Problem
Based Learning; AV Media; Role Play; Focussed Discussions; Group work. e-Modules; skill labs; Participatory Learning Appraisal (PLA);case Studies; Algorithms; workshops; Projects; seminars; Portfolio based; Virtual classrooms etc.
2 Eight Principles of Adult Learning: Characterstics of Adult Learners. 3 Techniques: Teacher oriented, Interactive and Independent Techniques of
teaching & Learning.
Learning: Learning is a process which results in a relatively permanent
change in the behaviour of the learner. This change can be in the way of
thinking, feeling or doing and is reflected in the acquisition of knowledge
and skills and the development of attitudes by the learner. Learning is an
outcome of one‟s interactive experience with the environment. Learning is an
active and continuous process
Theories of Learning:
Conditioning theories- these explain learning process in terms of stimuli and responses
Theories of connectionism- these explain learning in terms of the
formation and strengthening of bonds or neural connections between stimuli and responses (law of readiness, exercise, effect and
belongingness) Field theory Learning models- Ausbel‟s advance organizer model, inquiry training
model Levels of Learning:
Signal learning Stimulus response learning Chaining
Verbal chaining
Page | 20
Discrimination learning Concept learning
Rule learning Problem solving
Principles of Learning: Goal setting
Relevance of learning experience Motivation Personal nature of learning
Active involvement of learners Meaning orientation
Application of knowledge Realistic learning Facilitative instructional sequence
Feed back Teaching: It is a process which facilitates learning by encouraging learners
to think, feel and do. The learning experience results in the acquisition of
knowledge and skills and development of attitudes. The role of a Teacher
can be identified as a Manager, Communicator, Self-learner, Research
worker and a Role model
Adult Learning Principles: Adult learning is a process whereby persons,
whose major social roles are characteristic of adult status, undertake
systematic and sustained learning activities to acquire desirable changes in
knowledge, attitudes, values or skills. It involves a complex interaction
among psychological, personal, social and environmental factors that
influence how an adult participates and learns. When new facts, ideas or
concepts are presented to adults; They think dialectically and contextually;
look for embedded logic; apply working intelligence and common sense and,
form opinions and judgments based on their existing cognitive framework
Critical reflection is a characteristic of adult learning
How to incorporate adult Learning Principles: Training programmes should be relevant Adults need high levels of motivation
Adults need high levels of involvement Adults need a variety of experiences Adults have personal concerns
Adults need positive feedback Adults have plenty of past experiences
Adults have variable educational orientation A good lecturer is a text book plus personality…..
Page | 21
- Flexner But, all too often, the personality is missing and the lecture becomes
“a process by which information is transferred from the notes of the lecturer to the notes of the student without going through the minds
of either!” - Sir Joseph Bancroft
Exercise I
Enlist the merits and demerits of lecture as a method of teaching and
learning
Classification of teaching-learning experiences Control based-classification: This classification divides T-L methods according to the person(s)
controlling the activity. – Teacher-controlled T-L activities: Lecture, symposium, team
teaching, demonstration, bedside clinics, etc. – Learner-Controlled activities : Free-group discussion, project
work and self-learning methods like self-study, programmed
instruction etc. Group size based classification: This classification is more useful to a teacher to plan according to the
student strength. – Large group methods: Can take care of any number of students.
Example : Lecture, panel discussion, symposium. – Small group methods: Are useful for up to 30 learners.
Example: Group discussion, seminar, workshop, bedside
clinics, demonstration, field visit. – Individual T-L methods: Attend to one student only and permit
individual learning.
– Example : Counseling, project work, assignment, computer assisted learning and self-study.
“We all know that 50% of what we teach to our students is right. But no one knows which 50%.” Various Methods:
Symposium: It is a series of prepared talks given by a few experts (2 to 5) on many aspects of a topic or problem under a chair person
The talk should be short and to the point There is no discussion among speakers Audience is passive unless question/reaction time is allowed
Merits: Symposium: It is a series of prepared talks given by a few experts (2 to
5) on many aspects of a topic or problem under a chair person
The talk should be short and to the point There is no discussion among speakers
Audience is passive unless question/reaction time is allowed Demerits
Formal atmosphere
Page | 22
Passive audience
The Panel A group of four or more persons sit with a moderator in front of an
audience; they hold an orderly and logical conversation on an assigned topic
Each member makes an opening remark for 3 to 5 minutes before
exchanging ideas Each member has a special knowledge or holds a particular view of
the topic
Merits Identifies and explores a problem or issue from many angles
Audience can understand various aspects of the issue Frequent change of speaker and view point maintains attention and
interest of audience
Establishes informal contact with the audience Demerits
Panelists may not cover all aspects of the problem and may over emphasize only certain aspect
Skilled moderator is necessary to ensure logical and balanced
coverage by the panel Audience is passive unless some question time is permitted
Team Teaching It has evolved since late 50s‟ with the objectives of improving the
quality of teaching by utilizing better talents and skills of a team of teachers. The team may act in four styles:
– Relay style of team teaching
– Team teaching in the same period (like a symposium) – Ability based team teaching – Specialization based team teaching
Small Group Methods:
Group discussion Seminar Tutorial
Demonstration Practical/bedside teaching/field work
Role play Workshop
Varieties of Group Discussion Controlled discussion Free group discussion
Buzz group Brain storming
Syndicate T-group
Page | 23
Individual Methods Reading
Programmed learning Project
Individual assignment Conference Counseling
Simulation Always Remember This Span of concentration ---- 7 minutes
Span of attention ----------- 57 seconds 20% is FORGOTTEN in 3 days.
70% is FORGOTTEN in 7 days Exercise II
Select appropriate T-L method – Convince a woman to use copper T
– Inform about side effects of a medicine to patients – Educate about drug policy of India – How to inject BCG vaccine
– Preparing a patient for operation – Autoclaving – Vaccination campaign
A teacher can never truly teach, unless he is still learning himself.
RABINDRANATH TAGORE
Page | 24
Lesson III:
Research Methodology: Outline of Qualitative, Quantitative & Mixed Research
Designs & Methods
Objectives:
1. Describe the educational research proposal preparation 2. Enumerate the General research designs adopted for qualitative and
quantitative data. 3. Able to outline the Experimental research 4. Understand the importance of quasi-experimental and single-case designs in
educational research
Lesson Outline:
1 Learning to select a research topic and preparing a research proposal. Sources of research ideas. Review of Literature. Literature search through web. Statement of Research Problem and purpose. Framing of Research Questions. Hypothesis. Research Proposal.
2 Observatory and Experimental and quasi experimental designs. Experimental Approach. Independent & Dependent Variables. Controlling Confounding Variables. Random assignment. Matching. Counter balancing. Post-test and Pre-test design. Pretest-posttest control-group design. Factorial design. Repeated measures design.
3 Validity of tools and designs. Internal validity and threats to internal validity. External Validity. Methods to improve validity.
4 Quasi-experimental and single case designs. Non-equivalent Comparison Group Design. Interrupted Time Series Design. Regression Discontinuity Design. Single –Case experimental Design and Multiple Baseline Design.
Educational Research in general, and medical education in particular, are
not research disciplines per se, with their own specialised theories and
methodologies; rather, they are fields of inquiry of potential interest in
multiple disciplines involving Psychology, anthropology, statistics and
epidemiology. In medical education, research is conducted to study links
between teaching factors and learning outcomes. However, presently,
thousands of studies that have been published in medical education are
best characterized as a diverse collection of bits and pieces with no unifying
mechanism of inquiry. The advent of statistical methods has acquired
Page | 25
credibility to experiments in education. Pearson (chi-squared goodness-of-fit
distribution in 1900), Gosset‟s (Student-t in 1908), Fisher‟s (significance
testing in 1925), Neyman and Person‟s (null-hypothesis testing in 1933) and
Yates(magnitude of the effects in 1951) are significant contributors to the
advancement of statistics in medical education.
The Core purpose of experimental paradigms is the establishment of
causality. The panacea of experimental research is Randomised Controlled
Trial (RCT) is based on the Mill‟s Method of Difference. It asserts that if two
situations differ in only one respect and an effect is observed in one
situation but not the other, then, it can be concluded that the effect was due
to the factor that is different. The laudable goal of medical education is Best
Evidence Medical Education (BEME).
There are currently three major research paradigms (a perspective based on
a set of assumptions, concepts, and values that are held by researchers) in
education. They are quantitative research, qualitative research, and mixed
research.
1. Quantitative research – research that relies primarily on the collection
of quantitative data.
2. Qualitative research – research that relies on the collection of
qualitative data.
3. Mixed research – research that involves the mixing of quantitative and
qualitative methods or paradigm characteristics.
The differences between these methods are depicted as below.
Page | 26
Characteristics of Qualitative, Quantitative & Mixed Research
Quantitative Mixed Qualitative
Scientific Method
Deductive ”Top Down”
Deductive & Inductive
Inductive “Bottom-up”
Human Behaviour view
Regular, Predictable
Somewhat predictable
Fluid, dynamic, situational, contextual,
personal
Objectives Describe, explain, predict
Multiple Describe, Explore, discovery
Focus Narrow-angle, specific hypothesis
testing
Multi-lens focus
Wide-angle & Deep angle
Observation Study behaviour under control
Behaviour in more than one
context
Behaviour in natural
Nature of reality
Objective Realism & Pragmatic
Subjective, Personal
Data Collection
Quantitative data Multiple forms Qualitative data
Nature of data
Variables Mixture of variables,
words, images
Words, images, categories
Data Analysis Statistical relationship
Quantitative & qualitative
Search patterns, themes, holistic features
Results Generalizable Corroborated findings may
generalize
Particular finding multiple perspectives
Final Report form
Statistical Report Assorted & Realistic
Narrative, Direct quotations from
participants
Quantitative Research Methods: The basic building blocks of quantitative
research are variables. Variables (something that takes on different values
or categories) are the opposite of constants (something that cannot vary).
The types of variables are classified according to the measurement and the
role they play.
Variable Type Key Characteristics Example
Level of Measurement Categorical Non-quantitative
measurement scale used to categorize, label, classify, name, or identify variables. It classifies groups or
Place of birth, college name, personality type, gender (male, female).
Page | 27
types.
Ordinal This enables to make ordinal judgments (i.e., judgments about rank order). The distance between the levels may not be equal.
Rank in the class, Grades, Malnutrition grade, Socio-economic class
Interval This has the characteristics of rank order and equal intervals (i.e., the distance between adjacent points is the same). It does not possess an absolute zero point.
Intelligent Quotient, Fahrenheit temperature
Ratio This is a scale with a true zero point. It also has equal intervals , rank order, and ability to mark a value with a name.
number correct, weight, height, response time, Kelvin temperature, and annual income.
Role of Variable Independent Variable (IV) That is presumed to cause
changes to occur in Duration of study hours
Dependent Variable (DV)
The one which changes because of another variable OR the Outcome( output, effect and impact)
Test grades OR Assessment marks
Mediating (Intervening) Variable
It accounts the relation between the predictor and the criterion. Mediators explain how external physical events take on internal psychological significance
Memory OR Intelligent quotient
Moderator Variable It affects the direction and/or strength of the relation between an independent or predictor variable and a dependent or criterion variable
Number of books referred OR Number of times notes has been prepared
Predictor Variable It is used in regression to predict another variable
Intelligent Quotient
Extraneous variable compete with Independent variable in explaining the outcome
A particular school/ college and the higher grades obtained
Confounding variable An extraneous variable Which is the real reason for an outcome
Excellent teaching faculty in a school OR Innovative teaching methods responsible for higher grades
Page | 28
Types of Quantitative Research: 1 Experimental: The purpose of experimental research is to study
cause and effect relationships. It is characterised by active manipulation of
an independent variable and random assignment (which creates "equivalent"
groups).
Pre Test Treatment Post Test O1 Xe O2
O1 Xc O2 Where
E stands for the experimental group (e.g., new teaching approach)
C stands for the control or comparison group (e.g., the old or standard teaching approach)
2 Non-Experimental: In this approach, there is no manipulation of the
independent variable and there is no random assignment of participants to
groups. In this study, even though there exists a relationship between IV
and DV, the causal relationship can not be concluded because there will be
too many other alternative explanations for the relationship. There are two
basic strategies in this non-experimental approach. In the "basic case" of
causal-comparative research, there is one categorical IV and one
quantitative DV. Example: Gender (IV) and class performance (DV) where
the gender relationship is compared among male and female with the
performance levels. In the simple case of correlational research, there is
one quantitative IV and one quantitative DV. Example: Self-esteem (IV) and
class performance (DV). It can be concluded that stronger evidence for
causality can be obtained through experimental research than from non-
experimental research.
Selecting the right measures:
One of the most important components of a good research protocol is the
selection of the best outcome measures. However, these measurements are
Page | 29
difficult. There are often no pre-existing measures that can validate the
desired outcomes of a specific study. Often, researchers select the measures
on the basis of their ease of creation, their ease of administration, their
capacity to generate numbers, their perceived reliability, or their mere
existence. The outcome measures can be defined at abstract level but, it is
not clear whether these terms amount to in observable behaviours or
activities. Therefore, creating a valid and reliable measure is vital to the
interpretation of experimental results.
Critical & Practical issues:
1 Control group: Majority of the experimental studies require a control
group for comparison and relative effectiveness of the intervention.
Offering no intervention to the control group will result in a poor,
weak design of the study. At abstract level, selecting a control group
and the type of intervention/ no intervention seems to be relatively
simple. At the level of detail, it can get complicated quickly and
generate conundrums that are not easy to solve.
2 Motivational Factors: It plays an important role in improvement of
intervention, compliance and adherence to no intervention
3 Placebo Effect: Merely the belief that an intervention offered is
sufficient to produce some kind of improvement. It might be due to
the repeated follow-up, contact with researchers and enhanced
attention
4 Selection bias: Specific selection of volunteers for the intervention or
control group would result in a bias limiting the generalizability of the
study
Conditions for Causation:
There are three necessary conditions that must be established to conclude
that a relationship is causal.
1. condition: Variable A & Variable B must be related (Relationship)
2. condition: Temporal antecedence (time order) 3. Condition: Relationship between A & B not due to confounding/ third
Variable (Lack of alternative explanation)
Page | 30
A. Experimental Approach Research Methods:
Is a situation in which a researcher objectively observes phenomena which
are made to occur in a strictly controlled situation where one or more
variables are varied and the others are kept constant. The independent
variable is the variable that is assumed to be the cause of the effect. It is the
variable that the researcher varies or manipulates in a specific way in order
to learn its impact on the outcome variable.
Independent Variable Manipulation:
One of the important aspect of experimental research is the manipulation of
the independent variable by the researcher.
A. Presence versus Absence Technique: the independent variable can be
manipulated by presenting a condition or treatment to one group of
individuals and withholding the condition or treatment from another
group of individuals
B. Gradation/Amount Technique: the independent variable can be
manipulated by varying the amount of a condition or variable such as
increasing the number of training sessions or varying the amount of a
drug which is given to children with a learning disorder.
C. Type/ Modality Technique: independent variable is to vary the type of the
condition or treatment administered. One type of drug may be
administered to one group of learning disabled children and another type
of drug may be administered to another group of learning disabled
children
Confounding Variable Control:
Researcher should keep in mind the effect of confounding variables and the
research design should be able to eliminate the effect of known confounding
variables. The following methods can be employed to reduce the effect of
confounding variables.
1 Random Assignment / Random Selection: Random assignment
makes the groups similar on all variables at the start of the experiment. If
random assignment is successful, the groups will be mirror images of each
other.
Page | 31
Random Selection Random assignment
Purpose To generate a sample that represents a larger population.
To take a sample (usually a convenience sample) and use the process of randomization to divide it into two or more groups that represent each other. It is used to create probabilistically “equivalent” groups.
Uses It helps to ensure external validity
It helps to ensure internal validity. It also eliminates the problem of differential influence in the groups.
Importance in Experimental Research
Less important in educational research/ qualitative research
more important than random selection
2 Matching Variables: It controls for confounding extraneous variables
by equating the comparison groups on one or more variables that are
correlated with the dependent variable. Decide what extraneous variables
are to be matched (i.e.., decide what specific variables can be matched to
make the groups similar on). These decided variables are called the
matching variables. Matching eliminates any differential influence of the
matching variables. Matching can be carried out on one or more extraneous
variables.
3 Holding Extraneous variables constant: This technique controls for
confounding extraneous variables by insuring that the participants in the
different treatment groups have the same amount or type on a variable. Eg.
To study the influence of gender, either select only men or women but not
both. A problem with this technique is that it can seriously limit study‟s
ability to generalize the results.
4 Building Extraneous Variables into Design: This technique takes a
confounding extraneous variable and makes it an additional independent
variable in research study. E.g. Both Male and females can be studied
incorporating gender into the study design.
5 Counter Balancing for Sequencing Effects(Order/Carry-over) Priming
effect: It is a technique used to control for sequencing effects (eg. order
effects and carry-over effects). However, this technique is only relevant for a
design in which the participants receive more than one treatment condition
(e.g., such as the repeated measures design). Sequencing effects are biasing
Page | 32
effects that can occur when each participant must participate in each
experimental treatment condition. Eg. In a drug trial, when the same person
is used as intervention and control in sequence, there can be residual effect
of the drug. Hence, wash out period is allowed. Same can be applied to
educational research. It also point out that whether control should be tested
first or the intervention.
• Order effects arise from the order in which the treatments are
administered. For example, as people complete their participation in their
first treatment condition they will become more familiar with the setting and
testing process. When these people participate, later, in their second
treatment condition, they may perform better simply because are now
familiar with the setting and testing that they acquired earlier. Order effects
should be controlled.
• Carry-over effects occur when the effect of one treatment condition carries
over to a second treatment condition. That is, participants‟ performance in a
later treatment is different because of the treatment that occurred prior to it.
When this occurs the responses in subsequent treatment conditions are a
function of the present treatment condition as well as any lingering effect of
the prior treatment condition.
Counterbalancing is a control technique that can be used to control for
order effects and carry-over effects. This can be achieved by administering
each experimental treatment condition to all groups of participants, but
applying it in different orders for different groups of people. For example if
two groups making up the independent variable counterbalance can be
carried out by dividing the sample into two groups and giving this order to
the first group (treatment one followed by treatment two) and giving this
order to the second group (treatment two followed by treatment one).
6 Analysis of Co-variance: It is a statistical control technique that is
used to statistically equate groups that differ on a pretest or some other
variable. For example, in a learning research study intelligence level has to
be controlled, if there are more brighter students in one of two comparison
groups, then the difference between the groups might be because the groups
differ on IQ rather than the treatment variable. Analysis of covariance
Page | 33
statistically adjusts the dependent variable scores for the differences that
exist on an extraneous variable.
Types of Designs:
A research design is the outline, plan, or strategy that is going to be used to
obtain an answer to research question. Research designs can be weak or
strong or quasi which are moderately strong, depending on the extent to
which they control for the influence of confounding variables
A. Weak Designs: considered weak because they do not control for the
influence of many confounding variables.
1 The one-group posttest-only design: It is a very weak research
design where one group of research participants receives an
experimental treatment and is then post tested on the dependent
variable. A serious problem with this design is that it is not known
whether the treatment condition had any effect on the participants.
There is no pre-test or control group for comparison. Another problem
is that there can be confounding bias.
2 One-group pretest-posttest design: It is a design where one
group of participants is pretested on the dependent variable and then
posttested after the treatment condition has been administered. This
is a better design than the one-group posttest-only design because it
at least includes a pretest, that indicates how the participants did
prior to administration of the treatment condition. In this design, the
effect is taken to be the difference between the pretest and posttest
scores. It does not control for potentially confounding extraneous
variables such as history, maturation, testing, instrumentation, and
regression artifacts, so it is still difficult to identify the effect of the
treatment condition.
3 Posttest-only design with non-equivalent groups: It includes an
experimental group that receives the treatment condition and a
control group that does not receive the treatment condition or receives
some standard condition and both groups are posttested on the
dependent variable. While this design includes a control group, the
participants are not randomly assigned to the groups so there is little
Page | 34
assurance that the two groups are equated on any potentially
confounding variables prior to the administration of the treatment
condition. Because the participants were not randomly assigned to the
comparison groups, this design does not control for differential
selection, differential attrition, and the various additive and
interaction effects
1) Post Test Only Design
2) One Group-Pretest-Post test Design
3) Post test only with nonequivalent groups
B. Strong Designs:
A research design is considered to be a "strong research design" if it controls
for the influence of confounding extraneous variables. This is accomplished
by random assignment and presence of a control group (which is the
comparison group that either does not receive the experimental treatment
condition or receives some standard treatment condition).
Page | 35
1 Pretest-posttest control-group design: This design is one of the
most common and appealing design for educational interventions. In this
design group of research participants is randomly assigned to an
experimental and control group. Both groups of participants are pre tested
on the dependent variable and then post tested after the experimental
treatment condition has been administered to the experimental group. This
design controls for all of the standard threats to internal validity. Differential
attrition may or may not be a problem depending on what happens during
the conduct of the experiment. In this design, instead of two groups, it can
be expanded to include more than one experimental groups. This design
provides an opportunity to confirm effectiveness of randomised allocation
through direct visual inspection of the pre-test scores. This will ensure that
the experimental and control groups are identical/ relatively close on the
dependent variable (no/minimal sampling error). From a practical point of
view, sometimes the use of pre-test is impractical especially in medical
educational research. In such instances, both the experimental and control
groups are assumed to behave the same way at the beginning/ pre-test.
This is also called as Floor effect. Infrequently, the use of pre-test can limit
the generalizability of the conclusions of the study. Hence, it can be
concluded that Pre-test-Post-test Control group design is more powerful
statistically and more compelling as a demonstration of improvement. It is
somewhat limited in its generalizability and its feasibility.
2 Posttest-only control-group design: In this design, research
participants are randomly assigned to an experimental and control group
and then post tested on the dependent variable after the experimental group
Page | 36
has received the experimental treatment condition. This design includes a
control or comparison group and has random assignment. It also controls
for all of the standard threats to internal validity. Differential attrition may
or may not be a problem depending on what happens during the conduct of
the experiment. This design does not include a pretest of the dependent
variable. This design offers the advantage of treatment group not being
primed by the pre-test. A clean control group (no treatment has been
offered) scores can be interpreted as the pre-test scores for the treatment
group. However, a clean control group is inappropriate and unethical.
Offering no treatment to control group would be unfair and de-motivating
leading to low scores (under estimate of their true naïve potential). The
improvement in the treatment group can also be contributed by the placebo
effect (over estimate). The statistical power is lost in this post test only
design. Finally this design is easier to administer and has more true to life
condition.
3.The Solomon Four Group Design: In this complicated design, named
after Solomon(1946), all the four groups from Pre-test/Post-test and Post-
test only designs.
Group I: Pre-test Treatment. Pre-Test……Intervention……Post-test
Group II: Pre-test control Pre-test………………………….Post-test
Group III: Post-test only treatment ……………..Intervention…….Post-test
Group IV: Post-test only control …………………………………..Post-test
Page | 37
This design also has clean control group. The advantages of both pre-
test/post-test and post-test only designs can be found in this design. The
design also allows further comparisons between the two designs. Baseline
scores are available for comparisons. However, the disadvantages include
limited number of subjects availability and because of the noise of sampling
error it will be statistically difficult to demonstrate the effect of treatment.
Practically, this design is more complicated to enact and may lead to
decrease in statistical power due to lesser number of subjects in each group.
4 Factorial design: In this design two or more independent variables
are simultaneously investigated to determine the independent and
interactive influence which they have on the dependent variable. It also has
random assignment to the groups. Each combination of independent
variables is called a "cell." Research participants are randomly assigned to
as many groups are there are cells of the factorial design if both of the
independent variables can be manipulated. The participants are
administered the combination of independent variables that corresponds to
the cell to which they have been assigned and then they respond to the
dependent variable. The data collected from this research give information
on the effect of each independent variable separately and the interaction
between the independent variables. The effect of each independent variable
on the dependent variable is called a main effect. There are as many main
effects in a factorial design as there are independent variables. If a research
design included the independent variables of gender and type of instruction,
then there would potentially be two main effects, one for gender and one for
type of instruction.
An interaction effect between two or more independent variables occurs
when the effect which one independent variable has on the dependent
variable depends on the level of the other independent variable. For
example, if gender is one independent variable and method of teaching
physiology is another independent variable, an interaction would exist if the
lecture method was more effective for teaching males physiology and
individualized instruction was more effective in teaching females physiology.
Page | 38
5. Repeated-measures design: In this design all research participants
receive all experimental treatment conditions. For example, in case of effect
of type of instruction on learning physiology and two types of instructions
(lecture method and individualized instruction) were used, the participants
would experience both types of instruction, first one and then the other.
This design has the advantage of requiring fewer participants than other
designs because the same participants participate in all experimental
conditions. It also has the advantage of the participants in the various
experimental groups being equated because they are the same participants
in all of the treatment conditions. If counterbalancing is used with this
design, then all of the standard threats to internal validity are controlled for.
Differential attrition may or may not be a problem depending on what
happens during the conduct of the experiment.
Page | 39
6. factorial design based on a mixed model: This is based on a mixed
model is a factorial design in which different participants are randomly
assigned to the different levels of one independent variable but all
participants take all levels of another independent variable. All of the
standard threats to internal validity are controlled for with this design if
counterbalancing is used for the repeated measures independent variable.
Differential attrition may or may not be a problem depending on what
happens during the conduct of the experiment.
Quasi-Experimental Research Designs
The Quasi-experimental research designs are used when it is not possible to
control for all potentially confounding variables; when it is impossible to
randomly assign participants to comparison groups and when a researcher
is faced with a situation where only one or two participants can participate
in the research study (single case designs). These designs have manipulation
of the independent variable but are not able to satisfy the criterion of
random assignment to two or more groups. Causal explanations can be
Page | 40
made only when data collected demonstrate that plausible rival explanations
are unlikely, and the evidence will still not be as strong as with one of the
strong designs. Quasi-experiments fall in the center of a continuum with
weak experimental designs on the far left side and strong experimental
designs on the far right side.
A Non-equivalent Comparison-Group Design
This is a design also called as Cohort Design contains a treatment group
and a non-equivalent untreated comparison group about of which are
administered pre-test and post-test measures. The groups are “non-
equivalent” because there is no random assignment leading to no assurance
that the groups are highly are similar. Hence, confounding variables (rather
than the independent variable) may explain any difference observed between
the experimental and control groups. It is a good idea to collect data that
can be used to demonstrate that key confounding variables are not the
cause of the obtained results. Also it is advisable to use statistical control
techniques for confounding variables. The most common threat to the
internal validity of this type of design is differential selection. The drawbacks
of this design are Systematic differences, maturation effects, and self
selection bias. One solution to thee concerns is to use pre-test/post-test
design. Another solution is to repeat the comparison on several groups or to
use stratification of groups. The problem is that the groups may be different
on many variables that are also related to the dependent variable (e.g., age,
gender, IQ, reading ability, attitude, etc.).
The primary threats to this design are described as below.
Selection Bias: Non-equivalent groups in this design, leads to differential selection of experimental and control groups.
Selection Maturation: One group of research study participants might be
Page | 41
more experienced/ tired or bored than participants in the other group. Selection-Instrumentation: Instrument/measurement may vary among non-
equivalent groups Selection-Regression: Non-equivalent groups might lead to one group with
high reading scores and the other low resulting in regression to mean Selection-history: The groups might differ in their past experiences.
B Volunteer treatment Design:
Volunteers are recruited in this design and those who choose to participate
receive the training whereas those who choose not to participate in the
intervention act as the control group. These designs overcome the ethical
issues of with holding the treatment in a situation where the outcome
measure is important to the participants. However, if there is a difference
between the treatment group and control groups, it is difficult to attribute
the difference to the intervention because of systematic difference and
simple selection bias. This can be overcome by selecting control as those
who are completely not compliant with the intervention(due to unavoidable
reasons) among those who have agreed to participate.
C Interrupted Time-Series Design
This is a design in which a treatment condition is accessed by comparing
the pattern of pretest responses with the pattern of posttest responses
obtained from a single group of participants. In other words, the
participants are pretested a number of times ie. Baseline data and then
Volunteers
Agreed to participate Refused to participate
Completely compliant with intervention
Non compliant due to emergency
Intervention
Control
No Intervention (Contr
ol)
Page | 42
posttested a number of times after or during exposure to the treatment
condition. A treatment effect is demonstrated only if the pattern of
posttreatment responses differs from the pattern of pretreatment responses.
That is, the treatment effect is demonstrated by a discontinuity in the
pattern of pretreatment and posttreatment responses. Many confounding
variables are ruled out in this design because they are present in both the
pretreatment and posttreatment responses. However, the main potentially
confounding variable that cannot be ruled out is a history effect. The history
threat is a plausible rival explanation if some event other than the treatment
co-occurs with the onset of the treatment.
C. Single-Case Experimental Designs
These are weakest quasi-experimental designs where the researcher
attempts to demonstrate an experimental treatment effect using single
participants, one at a time. These designs are also called as Pre-
experimental design. One of the plausible explanation of improvement in
group might be due to the pre-test which itself was sufficient to generate
improvement. Another defect is that unstructured time gap between the pre-
test and the post-test. It is possible that, having been informed of their areas
of weakness with the pre-test, the participants were able to independently
obtain the information necessary to perform better on the post-test, even if
the formal intervention is not effective at all.
1 A-B-A and A-B-A-B Designs
The A-B-A design is a design in which the participant is repeatedly pretested
(the first A phase or baseline condition), then the experimental treatment
condition is administered and the participant is repeatedly posttested (the B
phase or treatment phase). Following the posttesting stage, the pretreatment
conditions are reinstated and the participant is again repeatedly tested on
Page | 43
the dependent variable (the second A phase or the return to baseline
condition).
Baseline (A) Post Test (B) Baseline (A)
The effect of the experimental treatment is demonstrated if the pattern of the
pre- and posttreatment responses ( the first A phase and the B phase) differ
and the pattern of responses reverts back to the original pretreatment level
when the pretreatment conditions are reinstated (the second A or return to
baseline phase). Including the second A phase controls for the potential rival
hypothesis of history that is a problem in a basic time series design (i.e., in
an A-B design).
One limitation of the A-B-A design is that it ends with baseline condition or
the withdrawal of the treatment condition so the participant does not receive
the benefit of the treatment condition at the end of the experiment. This
limitation can be overcome by including a fourth phase which adds a second
administration of the treatment condition so the design becomes an A-B-A-B
design. A limitation of both the A-B-A and the A-B-A-B designs is that they
are dependent on the pattern of responses reverting to baseline conditions
when the experimental treatment condition is withdrawn. This may not
occur if the experimental treatment is so powerful that its effect continues
even when the treatment is withdrawn.
Baseline (A) Post Test (B) Baseline (A) Post Test (B)
O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O11 O12 O12
Treatment/ Intervention
Remove Intervention
O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O11 O12 O13 O14 O15 O16
Intervention Remove Intervention
Intervention
Page | 44
2. Multiple-Baseline Design
This is a design that investigates two or more people, behaviors, or settings
to identify the effect of an experimental treatment. The key is that the
treatment condition is successively administered to the different people,
behaviors, or settings. The experimental treatment effect is demonstrated if
a change in response occurs when the treatment is administered to each
person, behavior, or setting. Rival hypotheses are unlikely to account for the
changes in the behavior if the behavior change only occurs after the
treatment effect is administered to each successive person, behavior, or
setting. This design avoids the problem of failure to revert to baseline that
can exist with the A-B-A and A-B-A-B designs.
Phase 1 Phase 2 Phase 3 Phase 4
Different people, Different behaviours, or Different settings
A Baseline Treatment Treatment Treatment
B Baseline Baseline Treatment Treatment
C Baseline Baseline Baseline Treatment
3 Time series Analysis Design:
In this design, test the participants several times prior to the intervention
and after the intervention. If the intervention has no effect, the change in
scores will be smooth and continuous. An effective intervention would be
signalled by a discontinuity in this smooth progression; either a discrete
discontinuity at the time of intervention. Time series analysis are widely
available.
4 Naturalistic Experiments:
In these designs the term experiment is used loosely as there is no formal
randomization of subjects or groups and there is no experimenter initiated
intervention that is systematically applied to one group. The control can be
historical control. The newly introduced educational intervention (not by the
researcher) group will serve as experimental group. Eg. There is a change in
the syllabus and curriculum prescribed by Maharashtra University of Health
Sciences, Nashik during 1998. The older curriculum followed by respective
university was followed earlier. Students in this category will serve as
historical control. The limitations of the study include systematic difference
Page | 45
between the two groups in time, and change in the outcome measures over a
period of time (eg. Evaluation was long theory questions earlier compared to
MCQs and SAQ in the revised curriculum), expert versus novice difference.
5 Changing-Criterion Design
This is a single-case design that is used when a behavior needs to be shaped
over time or when it is necessary to gradually change a behavior through
successive treatment periods to reach a desired criterion. This design
involves collecting baseline data on the target behavior and then
administering the experimental treatment condition across a series of
intervention phases where each intervention phase uses a different criterion
of successful performance until the desired criterion is reached. The
criterion used in each successive intervention phase should be large enough
to detect a change in behaviour but small enough so that it can be achieved.
Conclusion:
Selection of an appropriate research design is vital. Use traditional
qualitative techniques and pilot test to refine the research question prior to
the main experiment. Remember that the educational experimentation will
always involve a set of compromises (such as the nature of the control
group, the nature of evaluation process, the nature of the research design
selected). It is better to explore the limitations (compromises) and select the
research design that best satisfies the needs. Also decide whether these
compromises are reasonable.
Qualitative Research Methods:
There are five major types of qualitative research which are basically similar.
However, each approach has some distinct characteristics and tends to have
its own roots.
• Phenomenology – a form of qualitative research in which the researcher
attempts to understand how one or more individuals experience a
phenomenon. For example, interview 20 trainees and ask them to describe
their experiences of the training programme.
• Ethnography – is the form of qualitative research that focuses on
describing the culture of a group of people. Note that a culture is the shared
attitudes, values, norms, practices, language, and material things of a group
Page | 46
of people. E.g. The tribal anthropological studies, describing their beliefs,
attitudes and their culture.
• Case study research – is a form of qualitative research that is focused on
providing a detailed account of one or more cases. For an example, study a
PBL programme implemented in a institute.
• Grounded theory – is a qualitative approach to generating and developing
a theory form data that the researcher collects. For example, Collection of
data on how the habituation of tobacco has started and is being sustained
in youth. Based on this data a theory can be construed to explain the
mechanism of tobacco use.
• Historical research – research about events that occurred in the past.
Example, study the use of education system at Nalanda University.
Mixed Research Methods:
Mixed research is a general type of research in which quantitative and
qualitative methods, techniques, or other paradigm characteristics are
mixed in one overall study. There are two major types of mixed research.
• Mixed method research – is research in which the researcher uses the
qualitative research paradigm for one phase of a research study and the
quantitative research paradigm for another phase of the study. Mixed
method research is like conducting two mini-studies within one overall
research study.
• Mixed model research – is research in which the researcher mixes both
qualitative and quantitative research approaches within a stage of the study
or across two of the stages of the research process.
The Advantages of Mixed Research
Mixed research is advocated whenever it is feasible. It will help qualitative
and quantitative researchers to get along better and, will promote the
conduct of excellent educational research. The researcher who mixes
quantitative and qualitative research methods, procedures, and paradigm
characteristics will be able to get better results due to the complementary
strengths and non-overlapping weaknesses. When different approaches are
used to focus on the same phenomenon and they provide the same result,
you have "corroboration" which means superior evidence for the result.
Page | 47
Lesson IV:
Quantitative Methods: Data Collection, Questionnaire Preparation
Objectives:
4. Describe the various categories of data(Nominal, ordinal, interval & Ratio) 5. Understand the sampling procedures used in educational research. 6. Devise a questionnaire for data collection and data quality checks
Lesson Outline:
1 Methods of Data Collection: Tests; Questionnaires; Interviews; Focus Groups; Observation and secondary data. Scales of measurement. Assumptions underlying Testing and Measurement. Identifying a good test or Assessment Procedure. Assessment tools in Education. Achievement Tests. OSCE. Patient satisfaction questionnaire. Ratings. Student Write ups
2 Sampling Procedure. Terminology used. Random sampling, Systematic, Cluster sampling. Non random sampling techniques. Quota sampling, purposive sampling
3 15 Principles for preparation of Questionnaire. Rating Scales, Rankings and checklists. Strengths and weaknesses of questionnaire. Types of questions. Open, closed.
Measurement: It is defined as the act of measuring by assigning symbols or
numbers to something according to a specific set of rules.
Measurement can be categorized by the type of information that is
communicated. They are called the four "scales of measurement." The
numerical method of describing observations of materials or characteristics
is called as “Quantification”.
Scales of Measurement
1. Nominal Scale. This is a least precise method of nonquantitative
measurement scale. It is used to categorize, label, classify, name, or
identify variables. These scales are nonorderable. It classifies groups or
types. It describes differences between things by assigning them to
categories-such as professors, associate professors, lecturers, tutors,
residents –and subsets such as males or females.
2. Ordinal Scale.
It indicates not only that things differ but that they differ in amount or
degree. However, the real differences between adjacent ranks may not be
equal. Any variable where the levels can be ranked is an ordinal variable.
Page | 48
Some examples are order of academic position in a examination, top 10,
rank in class.
3. Interval Scale.
It is based on equal units of measurement indicating how much of a
given characteristic is present. It indicates the relative amount of a trait
or characteristic. It‟s primary limitation is the lack of zero. Psychological
tests and inventories are interval scales although they can be added,
subtracted, multiplied, and divided. Another example is Celsius
temperature, Fahrenheit temperature, IQ scores where zero degrees in
these scales does not mean zero or no temperature.
4. Ratio Scale. This is a scale having equal interval properties in addition to
a true zero point and can be added, subtracted, multiplied and divided
and expressed in ratio relationships. It also has all of the "lower level"
characteristics (i.e., the key characteristic of each of the lower level
scales) of equal intervals (interval scale), rank order (ordinal scale), and
ability to mark a value with a name (nominal scale). Some examples of
ratio level scales are number correct, weight, height, response time,
Kelvin temperature, and annual income.
In Qualitative behavioural research many of the qualities or variables of
interest are abstractions and can not be observed directly. It is necessary to
define them in terms of observable acts. This operational definition tells
what the researcher must do to measure the variable. Eg. Intelligence is an
abstract quality that can not be observed directly. However, it can be defined
operationally as scores achieved on a particular intelligence test. It must be
remembered that excessive emphasis on quantification may result in the
measurement of fragmentary qualities not relevant to the real behaviour.
Assumptions Underlying Testing and Measurement
Cohen, et al. considered twelve assumptions which are basic to testing and
assessment. Testing is defined as the process of measuring variables by
means of devices or procedures designed to obtain a sample of behaviour
and Assessment is the gathering and integration of data for the purpose of
making an educational evaluation, accomplished through the use of tools
Page | 49
such as tests, interviews, case studies, behavioural observation, and
specially designed apparatus and measurement procedures.
1. Psychological traits and states exist: A trait is a relatively enduring (i.e.,
long lasting) characteristic on which people differ; a state is a less enduring
or more transient characteristic on which people differ.
2. Psychological traits and states can be quantified and measured: For
nominal scales, the number is used as a marker. For the other scales, the
numbers become more and more quantitative. Most traits and states
measured in education are taken to be at the interval level of measurement.
3. Various approaches to measuring aspects of the same thing can be
useful: For example, different tests of intelligence tap into somewhat
different aspects of the construct of intelligence.
4. Assessment can provide answers to some of life's most momentous
questions: It is important that the users of assessment tools know when
these tools will provide answers to their questions.
5. Assessment can pinpoint phenomena that require further attention or
study: For example, assessment may identify someone as having dyslexia or
low self-esteem or at-risk for drug use.
6. Various sources of data enrich and are part of the assessment process:
Information from several sources usually should be obtained in order to
make an accurate and informed decision. For example, the idea of portfolio
assessment is useful.
7. Various sources of error are always part of the assessment process:
There is no such thing as perfect measurement. All measurement has some
error. The two main types of error are random error (e.g., error due to
transient factors; also leads to less reliability) and systematic error (e.g.,
error present every time the measurement instrument is used; leads to
decreased validity).
8. Tests and other measurement techniques have strengths and
weaknesses: It is essential that users of tests understand this so that they
can use them appropriately and intelligently.
Page | 50
9. Test-related behaviour predicts non-test-related behaviour: The goal of
testing usually is to predict behaviour other than the exact behaviours
required while the exam is being taken.
10. Present-day behaviour sampling predicts future behaviour: Perhaps the
most important reason for giving tests is to predict future behaviour. Tests
provide a sample of present-day behaviour. However, this "sample" is used
to predict future behaviour.
11. Testing and assessment can be conducted in a fair and unbiased
manner: This requires careful construction of test items and testing of the
items on different types of people to make sure tests are fair and unbiased.
12. Testing and assessment benefit society: Many critical decisions are
made on the basis of tests (e.g., employability, presence of a psychological
disorder, degree of teacher satisfactions, degree of student satisfaction, etc.).
In Medical Education, qualitative data collection techniques such as
projective tests, observation, open ended questionnaires and opinionnaires,
and interviews are used for quantitative data. It is unwise to draw a hard-
and-fast distinction between qualitative and quantitative studies. The
difference is not absolute; it is one of emphasis. One emphasis should not
be considered superior to the other. The appropriate approach depends on
the nature of the questions under consideration and the objectives of the
researchers.
Psychological and Educational Tests and Inventories
Psychological tests are among the most useful tools of educational research.
These tests are instruments designed to describe and measure a sample of
certain aspects of human behaviour. He tests yield objective and
standardised descriptions of behaviour, quantified by numerical scores.
Tests and inventories are used to describe status or prevailing condition, to
measure changes in status produced by modifying factors, or to predict
future behaviour on the basis of present performance. Tests can be
classified as Performance tests (to assess the skill) and paper-and-pencil
tests (to measure knowledge). They can be also divided into Power versus
timed or speed tests. Power tests have no time limit, and the subjects
attempt progressively ore difficult tasks until they are unable to continue
Page | 51
successfully. Speed tests limit the time; the subjects have to complete
certain tasks. Alternatively, tests can be either non-standardised (teacher
made: terminal internal examinations) and Standardised (tailor-made:
university examinations).
Good measurement is fundamental for research. Testing and assessment
procedures are characterized by high reliability and high validity. While
devising test one should also consider the economy and interest of the test.
Validity It is the best available approximation to the truth of a given
proposition, inference, or conclusion
Validity refers to the degree to which evidence and theory support the
interpretation of test scores entailed by proposed uses of tests. It is the
accuracy of the inferences, interpretations, or actions made on the basis of
test scores. Validity has to do with both the attributes of the test and the
uses to which it is put. Technically speaking, it is incorrect to say that a
test is valid or invalid. It is the interpretations and actions taken based on
the test scores that are valid or invalid. All of the ways of collecting validity
evidence are really forms of what used to be called construct validity. All
that means is that in testing and assessment, we are always measuring
something (e.g., IQ, gender, age, depression, self-efficacy).
The overall purpose of educational and psychological testing is to draw an
inference about an individual or group of individuals. Because these
inferences are made on the test scores, the tests must have appropriate
evidence of their validity for these uses.
Introduction to Validity
Virtually all educational research involves measurement or observation.
And, whenever we measure or observe we are concerned with whether we
are measuring what we intend to measure or with how our observations are
influenced by the circumstances in which they are made. We reach
conclusions about the quality of our measures -- conclusions that will play
an important role in addressing the broader substantive issues of our study.
When we talk about the validity of research, we are often referring to these
to the many conclusions we reach about the quality of different parts of our
research methodology.
Page | 52
Sources of validity evidence
It should be remembered that the evidence for the specific use of a given test
is available rather than validity as residing in the test itself. Hence instead of
using the traditional terms such as content validity, predictive validity and
construct validity, general term validity evidence is being used in
educational research. Validity evidence is based on three broad sources:
content, relations to other variables, and construct. Not all test uses must
meet all three types. Different types of tests are used for different purposes
and, therefore, need different types of evidence. E.g. Intelligence test is
designed to predict academic achievement and is based on psychological
theory or construct. Thus it needs demonstration of evidence for both
construct and prediction but not necessarily demonstrate evidence for the
content.
1 Evidence of test content
It refers to the degree to which the test items actually measure, or are
specifically related to, the traits for which the test was designed and is to be
used. The content includes the issues, the actual wording, the design of the
items, or questions, and how adequately the test samples the universe of
knowledge and skills that a student is expected to master. The content
should match the course text books, syllabi, objectives and the judgements
of subject experts. It is high importance for achievement tests but not so for
aptitude tests. Also assess whether the test‟s content is appropriate for the
persons to be tested.
2 Evidence based on relations to other variables
This type of evidence has traditionally been referred to as criterion-related
validity. There may be two types namely, Predictive (the test is designed and
used to predict other variables) and concurrent (other tests that are
supposed to measure the same or similar construct). E.g. Usefulness of MH-
CET scores in predicting medical college performance scores. The faults in
the prediction can be attributed to the test itself or in the criteria of success,
Page | 53
or both. Hence it can be concluded that predictive validity is not easy to
assess. Evidence for the validity of the relationship to other measures refers
to whether test is closely related to other measures, such as institute ratings
(NAAC/ NBA), teacher‟s experience, or scores on another test of known
validity. Through this process, more convenient and more appropriate tests
can be devised to accomplish the measurement of behaviour more
effectively. In such cases, evidence will be convergent; otherwise, it would
be discriminate evidence.
3 Evidence based on Internal Structure
This type of evidence is also called as construct validity. It is the degree to
which test items and the structure of a test can be accounted for by the
explanatory constructs of a sound theory. A construct is a trait that can not
be observed. If one were to study such a construct as dominance, one would
hypothesize that people who have this characteristic will perform differently
from those who do not. Theories can be built describing how dominant
people behave in a distinctive way. Different Intelligence tests are based on
different theories; each test should be shown to measure what the
appropriate theory defines as intelligence. Evidence of internal structure is
important for personality and aptitude tests.
Validity is a unitary concept based on all of the evidence, the totality of the
evidence should be considered as evidence for validity of a given test use.
Page | 54
Reliability
Reliability refers to consistency or stability. In psychological and educational
testing, it refers to the consistency or stability of the scores that we get from
a test or assessment procedure. A test is reliable to the extent that it
measures whatever it is measuring consistently. Reliable tests are stable
and yield comparable scores on repeated administration. It is usually
determined using a correlation coefficient (it is called a reliability coefficient
in this context). The correlation coefficient is a measure of relationship that
varies from -1 to 0 to +1. Increase in the number of items in a test would be
able to increase the reliability because a test with few items has a great deal
of measurement error. There are a number of types of reliability.
1 Stability over time (test-retest): The scores on a test will be highly
correlated with scores on a second administration of the test to the
same subjects at a later date.
2 Stability over item samples (equivalent or parallel forms): Some tests
have two or more forms that may be used interchangeably. In these
cases, scores on one version will be very similar with scores on the
alternate form of the test. This can be practically carried out through
Page | 55
the administration of a longer test comprising of both the versions of
the test. Later separating the two versions, scores can be compared.
3 Stability of items (internal consistency): Test items should be highly
related to other test items. This is important because the test, or in
some cases the subtest, needs to measure a single construct. This can
be achieved by a) Split halves (test through Spearman-Brown formula)
or Coefficient of consistency (test through Kuder-Richardson
formula).
4 Stability over scorers (inter-scorer): Projection tests have a great deal
of judgement of the person scoring the test. Scorer reliability can be
determined by two independent scorers scoring the same test papers
or video tapes of the test.
5 Stability over testers: Differently trained testers and their personality
or other attributes can affect the test scores. This can overcome
through two different testers administer the two testings, with each
one giving the test first half of the time.
6 Standard error of measurement: This statistic permits the
interpretation of individual scores obtained on a test. No tests are
perfectly reliable. The standard error of measurement tells how much
difference can be expected by obtained score which is away from the
true score.
A test may be reliable even though it is not valid. However, for a test to be
valid, it must be reliable. That is, a test can consistently measure (reliability)
nothing of interest (be invalid), but if a test measures what is designed to
measure (validity), it must do so consistently (reliability).
Economy
Tests that can be given in a short period of time are likely to gain
cooperation of the subjects. Ease of administration, scoring, and
interpretation are important factors in selection of the test.
Interest
Tests that are interesting and enjoyable help to gain the cooperation of the
subject. Those that are dull or seem silly may discourage or antagonise the
Page | 56
subject. Under these unfavourable conditions the test is not likely to yield
useful results.
When psychological tests are used in educational research, one should
remember that standardised test scores are only approximate measures of
the traits under consideration. This limitation is inevitable and may be
ascribed to a number of possible factors.
1 Errors inherent in any psychological test-no test is completely
valid or reliable
2 Errors that may result from poor test conditions, inexpert or
careless administration or scoring of the test, or faulty tabulation
of test score
3 Inexpert interpretation of test results
4 The choice of an inappropriate test for the specific purpose in
mind.
Methods of Data Collection
There are six major methods of data collection.
• Tests (i.e., includes standardized tests that usually include
information on reliability, validity, and norms as well as tests
constructed by researchers for specific purposes, skills tests, etc).
• Questionnaires (i.e., self-report instruments).
• Interviews (i.e., situations where the researcher interviews the
participants).
• Focus groups (i.e., a small group discussion with a group moderator
present to keep the discussion focused).
• Observation (i.e., looking at what people actually do).
• Existing or Secondary data (i.e., using data that are originally
collected and then archived or any other kind of “data” that was
simply left behind at an earlier time for some other purpose).
Tests
Tests are commonly used in research to measure personality, aptitude,
achievement and performance. Tests can also be used to complement
other measures (following the fundamental principle of mixed research).
A researcher must develop a new test to measure the specific knowledge,
Page | 57
skills, behaviour, or cognitive activity that is being studied. For example,
a researcher might need to measure response time to a memory task
using a mechanical apparatus or develop a test to measure a specific
mental or cognitive activity (which obviously cannot be directly observed).
Remember that if a test has already been developed that purports to
measure what is intended to measure, then consider that test.
Strengths and Weaknesses of Tests
Strengths Weaknesses
Can provide measures of many
characteristics of people.
Can be expensive if test must be
purchased.
Often standardized (same stimulus is provided to all participants).
Reactive effects such as social desirability can occur.
Allows comparability of common measures across populations.
Test may not be appropriate for a local or unique population.
Strong psychometric properties (high measurement validity).
Open-ended questions and probing not available.
Availability of reference group data. Tests are sometimes biased against certain groups of people.
Many tests can be administered to groups which saves time.
Non response to selected items on the test.
Can provide “hard,” quantitative data.
Some tests lack psychometric data.
Tests are usually already developed. Can be expensive if test must be purchased.
A wide range of tests is available. Reactive effects such as social desirability can occur.
Response rate is high for group administered tests.
Test may not be appropriate for a local or unique population.
Ease of data analysis because of quantitative nature of data.
Open-ended questions and probing not available.
Educational and Psychological Tests in Medical Education
Three primary types of tests viz. intelligence tests, personality tests, and
educational assessment tests are commonly used in educational research.
1) Intelligence Tests: Intelligence has many definitions because a single
prototype does not exist. Although the construct of intelligence is hard to
define, it still has utility because it can be measured and it is related to
many other constructs.
2) Personality Tests: Personality is a construct similar to intelligence in
that a single prototype does not exist. Personality is the relatively permanent
Page | 58
patterns that characterize and can be use to classify individuals. Most
personality tests are self-report measures. Performance measures of
personality are also used. A performance measure is a test-taking method in
which the participants perform some real-life behaviour that is observed by
the researcher. Personality has also been measured with projective tests. A
projective test is a test-taking method in which the participants provide
responses to ambiguous stimuli. The test administrator searches for
patterns on participants‟ responses. Projective tests tend to be quite difficult
to interpret and are not commonly used in quantitative research.
3) Educational Assessment Tests.
There are four subtypes of educational assessment tests:
• Achievement Tests: These tests are important in Medical
Education. These are used in placing, advancing, or retaining
students at particular grade levels. These will measure the degree of
learning that has taken place after a person has been exposed to a
specific learning experience. They can be teacher constructed or
standardized tests. Many of these achievements tests are non-
standardised, teacher-designed tests which lack content validity.
Concurrent validity might be used to help establish a new
achievement test‟s validity. The only forms of reliability that are
critical are test-re-test, stability over test items, and the standard
error of measurement.
• Aptitude Tests: These focus on information acquired through the
informal learning that goes on in life. They are used to predict future
performance whereas achievement tests are used to measure current
performance. Aptitude tests attempt to predict an individual‟s capacity
to acquire improved performance with additional training. Eg.
Stanford-Binet Intelligence scale. These tests, particularly those that
deal with academic aptitude, that are used for purpose of placement
and classification have become highly controversial because of the
culturally different content. It is extremely difficult to eliminate
culture totally and develop one test that is equally fair to all
communities including the minority. For these tests, Predictive
Page | 59
validity and construct validity are important. The forms of reliability
that are critical to these tests are test-retest, stability over test items,
and the standard error of measurement. The tests that have some
degree of subjectivity also require inter-scorer and inter-tester
reliability.
• Personality Inventories: Personality scales are usually self-report
instruments. Because of the individual‟s inability or unwillingness to
report their own reactions accurately or objectively, these instruments
may be of limited value. They provide data useful in suggesting the
need for further analysis. Test setting also influence the results. Eg. A
test applied in clinical setting correlate well with psychiatrist‟s
diagnosis; but when applied to college students, it‟s diagnostic value
might be disappointing.
Diagnostic Tests: These tests are used to identify the locus of
academic difficulties in students.
Questionnaires: Inquiry Forms
A questionnaire is a self-report data collection instrument that is filled out
by research participants to record the factual information desired. When
opinions rather than facts are desired, an Opionnaire or Attitude scale is
used. Questionnaires are usually paper-and-pencil instruments, but they
can also be placed on the web for participants to go to and “fill out.”
Questionnaires are sometimes called survey instruments, but the actual
questionnaire should not be called “the survey.” The word “survey” refers to
the process of using a questionnaire or interview protocol to collect data.
Whenever questionnaires are mailed, the accompanying letter should not
commit that the sender needs the information to complete the requirements
for a graduate course, thesis or dissertation. This would lead to poor
response rates. Less than 50% ( 50% is adequate; 60% is good; and 70% is
very good) of response rates lead to limited validity. A questionnaire is
composed of questions and/or statements. Researcher should learn to write
questionnaires. Questionnaires that call for short, check mark responses are
known as restricted, or closed-form, type. In such forms besides providing
Page | 60
all the possible alternatives, add another alternative called other. This
category of response permits respondents to indicate what might be their
most important reason. When the questionnaire contains open questions, it
is called as open form. When developing a questionnaire make sure to follow
the 20 Principles of Questionnaire Construction to improve questionnaire
items.
Principle 1: Make sure the questionnaire items match the research
objectives.
Principle 2: Understand the research participants as they will be filling out
the questionnaire. Consider the demographic and cultural characteristics of
potential participants so that they can understand the questions properly.
Principle 3: Be careful in using descriptive adjectives and adverbs that have
no agreed upon meaning. Use natural and familiar language which is
comforting. If required, underline a word to indicate special emphasis.
Principle 4: Define or qualify terms that can be easily misunderstood. Write
items that are clear, precise, and relatively short. Short items are more
easily understood and less stressful than long items.
Principle 5: Do not use "leading" or "loaded" questions. Leading questions
lead the participant to ideal/ desired situations. Loaded questions include
loaded words (i.e., words that create an emotional reaction or response by
participants).
Principle 6: Avoid double-barreled questions. A double-barreled question
combines two or more issues in a single question (e.g., here is a double
barreled question: “Do you elicit information from parents and other
teachers?” It‟s double barreled because if someone answered it, it would not
possible to know whether they were referring to parents or teachers or both).
If the question includes the word "and"? If yes, it might be a double-barreled
question. Answers to double-barreled questions are ambiguous because two
or more ideas are confounded.
Principle 7: Avoid double negatives. Does the answer provided by the
participant require combining two negatives? (e.g., "I disagree that teachers
should not be required to supervise their students during library time"). If
yes, rewrite it.
Page | 61
Principle 8: Be careful of inadequate alternatives. Determine whether an
open-ended or a closed ended question is needed. Open-ended questions
provide qualitative data in the participants' own words. Open-ended
questions are common in exploratory research and closed-ended questions
are common in confirmatory research. Here is an open ended question:
How can your principal improve the morale at your school?
______________________________________________________________
Closed-ended questions provide quantitative data based on the researcher's
response categories.
Principle 9: Use mutually exclusive and exhaustive response categories for
closed-ended questions. Mutually exclusive categories do not overlap (e.g.,
ages 0-10, 10-20, 20-30 are NOT mutually exclusive and should be
rewritten as less than 10, 10-19, 20-29, 30-39, ...). Exhaustive categories
include all possible responses (e.g., if in a national survey of adult citizens
(i.e., 18 or older) then these categories (18-19, 20-29, 30-39, 40-49, 50-59,
60-69) are NOT exhaustive because there is no where to put someone who is
70 years old or older.
Principle 10: When asking for ratings or comparisons, provide a point of
reference. Also, provide if necessary, systematic quantification of response.
Consider the different types of response categories available for closed-ended
questionnaire items. Rating scales are commonly used. Numerical rating
scales (where the endpoints are anchored; sometimes the center point or
area is also labelled) are as below.
1 2 3 4 5 6 7
Very Low Very
High
1 2 3 4 5
Strongly Agree Neutral Disagree Strongly Agree
Disagree
Omitting the centre point on a rating scale (e.g., using a 4-point rather than
a 5-point rating scale) does not appreciably affect the response pattern.
Page | 62
Some researchers prefer 5- point rating scales; other researchers prefer 4-
point rating scales. Both generally work well.
Rankings (where participants put their responses into rank order, such as
most important, second most important, and third most important) can be
converted into Likert Scale.
Semantic differential (i.e., where one item stem and multiple scales that are
anchored with polar opposites or antonyms, are included and are rated by
the participants). It is similar to Likert Method in that the respondent
indicates an attitude or opinion between two extreme choices.
Checklists (i.e., where participants "check all of the responses in a list that
apply to them") can also be used.
Principle 11: Use multiple items to measure abstract constructs. This is
required to have high reliability and validity. One approach is to use a
summated rating scale also known as Likert Scale(such as the Rosenberg
Self-Esteem Scale that is composed of 10 items, with each item measuring
self-esteem).
Principle 12: Use multiple methods to measure abstract constructs. Use of
only one method might result in artefact of that method of measurement. If
more than one method is used, check can be kept whether the answers
depend on the method.
Principle 13: Use caution if reverse wording is used in some of the items to
prevent response sets. (A response set is the tendency of a participant to
respond in a specific direction to items regardless of the item content).
Reversing the wording of some items can help ensure that participants don't
just "speed through" the instrument, checking "yes" or "strongly agree" for
all the items. Evidence suggests that the use of reverse wording reduces the
reliability and validity of scales.
Principle 14: Phrase questions so that they are appropriate for all
respondents. Also, avoid unwanted assumptions. Design questions that will
give a complete response. Develop a questionnaire that is easy for the
participant to use. The participant must not get confused or lost anywhere
in the questionnaire.
Page | 63
Principle 15: Always pilot test the questionnaire with a small group of
persons similar to those who will be used in the study. These dry runs will
be worth the time and effort. Based on the observations of pilot study, revise
and re-revise the questions if necessary.
Principle 16: Questionnaire should seek information that can not be
obtained from other sources such as college record/ reports or census data.
Principle 17: Questions are to be presented in good psychological order,
proceeding from general to more specific questions.
Principle 18: It is advisable to pre-construct a tabulation sheet, anticipating
how the data will be tabulated (Dummy Tables) and interpreted, before the
final form of the questionnaire is decided on.
Principle 19: Questionnaire should be attractive in appearance, neatly
arranged, and clearly duplicated or printed.
Principle 20: It should be as short as possible and only long enough to get
the essential data
Strengths and Weaknesses of Questionnaires
Strengths Weaknesses
Good for measuring attitudes and eliciting other content
Usually must be kept short.
Inexpensive (mail questionnaires & group administered questionnaires).
Reactive effects may occur (e.g., interviewees may show only what is socially desirable).
Can provide information about participants‟ internal meanings and ways of thinking.
Non-response to selective items.
Can administer to probability samples. People may not recall important information and may lack self-awareness.
Quick turnaround. Response rate may be low for mail and email questionnaires.
Can be administered to groups. Open-ended items may reflect differences in verbal ability, obscuring the issues of interest.
Perceived anonymity by respondent may be high.
Data analysis can be time consuming for open-ended items.
Moderately high measurement validity (i.e., high reliability and validity)
Measures need validation.
Closed-ended items can provide exact information
Open-ended items can provide detailed information
Ease of data analysis for closed-ended items.
Useful for exploration as well as confirmation.
Interviews
Page | 64
In an interview, the interviewer asks the interviewee questions (in-person or
over the telephone). It is often superior to other data gathering devices.
Through this technique, the researcher may stimulate the subject‟s insight
in to his/her own experiences, thereby exploring significant areas not
anticipated in the original plan of investigation. It is the appropriate
techniques when dealing with children. Trust and rapport are important.
Probing is available and is used to reach clarity or gain additional
information. It is necessary to consider the gender, race, and possibly other
characteristics of the interviewer. However, distilling the essence of the
reaction is difficult, and interviewer bias may be a hazard. In interviews
actual wording of the responses should be retained. The validity can be
increased by conducting a structured interview. Though time consuming,
this technique, is useful in areas where human motivation is revealed
through actions, feelings, and attitudes. Interviews can be classified as
qualitative and quantitative. Quantitative interviews utilise closed-ended
questions and are standardised. Unlike qualitative interviews consist of
open ended questions. These can be further subdivided into informal
conversational interview (which is spontaneous and is loosely structured);
Interview Guide Approach (which is more structured having interview
protocol. Wording and sequence of questions can be altered by the
interviewer); and
Standardized Open-Ended Interview (where the questions are in a protocol
strictly adhered to and the wording can not be changed).
Strengths and Weaknesses of Interviews
Strengths Weaknesses
Good for measuring attitudes and most other content of interest.
In-person interviews are expensive and time consuming.
Allows probing and posing of follow-up questions by the interviewer.
Reactive effects (e.g., what is socially desirable).
Can provide in-depth information. Investigator effects may occur (e.g., untrained interviewers distort data due to personal biases and poor interviewing skills).
Can provide information about participants‟ internal meanings and ways of thinking.
Interviewees may not recall important information and may lack self-awareness.
Closed-ended interviews provide exact information.
Perceived anonymity by respondents may be low.
Telephone and e-mail interviews provide very Data analysis is time consuming for open-ended
Page | 65
quick turnaround. items.
Moderately high measurement validity (i.e., high reliability and validity)
Measures need validation
Can use with probability samples.
Relatively high response rates are often attainable.
Useful for exploration as well as confirmation.
.
Q Methodology
Q-Methodology is a technique for scaling objects or statements. It is a
method of ranking attitudes or judgements and is primarily effective when
the number of items to be ranked is large. The procedure is known as Q-
Sort, in which cards or slips bearing the statements or items are arranged in
a series of numbered piles.
Social Scaling
Sociometry: It is a technique for describing the social relationships among
individuals in a group. In an indirect way this technique attempts to
describe attraction or repulsion between individuals by asking them to
indicate whom they would choose or reject in various situations.
Diagrammatically, it can be represented as Venn or Chapati. In medical
education, health care seeking behaviour of a community or group can be
studied.
Sociogram: Sociometric choices may be represented graphically on a chart
known as sociogram. In this chart, those most chosen are referred as Stars,
and those less chosen as Isolates. Small groups made up of individuals who
choose one another are Cliques. Sociometry is a peer rating rather than a
rating by superiors. Students of group relationships and classroom
teachers may construct a number of sociograms over a period of time to
measure changes that may have resulted from efforts to bring isolates into
closer group relationships or to transform cliques into more general group
membership. Another technique used also determines the social-Distance.
This Social-Distance Scale attempts to measure to what degree an
individual or group of individuals is accepted or rejected by another
individual or group. The target sociogram is depicted as below. In this
Page | 66
diagram, nested series of concentric circles are drawn based on the points
that are equal in terms of how frequently they were chosen. Points in the
central circle are more central in the sense that they were chosen more
often. Points at the edge were chosen less often. The lines connecting them
represent the primary links between pairs. And all the points are placed in
the rings in such a way that the lines connecting them are relatively short. d
“Guess-Who” Technique
Developed by Hartshorne and May, 1929, Guess-Who technique is a
process, consists of descriptions of the various roles played by children in a
group. Children are asked to name the individuals who fit certain verbal
descriptions.
Name the teacher who always comes late to the class
Name the teacher who always uses the word” you know”
Name the student who always smiles and happy
Focus Groups
A focus group is a situation where a focus group moderator keeps a small
and homogeneous group (of 6-12 people) focused on the discussion of a
research topic or issue. Focus group sessions generally last between one
and three hours and they are recorded using audio and/or videotapes.
These groups are useful for exploring ideas and obtaining in-depth
information about how people think about an issue.
Strengths and Weaknesses of Focus Groups
Page | 67
Strengths Weaknesses Useful for exploring ideas and concepts.
Sometimes expensive.
Provides window into participants‟ internal thinking.
Difficult to find a moderator with good facilitative and rapport building skills.
Can obtain in-depth information. Reactive and investigator effects occur if participants feel they are being watched or studied (Hawthorne Effect).
Can examine how participants react to each other.
May be dominated by one or two participants.
Allows probing. Difficult to generalize results if small, unrepresentative participants sample
Most content can be tapped. May include large amount of extra or unnecessary information.
Allows quick turnaround. Measurement validity may be low.
Usually should not be the only data collection methods used in a study.
Data analysis can be time consuming because of the open-ended nature of data
Observation
In the method of data collection called observation, the researcher observes
participants in natural and/or structured environments as participant or
non-participant. Observation can be carried out as time sampling technique
(based on observation of individuals behaviour for every 60 seconds or more)
or it can be carried out as a frequency count (based on the number of
occurrences of a particular type of behaviour). Observation is specifically
used effectively to scout the performance of opposing teams in sports. It is
important to collect observational data (in addition to attitudinal data)
because what people say is not always what they do! Observation can be
carried out in two types of environments namely Laboratory observation
(which is done in a lab set up by the researcher) and Naturalistic
observation (which is done in real-world settings). However, observation
should always be systematic, directed by a specific purpose, carefully
focussed and thoroughly recorded. Criterion-related and construct validity
are necessary. It is recommended that observations should be double-blind
(both the observers and the observed are unaware of the purpose of the
study and are unaware of the observation process). It is also suggested in
order to reduce observer bias, conduct study by more than one observer.
Always have a dry-run phase before the implementation. Simultaneous
Page | 68
recording of observations is highly recommended. Educational research
seeks to describe behaviour under less rigid controls and more natural
conditions. There are two important forms of observation.
1 Quantitative observation involves standardization procedures, and it
produces quantitative data (The following data is collected: Who is observed;
what is observed; when the observations are to take place; where the
observations are to take place; and how the observations are to take place).
Standardized instruments (e.g., checklists) are often used in quantitative
observation. Two sampling procedures are also often used in quantitative
observation. Time-interval sampling (i.e., observing during time intervals,
e.g., during the first minute of each 10 minute interval) and Event sampling
(i.e., observing after an event has taken place, e.g., observing after teacher
asks a question).
2 Qualitative observation is exploratory and open- ended, and the
researcher takes extensive field notes. The qualitative observer may take on
four different roles that make up a continuum:
• Complete participant
• Participant-as-Observer (i.e., spending extensive time "inside" and
informing the participants that you are studying them).
• Observer-as-Participant (i.e., spending a limited amount of time "inside"
and informing them that you are studying them).
• Complete Observer
Strengths and Weaknesses of Observational Data
Strengths Weaknesses Allows one to directly see what people do without relying on what they say.
Reasons for observed behaviour may be unclear.
Provides firsthand experience. Reactive effects may occur when respondents know they are being Observed( Hawthorne Effect).
Can provide relatively objective measurement of behaviour.
Investigator effects (e.g., personal biases and selective perception)
Observer may see things that escape the awareness of people in the setting.
Sampling of observed people and settings may be limited.
Excellent way to discover what is occurring in a setting.
Cannot observe large or dispersed populations.
Helps in understanding importance of contextual factors.
Some settings and content of interest cannot be observed.
Can be used with participants with weak Collection of unimportant material
Page | 69
verbal skills. may be moderately high.
Provide information on things people would otherwise be unwilling to talk about.
More expensive to conduct than questionnaires and tests.
Observer may move beyond selective perceptions of people in the setting.
Data analysis can be time consuming.
Good for description.
Provides moderate degree of realism.
Secondary/Existing Data
Secondary data (i.e., data originally used for a different purpose) are
contrasted with primary data (i.e., original data collected for the new
research study). The most commonly used secondary data are documents,
physical data, and archived research data.
1. Documents. These are Personal documents (i.e., Letters, diaries,
family pictures) and Official documents (i.e., Newspapers, annual reports,
yearbooks, minutes).
2. Physical data (are any material thing created or left by humans that
might provide information about a phenomenon of interest to a researcher).
3. Archived research data (i.e., research data collected by other
researchers for other purposes, and these data are save often in tape form or
CD form so that others might later use the data).
Strengths and Weaknesses of Secondary Data
Physical Data
Strengths Weaknesses
Can provide insight into what people think and what they do.
May be incomplete.
Unobtrusive, making reactive and investigator effects very unlikely.
May be representative only of one perspective.
Can be collected for time periods occurring in the past (e.g., historical).
Access to some types of content is limited.
Provides background and historical data on people, and organizations.
May not provide insight into participants‟ thinking for physical data.
Useful for corroboration. May not apply to general populations.
Grounded in local setting.
Useful for exploration.
Archived research data
Strengths Weaknesses
Page | 70
• Archived research data are available on a wide variety of topics.
• May not be available for the population of interest to you.
• Inexpensive. • May not be available for the research questions of interest to you.
• Often are reliable and valid (high measurement validity).
• Data may be dated.
• Can study trends. • Open-ended or qualitative data usually not available.
• Ease of data analysis. • Many important findings have already been mined from the data.
• Often based on high quality or large probability samples.
Page | 71
Lesson V:
Qualitative and Mixed Methods in Educational Research
Objectives:
1. Understand the types of qualitative research (phenomenology, ethnography, grounded theory, case study and concept maps)
2. Describe the various qualities (SWOT) of qualitative research 3. Understand the data collection tools and techniques for qualitative
data(questionnaire, Interview, Focus group, Observation ) 4. Types of Qualitative data. Common misconceptions about qualitative research 5. Mixed Methods
Lesson Outline:
1 Definition and General Characteristics of qualitative research namely Phenomenology, Ethnography, Discourse Analysis, Grounded Theory and Case Study. The characteristics of each method. Concepts in Cognitive mapping.
2 The design, data collection, field work and Analysis strategies of qualitative research. SWOT Analysis of each method. Utilisation of these methods in medical education.
3 Quality in qualitative research. General Characteristics of Data collection methods such as questionnaire, Interviews, Focus group discussions and Observations. How can these methods be utilised in Educational Research
4 Types of data such as video recordings, Audio recordings, unstructured text. Common questions and doubts expressed against qualitative research such as is it scientific, Are the findings generalizable, effect of presence of researcher on the observations etc.
5 What are the mixed methods? How effectively qualitative and quantitative methods can be amalgamated.
Qualitative research relies primarily on the collection of qualitative data (i.e.,
nonnumeric data such as words and pictures). There are four major types of
qualitative research:
• Phenomenology.
• Ethnography.
• Grounded theory.
• Case study.
1 Phenomenology
Page | 72
Phenomenology is a descriptive study of how individuals experience a
phenomenon. In this , researcher study what is the meaning, structure, and
essence of the lived experience of this phenomenon by an individual or by
many individuals. The researcher tries to gain access to individuals' life-
worlds, which is their world of experience; it is where consciousness exists.
Conducting in-depth interviews is a common method for gaining access to
individuals' life- worlds. The researcher, next, searches for the invariant
structures of individuals' experiences. Phenomenological researchers often
search for commonalities across individuals (rather than only focusing on
what is unique to a single individual). For example, what are the essences of
peoples' experience of the death of a loved one?
After analysing the phenomenological research data, a report has to be
documented which provides rich description and a "vicarious experience" of
being there for the reader of the report.
2 Ethnography
Ethnography is the discovery and description of the culture of a group of
people. Here is the foundational question in ethnography: What are the
cultural characteristics of this group of people or of this cultural scene?
Because ethnography originates in the discipline of Anthropology, the
concept of culture is of central importance. Culture is the system of shared
beliefs, values, practices, language, norms, rituals, and material things that
group members use to understand their world. One can study micro
cultures (e.g., such as the culture in a classroom) as well as macro cultures
(e.g., such as the Pawra or bhil tribal culture in nandurbar district.).
There are two additional or specialized types of ethnography.
1. Ethnology (the comparative study of cultural groups).
2. Ethnohistory (the study of the cultural past of a group of people). An
ethnohistory is often done in the early stages of a standard
ethnography in order to get a sense of the group's cultural history.
Page | 73
The final ethnography (i.e., the report) should provide a rich and holistic
description of the culture of the group under study.
3 Case Study Research
Case study research is the detailed account and analysis of one or more
cases. Here is the foundational question in case study research: What are
the characteristics of this single case or of these comparison cases? A case
is a bounded system (e.g., a person, a group, an activity, a process). Because
the roots of case study are interdisciplinary, many different concepts and
theories can be used to describe and explain the case.
Robert Stake classifies case study research into three types:
1. Intrinsic case study (where the interest is only in understanding the
particulars of the case).
2. Instrumental case study (where the interest is in understanding
something more general than the case).
3. Collective case study (where interest is in studying and comparing
multiple cases in a single research study).
Multiple methods of data collection are often used in case study research
(e.g., interviews, observation, documents, questionnaires). The case study
final report should provide a rich (i.e., vivid and detailed) and holistic (i.e.,
describes the whole and its parts) description of the case and its context.
4 Grounded Theory
Grounded theory is the development of inductive, "bottom-up," theory that is
"grounded" directly in the empirical data. Here is the foundational question
in grounded theory: What theory or explanation emerges from an analysis of
the data collected about this phenomenon? It is usually used to generate
theory. Grounded theory can also be used to test or elaborate upon
previously grounded theories, as long as the approach continues to be one of
constantly grounding any changes in the new data.
Four important characteristics of a grounded theory are
• Fit (i.e., Does the theory correspond to real-world data?),
• Understanding (i.e., Is the theory clear and understandable?),
Page | 74
• Generality (i.e., Is the theory abstract enough to move beyond the
specifics in the original research study?),
• Control (i.e., Can the theory be applied to produce real-world results?).
Data collection and analysis continue throughout the study. When collecting
and analyzing the researcher needs theoretical sensitivity (i.e., being
sensitive about what data are important in developing the grounded theory).
Data analysis often follows three steps:
1. Open coding (i.e., reading transcripts line-by- line and identifying and
coding the concepts found in the data).
2. Axial coding (i.e., organizing the concepts and making them more
abstract).
3. Selective coding (i.e., focusing on the main ideas, developing the story,
and finalizing the grounded theory).
The grounded theory process is "complete" when theoretical saturation
occurs (i.e., when no new concepts are emerging from the data and the
theory is well validated). The final report should include a detailed and clear
description of the grounded theory.
Mixed Research:
Mixed research is research in which quantitative and qualitative techniques
are mixed in a single study. Proponents of mixed research typically adhere
to the compatibility thesis as well as to the philosophy of pragmatism. The
compatibility thesis is the idea that quantitative and qualitative methods are
compatible, that is, they can both be used in a single research study. The
philosophy of pragmatism says that researchers should use the approach or
mixture of approaches that works the best in a real world situation. In
short, what works is what is useful and should be used, regardless of any
philosophical assumptions, paradigmatic assumptions, or any other type of
assumptions.
Today, proponents of mixed research attempt to use what is called the
fundamental principle of mixed research. According to this fundamental
principle, the researcher should use a mixture or combination of methods
that has complementary strengths and non-overlapping weaknesses.
Page | 75
Quantitative research Qualitative research
Strengths
Already constructed theories Data based on participants meaning
Already constructed hypothesis Useful to describe complex phenomenon
Random samples, sufficient size
generate findings
Provide individual case information
Quantitative predictions possible Provides insider‟s view point
Confounders eliminated Dynamic process is documented
Data collection is quick Data collected in naturalistic setting
Precise quantitative data Data useful to generate hypothesis/
theory
Analysis less time consuming
Researcher independent results
Higher credibility
Large population can be studied
Weaknesses
Local constituents may not
understand
May not be generalizable
Focussing on hypothesis might miss
some phenomenon
Difficult to quantitative prediction
Abstract knowledge Difficult to test hypothesis/ theory
Low credibility
Data analysis time consuming
Researcher bias can influence results
Data collection takes more time
Mixed Research
Strengths Weaknesses
Words, pictures, narrative add
meaning to numbers
Difficult for single researcher to
conduct qualitative & quantitative
Can have strengths of both
qualitative and qualitative methods
More exepensive
Page | 76
Can generate and test theory More time consuming
Can answer broad range of questions Complex analysis
Can provide stronger evidence
Increases generalizability
The Research Continuum
Research can be viewed as falling along a research continuum with “mono
method” research placed on the far left side, “fully mixed” research placed
on the far right side, and “partially mixed” located in the center.
Types of Mixed Research Methods
There are two major types of mixed research: they are mixed model research
and mixed method research.
Mixed Model Research : In this type, quantitative and qualitative
approaches are mixed within or across the stages of the research process.
Here are the two mixed model research subtypes: within-stage and across-
stage mixed model research.
1. In within-stage mixed model research, quantitative and qualitative
approaches are mixed within one or more of the stages of research. An
example of within-stage mixed model research would be where a
questionnaire during data collection that included both open-ended (i.e.,
qualitative) questions and closed-ended (i.e., quantitative) questions.
2. In across-stage mixed model research, quantitative and qualitative
approaches are mixed across at least two of the stages of research. A
researcher wants to explore (qualitative objective) why people take on-line
Mixed Research
Collect Qualitative data
Collect Quantitative
Data
Perform Qualitative
Analysis
Perform quantitative
Analysis
Perform Qualitative
Analysis
Perform quantitative
Analysis
Page | 77
college courses. The researcher conducts open-ended interviews (qualitative
data collection) asking them why they take on-line courses, and then the
researcher quantifies the results by counting the number of times each type
of response occurs (quantitative data analysis); the researcher also reports
the responses as percentages and examines the relationships between sets
of categories or variables through the use of contingency tables.
Mixed Method Research
In this method, a qualitative phase and a quantitative phase are included in
the overall research study. It‟s like including a quantitative mini-study and a
qualitative mini-study in one overall research study.
Mixed method research designs are classified according to two major
dimensions:
1. Time order (i.e., concurrent versus sequential) and
2. Paradigm emphasis (i.e., equal status versus dominant status).
Stages of Mixed Research Process
There are eight stages in the mixed research process. It is important to note
that although the steps in mixed research are numbered, researchers often
follow these steps in different orders, depending on what particular needs
and concerns arise or emerge during a particular research study. For
example, interpretation and validation of the data should be done
throughout the data collection process.
Page | 78
(1) Determine whether a mixed design is appropriate
(2) Determine the rationale for using a mixed design
Rationale for Mixed Research
Purpose Explanation
Triangulation Seeks convergence, corroboration of results from
different methods
Complementary Seeks elaboration, enhancement, illustration and
clarification of results from one method to the other
Development Seeks the results of one method to improve, develop
the other method
Initiation Seeks new perspectives
Expansion Seeks expansion of results by different methods
(3) Select the mixed method or mixed model research design
(4) Collect the data
(5) Analyze the data
(6) Validate the data
(7) Interpret the data
(8) Write the research report.
In conclusion, mixed research is the newest research paradigm in
educational research. It offers much promise, and we expect to see much
more methodological work and discussion about mixed research in the
future as more researchers and book authors become aware of this
important approach to empirical research
Page | 79
Lesson VI:
Qualitative Techniques and Computer aided Analysis
Objectives:
1 Enumerate the various qualitative methods commonly practiced in anthropology
2 How these methods can be applied to medical education with particular reference to under-graduate medical education
3 Enumerate various qualitative data analysis software and their familiarity Lesson Outline:
1 Qualitative methods include Participatory techniques, in-depth techniques and systematic techniques. The participatory techniques are valuable and popular. Identifying the health resources, drawing socio-cultural relationships, mapping of health needs, transects etc are utilised by the educationalists for teaching and learning in medical curriculum.
2 These methods can be either demonstrated in the field practice area of
department of community medicine. Or it can be practiced in the hospital
3 Anthropac, Answer, Atlas-ti are some of the qualitative freeware available
for qualitative data analysis.
Introduction:
Qualitative research is type of formative research that includes specialized
techniques for obtaining „in-depth responses‟ from respondents. Qualitative
research is often conducted to answer the question - why? A purpose of qualitative
research is the construction (not the discovery) of new understanding. At present
there is a revival in the qualitative methods. The reasons for this revival of interest
in qualitative Methods are depicted below. 1) Growing realization of unsuitability of
survey research methods in the context of developing countries where population is
predominantly oral and illiterate and where magnitude of non-sampling errors is
high in surveys. 2) Increased interdisciplinary team work. 3) Demand of quick
results from the ethnographic work.
The latest trend in the field of research is the combined use of quantitative and
qualitative research methods i.e. mixed-method design within a single data set.
According to Morse (2005), it is in this area that the largest abuses of qualitative
data are occurring, largely because methodological principles have not been
followed.
Types of Qualitative Methods:
Page | 80
The qualitative data collection techniques range from the highly structured
systematic techniques to the highly flexible people-centered participatory
techniques.
Participatory research (PR) techniques
In-depth techniques
Systematic techniques
1 Participatory research (PR) Techniques: In conventional research, knowledge is generated by the researchers and the study
subjects have no control over it. PR process intends to change existing local
problems and synthesize local people‟s knowledge with existing scientific
knowledge. PR involves people as „stakeholders‟ for their empowerment. It assumes
that the ordinary people already possess knowledge and have an understanding of
their reality.
Classification of PR Tools: Space-related PRA applications: Social mapping, transect, mobility map,
mapping
Time-related PRA applications: Daily activity schedule, seasonal diagram,
trend analysis
Relational PRA applications: Venn diagram, spider diagram, force field
analysis, pair wise ranking
Application of PR in Medical Education:
E.g. Transect walk as Public health teaching-learning tool
Social Mapping exercise for medical students, Pulai
Participatory mapping of stool positive cases
Venn Diagram with the group of students
Participatory Group work
E.g. Force Field Analysis (FFA) with medical undergraduates to explore pressing
issues in their academic life
Further resources/ Reading:
Training in Participation Series [PRA tips on CD-ROM]. Patna (India): Institute for Participatory Practices; 2004.
Rajesh Tandon (Ed). Participatory Research: Revisiting the Roots. New Delhi (India), Mosaic books; 2005.
Page | 81
2 In Depth Techniques:
These are qualitative in-depth flexible discussions or interviews with the group or
person who knows what is going in community about the topic on which we want
to get information. Some commonly used methods are Focus Group Discussion
(FGD), Key Informant Interviews (KII) and In-depth Interview (IDI).
Application of in-depth techniques in medical education
Formative exploration of students‟ perception about Community Medicine teaching at Mahatma Gandhi Institute of Medical Sciences, Sewagram, India. Online J Health Allied Scs. 2008;7(3):2. Link: http://www.ojhas.org/issue27/2008-3-2.htm
Portfolio based approach for teaching public health among medical under-graduates and assessment of their learning in a Medical college of rural India
3 Systematic Techniques:
These techniques can be used with almost any qualitative research methods such
as focus group or participatory research to collect systematic and structured data.
Page | 82
Principle - Respondent make sense of their words by grouping their observation or
experiences in class known as domain. Examples - Free listing, Pile sorting, Delphi
panel etc. Free list combined with pile sort can be used for systematic exploration
of the perceptions of respondents on a given research topic.
Application of Systematic Techniques in Research
Process Documentation of Health Education Interventions for School Children and Adolescent Girls in Rural India. Education for Health, Volume 22, issue 1, 2009.
Available from: http://www.educationforhealth.net/
Eliminating Childhood Malnutrition : Discussions with Mothers and
Anganwadi Workers. Journal of Health Studies / I: 2,3 / May - Dec. 2008.
Available from: http://www.esocialsciences.com/essJournals/ essJournalIssuesMain.asp?jid=1&issue=Current
Sample Size and Sampling Techniques:
Sample size: No mathematical formula to calculate sample size in qualitative
research. The validity, meaningfulness and insights generated from the qualitative
data have more to do with the richness of data obtained. The process of data
collection is continued till the saturation point i.e. where no new information is
added after additional interviews or focus group discussions.
Sampling Techniques:
1) Purposive sampling, where sample units are selected with definite purpose in
view, e.g. victims of some events etc.
2) Convenient sampling, where the conveniently available respondents are
selected, e.g. participants of camp or workshop
3) Quota sampling is a restricted type of convenient or purposive sampling
defining the quota of sample to be drawn from different strata and then drawing
the required sample.
Page | 83
4) In Snow-ball sampling, the sample is driven by the respondents. It is used when
the target population is unknown or difficult to approach, e.g. such as MSM
population, Sex workers etc.
Sequencing of the Methods:
The qualitative data collection should be „on-going‟ or „iterative‟ process where one
method directs the other. In mixed methods design, quantitative (survey) and
qualitative methods are used in same research design. Pre-survey qualitative
research: for better pre understanding of the underlying dynamics, for exploring
local terms. Post- survey qualitative research: to bridge the gaps of information in
survey.
Analysis of Qualitative Data:
It is a multi-faceted endeavor. It requires planning, capacity for being open to views
that that are different from your very own, an appreciation of provisional nature of
human knowledge, strong conceptual skills and excellent scholarship
Some Commonly Used Terms:
Interim analysis: On-going and iterative (non-linear) process. Interim analysis
continues until the process or topic the researcher is interested in is understood.
Coding: It is defined as making the segments of data with symbols, descriptive
words or category name.
Memo: It is recording reflective notes about what you are learning from your data.
Include those memos as „additional data‟ to be analyzed.
Content analysis: Subjective interpretation of content of text data through the
systematic classification process of coding and identifying themes or patterns.
Steps in the process of Content Analysis:
Step1: Transcription: The raw data need to be transformed into written text
format before analysis.
Step 2: Deciding the unit of analysis: Defining the coding unit is one of the most
fundamental and important step. The commonly used coding units are word,
concept, sentence, paragraph and theme.
Step 3: From units to categories: categories and code schemes can be derived
from three sources such as a) data itself, 2) previous related studies, 3) theories.
Step 4: Test coding on sample test: If there is low inter-coder agreement then
revise the rules of coding sample text and checking coding consistency.
Step 5: Code all text data: When sufficient consistency is achieved then coding
rules can be applied to code all text data.
Page | 84
Step 6: Assess the coding consistency: After coding all text data, coding
consistency needs to be re-checked. Human coders are subject to fatigue and are
likely to make mistakes as coding proceeds.
Step 7: Drawing conclusions from the coded data: This is a crux of qualitative
data analysis. It involves reading and re-reading of text data. The activities involve
exploring properties and dimensions of categories and identify relationships
between categories.
Step 8: Reporting: While writing report it is important to maintain the balance
between description and interpretation. Here, one can use conceptual frameworks
derived from the data set.
Methods to ensure Validity in Qualitative Research:
Researcher as detective: The researcher has to develop the understanding of the
data through careful consideration of potential causes and effects by systematically
eliminating the rival explanations and hypothesis until the final cause is made
beyond a reasonable doubt.
Extended field work: The researcher should collect data in the field over the
extended period of time.
Low-inference descriptors: The use of descriptions phrased very close to the
participant‟s account or researcher‟s field notes. Verbatim i.e. direct quotations are
used as low-inference descriptors.
Triangulation: A combination of multiple methods, multiple investigators to collect
and interpret data adds to the validity of the results.
Participant feedback: The feedback and discussion on the researcher‟s
interpretation and conclusions with actual participants and other members of the
community helps in verification and better insight into the research problem.
Peer-review: Discuss the findings with the disinterested peer e.g. other researcher
who is not directly involved. Peer should be skeptical and play the devil‟s advocate,
challenging the researcher to provide solid evidence for any interpretation or
conclusion
Use of Software in the Analysis of Qualitative Data:
For smaller data set „manual content analysis‟ is undertaken. Here, coding is done
manually along a narrow blank column of the text document. A computer assisted
coding using software packages has significantly reduced the need for the
traditional filing technique. The most popular qualitative data analysis packages
are ATLAS-ti and Anthropac
Page | 85
Lesson VII:
Focus Group Discussions: Participatory and Non-Participatory techniques of
Qualitative Data Collection in Medical Education
Objectives:
1. Describe the merits and demerits of Focus group discussions 2. Understand the method of conducting Focus group and small group
Discussions. 3. Understand the method of Participatory Learning Appraisal(PLA) 4. Describe the concepts of non participatory educational techniques like Objective
structured examinations/assessments
Lesson Outline:
1 Definition and Characteristics of Focus Group. When to use Focus Groups. Role in Educational Research. What are advantages and disadvantages of FGD
2 How to conduct the discussions and the ground rules. Purpose, Preparations and Process. Recording of Focus Group Data.
3 Triangulation of Qualitative Research (Team members, Participants and Methods). Other methods of Participatory (Interviews, Venn Diagram, Pile sorting, Transects, Direct Observation (Walkabout), Portfolio, Participatory Learning for Assessment, Rapid Assessments, case/ event narratives, Role Play) and Non-participatory techniques(Time Line, Free Listing, Priority Matrix, Objective Structured assessments ). Details of purpose, preparation and process of Participatory Learning for Assessment.
4 Details of conducting the Objective structured Assessments both clinical and practical (OSCE,OSPE)
What is Focus Group Discussion? It is a technique of gathering data and insights from discussions and interactions among participants in a group, facilitated by a moderator . It promotes exchange of ideas among participants and is focused, but flexibly structured discussion. It is ideal for exploring norms, expectations, values and beliefs and NOT personal experiences. FGD is a group discussion of approximately 6-12 persons guided by a facilitator, during which the members of the group talk freely and spontaneously about a certain topic. It thus, aims to be more than question-answer interaction
where in the members are also encouraged to discuss the topic among themselves. When to use FDG:
o Focus research & develop relevant research hypotheses o Formulate appropriate questions for more structure and large scale surveys o Help understand and solve unexpected problems in interventions o Develop appropriate messages for health education programs o Explore controversial topics
Why Focus Group?
o Defining the research concept o Developing hypothesis o Generating Vocabularies
Page | 86
o Framing a questions in large scale surveys o Providing supplementary information related to the community's
knowledge, beliefs, attitudes and behavior on specific issues Composition and Selection of Participants: Select 8-10 individuals in a group willing participants NOT individuals who will dominate the discussion or inhibit the participation of others in the group. Participants are selected in advance by either random sampling or by any alternative criteria. The members are homogenous viz. regarding major social divisions and/or background characteristics. Age and sex often considered for assigning participants into different groups. Inform the participants about the topic of exploration through personal experience or interest arising from a particular role or position. The date, time and venue of the FGD is fixed in advance. A time limit of one and one-half hours is desirable and two hours is the maximum. Anonymity of the participants is preferred Members of the research team: a moderator (facilitator), a note-taker & recorder Guidelines for the FDG Participants: One participants speak at one time and clearly. Try gathering everyone‟s perspective/opinion and encouraging participation. Process of FDG: FGD guidelines are to be pre-tested in advance. More than one FGD is to be conducted Moderator /note-takers should be trained in advance. In recruitment of the participants take help from key informant so that homogeneity can be maintained. The process need to be recorded in addition to the routine note-taking. Ideally FGD should be of 90 minutes duration. Make physical arrangements for setting, equipment, food and drinks, and child care if necessary. Select the location and time for FGD. Essential Steps: Starting the Discussion
• Collect socio-demographic details informally • Summarize the purpose of the study • Describe the focus group discussion process
- No right or wrong answers - All should participate - All should respect the opinions of others
• Make sure everyone understood the informed consent • Ask participants to guard the confidentiality of others in the groups
Conducting the Discussion • Begin with warm-up questions • Be aware of who is talking and who is not
- Do not allow one or two individuals to dominate • Use broad, open-ended questions
- Avoid yes or no or short answer questions • Always probe • Record body language, nonverbal communication
Documenting the Discussion • Expand notes of the discussion • Record in writing any nonverbal data • Don‟t imply judgement • Add researcher‟s comments in parentheses • Finalise field notes as soon as possible after the discussion • If tape recorder is used, complement taped transcripts with field notes in
preparing final transcripts
Page | 87
Role of Moderator
• Introduce the session • Encourage discussion • Creating a climate for open exchange
- explaining the goal of discussion - setting ground rules - encouraging participation by all
• Guiding the discussion - introducing topics with main questions - eliciting detailed information with follow up questions - probing meaning of responses - Should not dominate the discussion
• Keeping the discussion focused • Encourage involvement of every member
• Monitor involvement & interaction among participants • While maintaining the core theme of the discussion ensure flow of
conversation. • Avoid being placed in the role of expert • Control of rhythm of the meeting, but avoid an unobtrusive way • Take time to end the meeting to summarize and check for agreement /
disagreement on important topics • Build rapport, empathize • Thank each of the participants personally for their participation • Monitor involvement & interaction among participants • Encourage involvement of every member • While maintaining the core theme of the discussion ensure flow of
conversation. Role of Note Taker/ Recorder Items to be recorded
• Date, time and place • Number, names and description of each participant • General description of group dynamics (level of participation, presence of
dominant participant, level of interest etc.) including non-verbal interaction among the participants
• Opinions of the participants including key statements • Emotional aspects (reluctance, strong feelings attached to a certain topic)
including any non-verbal communication • Taking notes without disturbing the discussion including identity of the
speakers • Spontaneous relevant discussion during breaks or after the formal session /
discussion • Works as back up to the moderator by drawing attention to missed
comments from participants and missed topics Designing the Interview Guide for FGD
o Must provide the moderator with he topics and issues that are, to the extent possible, to be covered at some point during the discussions
o The guide is loosely structure and does not suggest potential responses. o The questions should be unstructured, unbiased and non-threatening o Progression of the topics in the guidelines should be logical and should
move from general topic to specific topic
Page | 88
o The guide should not overly done or have too many questions (preferably should have no more than 20 questions / topics)
o Pretesting of guidelines with several mock sessions is essential Strengths and Limitations of FGD
o Should not be used for quantitative purpose, e.g. the testing of hypotheses or generalization of findings for larger areas that may need more elaborate surveys
o FGD can be used to complement findings from the surveys and other qualitative techniques as using it alone may be risky as the people tend to centre their opinion on the most common ones on Social Norms.
o FGD may not be very useful on sensitive topics where members may hesitate to air their feelings and experiences freely (sexual behavior/HIV AIDS)
o Evaluator has less control than individual interview o Groups are often difficult to assemble
Do‟s and Don‟ts of Moderator
Dos Don‟ts
Make everyone feel welcome Speak in a loud and clear voice Be flexible Include everyone in the discussion Leave enough time for people to answer question (enjoy the silence) Vary your style of asking questions to get a variety of answers Probe for clarity Allow diverse opinion to emerge
Talk too much Let one person dominate Fail to stay neutral on the issue More than one question at a time Ask „Yes‟ or „No‟ questions (instead ask open ended questions) Go over the allotted time Forget to thank people for participating
Exercise:
• Select a study topic • Prepare three questions for FGD discussion
Key Informant‟s Interview: Definition
• “Qualitative, in-depth, flexible interviews with persons who know what is going in the community, “experts” (knowledgeable) about a topic on which we want to get information.”
• Note that the key informant interview is usually not about that person herself, but about the topic on which she has information.
Objective: • The purpose of Key Informant Interviews is to collect information from a wide
range of people- including community leaders, professionals, or residents-who have first hand knowledge about the community , and our research topic.
• To get general information about the local community • These community experts, with their particular knowledge and
understanding, can provide insight on the nature of problems and give recommendations for solution.
When to Conduct Key Informant Interview:
Page | 89
• To get more candid or in-depth answers. Focus Group dynamic may prohibit you from candidly discussing sensitive issues or getting the depth of information you need. Sometimes group dynamic can prevent some participants from voicing their opinions about sensitive topics
Choosing Key informants: • KI must have first- hand knowledge about community, its residents and
issues or problems you are trying to investigate • KI can be a wide range of people, agency representatives, community
residents, community leaders, or community business owners. • ex., Religious leaders, government officials, young mothers, youth, minority
population etc. • Should have a diverse mix of key informants to ensure variety of
perspectives Exercise:
• Select a study topic • Identify key informant for your research topic
Page | 90
Lesson VIII:
Descriptive & Inferential Statistics
Objectives:
1. Understand the concepts in Frequency distribution, Measures of central tendency, measures of variability.
2. Describe the various sampling distributions and Procedures used in educational research ( purposive, opportunistic, critical case)
3. Able to conduct Hypothesis testing. 4. Understand the concept of t-test, Analysis of Variance and Chi-square tests.
Lesson Outline:
1 Basics of Frequency Distribution both tabular and graphic representations. Comparison of Mean, Median, Mode. Utility of Percentiles in education. Normal and Skewed curves. Rankings. Z-Scores and Regression Analysis.
2 Types of Samples used in education for qualitative research. Purposive sampling, Random Sampling. Opportunistic and Critical case sample. How to calculate sample size through computers.
3 What is null and alternate hypothesis? How they can be constructed for educational studies. What is the probability value and Significance level. Various steps involved in Hypothesis testing.
4 Basics on performing t-test and Analysis of Variance (both one way and two way). Utilisation of x2 test for contingency tables.
Descriptive and inferential statistics
A field of statistics can be divided into Descriptive and Inferential statistics.
Flow charts are as below Statistics
Descriptive Inferential
Estimation Hypothesis
Testing
Point Interval
Descriptive statistics: to describe, Summarize and explain the data.
Page | 91
How to prepare the data set?
A data set with the “cases” going down the row and the “Variable” going
across the columns.
Once you put your data set into a statistical software Programme such as
SPSS, Minitab, Epi-info, SAS etc., you are ready to obtain all the descriptive
statistics.
In descriptive statistics you will get
Inferential statistics: is the branch of statistics that is used to make inference
about the characteristics of a population based on sample data.
Medical Uncertainty: Uncertainties arising from various kinds of variations, lack
of knowledge, partial compliance, errors etc., in medical decision process.
A surgeon carried out the major surgery of his career and was successful. He
was quite uncertain about the success of the second. Uncertainty level
decreased as the number of successful surgeries piled up, Naturally! All of 1st
10 major surgeries were successful. An amazing feat indeed. The Surgeon was
certain that the Eleventh too would be successful. But it failed for no apparent
reason. This could be one of those unlucky flukes that can always occur in
Medical Sciences due to biological and other variations. The Surgeon had not
cared to examine the statistical chances. If the long-term failure rate is as high
as 25%, there is still a significant chance that all 10 surgeries would be
successful. Failure in the eleventh was not such a surprise after all at least
statistically.
Biostatistics
There are many fields of applied statistics depending on the science where it is
applied. Biostatistics is the branch of statistics applied to biological or medical
sciences. It is also called biometry. The Greek roots are bios (life) and metron
(measured); hence biometry means measurement of life. It may be stated as the
application of statistical methods to the solution of biological problems.
Biostatistics covers applications and contributions not only from health,
medicine and nutrition but also from fields such as agriculture, genetics,
biology, biochemistry, demography, epidemiology, anthropology and many
others. Biostatistics today has a wide coverage of applications.
Application of statistical methods in biological sciences, which include Medical
and Dental sciences. Or The science dealing with medical uncertainties – their
identification, measurement, and control, leading to decision with less error.
Page | 92
Sources of Medical Uncertainties:
This can be divided into three broad groups.
1. Genuine variability:
This arising from natural variations due to Biological, Environmental, and
Sampling factor.
a. Biological variability:
*The primary source of uncertainty in health and disease is variations in
biological characteristics among individuals. You know that, every person is
unique because of decreasing morphological features.
*Individuals also vary according to age and sex, height and weight, heredity and
parity etc. They are also vary with regard to parameters such as blood glucose level,
cholesterol level and creatinine excretion.
b. Environmental variability:
* The environmental factors give variability of these biological characteristics.
*Unclean water and lack of sanitation are responsible for diseases such as Typhoid,
Cholera, and Polio. Goitre is found in area with Iodine deficiency.
*The environmental factors affect not only the incidence of various diseases but
also the patient management methods.
Chance variability:
*Identical twins born at the same time they may have different birth weight. How do
you explain such a variation?
*Nobody knows why one set of parents has three daughters and the other has three
sons. „Chance‟ determines the gender of the unborn child.
*If you repeatedly analyze same blood sample two or more times in the same
laboratory with same method and reagents, there could still be some variation,
although this could be minor.
2. Variability due to Unstandardized Methods (Experimental
variability):
Variation occur evening case of perfectly executed standardized methods. Also
realize that it is not all that easy to attain complete standardization.
Unstandardized methods and procedures cause another type of uncertainty.
a. Observer Variability:
We can say with confidence that physician in India dramatically differ on the
average duration of survival after detection of HIV infection. This varies from
Page | 93
patient to patient but the average should not vary too much. Eg. One physician
insisting that the average survival is just 3 years and the other insisting that it is
10 years.
b. Instrumental Variability / Error:
Pain intensity measured by visual analogue scale can differ from visual rating
scale.
c. Laboratory Variability:
For inter-laboratory variability, suppose we split the same blood sample into two
bottle and send to two different laboratories for estimation of Hb / lipoprotein (a) /
CBC etc. One laboratory reported 12 mg./dl and other 14 mg./dl. Both used the
same technique. Possibly the quality of chemicals and reagents was not the same.
Above all, the human skill in the two laboratories could be vary different.
3. Variability due to Partial Information:
a. Unavoidable incomplete information:
We have already discussed sources of uncertainty that are genuine and can be
avoided. Incomplete information on a patient can be due to carelessness of the
assessor but for the time being we are concern with unavoidable situation. Some
time this can happen eg. When a patient comes in coma after an injury, in this
case you have to observe the case on this basis you have to manage the case, there
is not much that you can obtain by way of history.
b. Avoidable incomplete information:
Needless to say that you would like to have as complete information on a patient as
possible before advising him. In an OPD of a crowed hospital where 100 patients
are attended in a three-four period, you will not have time to go into details even if
there are interns and residents to assist. Prescriptions are made on the basis of
incomplete information that can produce uncertain results. Sometime you may like
to get an investigation done such as thyroid function test or CT Scan etc. but this
is not done because it is either to expensive for the patient or simply the facility is
not available.
It is not uncommon that patients intentionally suppress part of the history as in
the case of sexually transmitted diseases (STDs). In some cases you might forget to
ask about a vital aspect or consider at that time that this is not necessary.
c. Partial Compliance:
Medical uncertainties seem to never end. If a patient is properly assessed for his
conditions and adequately prescribed, it is difficult to ensure that he is following
the complete regimen. Incomplete therapy in the case of tuberculosis is very well
Page | 94
known, which causes its resurgence. In a case of myocardial infarction (MI), the
patient may not follow, fully or partially, advice on exercise, dietary restrictions,
and drug intake. Hospital patient may fail to take rest as advised. The response
could accordingly vary, and the outcomes remain uncertain.
Sources of Health Data:
1. Primary: Data generate with specific objective from subject under study. or Data
collected for specific purpose directly from the field of enquiry and are original in
nature. This data gives detailed information and very less errors.
a. Experiments: It is performed in the laboratories of physiology, biochemistry,
pharmacology, clinical pathology, hospital wards, in the community etc. for
investigation and fundamental research.
b. Surveys: It is carried out for epidemiological studies in the field by trained team.
Eg. Census, Population survey, Disease survey etc.
2. Secondary Data: Data generated as a routine administrative procedure. or
When we collect the data from reports which are already published for some other
purposes which may be a processed one. While collecting secondary data we have
to observe that the source must be reliable.
a. Records / Registers: It is maintained as a routine in registers or books over a
long period of time for various purposes such as vital statistics.
b. Publications: Reports of various national, international health agencies,
scientific journal papers, books etc.
Random Number: The functions define by sample space to real number. Eg. A
couple may decide to stop the reproduction till they may get the male/female child.
Variable: Any character that varies. Or Variables are those characteristics or
attributes that varies from person to person, from time to time, or from Place to
place.
Types of Data:
There are two types of data: Qualitative & Quantitative
Scale of Measurement:
Qualitative variable are measured either on a Nominal or an Ordinal scale and
quantitative variable are measured on an Interval or Ratio scale.
1. Nominal scale: Observations are placed into broad categories, which may be
denoted, by symbols or labels or names or divide qualities into two types. One that
can be graded and the other cannot be graded.
Eg. Diagnostic groups like cancer heart disease etc. site of malignancy such as
lung, mouth, breast or ovary have no order complaints such as pain; vomiting and
Page | 95
constipation have no order among themselves. You can write constipation before
vomiting. Characteristics or attributes with this feature are said to be on nominal
scale. Eg.
i. Disease as present / absent,
ii. Sex as male / female,
iii. Occupation as farming / business / labour / service, etc. and
iv. Diagnosis as hepatitis / cirrhosis / malignancy, etc.
2. Ordinal scale: Categories are ranked or ordered/graded. Each category is in
unique position in relation to other categories but distances between the categories
are not known. Eg. Severity of illness such as cancer is graded as stage I, II, III, IV.
Inability to do routine work of life can be graded as none, mild, moderate, and
serious. There is a definite order in these grades. Try writing them as moderate,
serious and mild and see how awkward do you feel. Such measurements are said
to be ordinal scale.Social status as low, middle and high. Age as child, adult and
old. Health as poor, fair, good and excellent.
It is used in two situations:
a. Hypertension measured as mild, moderate, and serious, although blood pressure
can be measured exactly as a quantity. The ordinal scale is adopted in such
situations for convenience.
b. When quantitative scale is not available or is extremely difficult to adopt stage of
cancer comes under this category.
Suppose you are a medical superintendent of a hospital and want to know how
much the patients are satisfied with the service of the hospital. This can be graded
as fully satisfied, partially satisfied, and not satisfied.
3. Metric scale: When a characteristic is measured exactly in terms of quantity, it
is said to be measure on a metric scale. Eg. Duration of disease, body temperature,
pulse rate, and number of deaths in one year.
Interval scale: Distance between any two number (values of the variable) is fixed
and equal. The origin is arbitrary. Eg. Temperature in C or F. Can you say that
body temperature of 102 F is double of 51 F? Even for whether, 0 C does not mean
no temperature such measurement are said to be on interval scale.
Ratio scale: In addition to interval scale / level of measurement, it has true zero
point as its origin. Eg. Weight in kg, or pounds, height in Cms, or Kgs. Weight 30
kg is triple of 10 kg. , Parity is on ratio scale because there is absolute zero and
parity 4 is double of parity 2. Measurements on ratio scale are easy to handle.
Page | 96
Since you can add, multiply or divide. This convenience is not easily available with
measurements on interval scale.
Methods of Presentation:
The three basic method of summarization are:
1. Tabular, 2. Graphic / Drawing,
1. Tabulation: It consists in a systematic arrangement of data in rows and
columns.
Rules: Tabulation can develop by experience. No hard and fast rule can be stated
for satisfactory tabulation.
a. A table should be a brief but self-explanatory title, which can answer what, when
and where about the data.
b. Heading of rows and columns should be clearly stated.
c. Unit should be mention whenever necessary.
d. Avoid short forms as far as possible and large figure should be abbreviated.
e. Classes and subclasses should be clearly separated by lines.
f. Figures to be compared should be placed in neighboring columns.
g. Explanation of sign, rounding and abbreviation of figure etc. should be given in
footnote and reference or sources of data should be given in source note.
Eg. Sexwise prevalence of carriers of filaria in Miraj during the year 2003.
Particulars Sex Total
Male Female
Carriers
Non-
carriers
Total
Footnote: - -Source note: -
The above example of qualitative data. The presentation of frequency is very small
because the characteristic is not variable.
Simple table or One way classification
Eg. Distribution of girls by timing of menarche.
Timing of
menarche
Number of
girls
Early (<11Yrs) 80
Average (11-14Yrs) 140
Page | 97
Late (>=14Yrs) 40
Total 260
Eg. Distribution of girls by birth order.
Birth order Number of girls
1 60
2 80
3 40
4+ 80
Total 260
Cross tabulation or Two-way classification.
A frequency table involving at least two variable that have been cross – classified
(tabulated against each other).
Eg. Distribution of girls by timing of menarche and birth order.
Timing of
menarche
Birth order Total
1 2 3 4+
Early (<11Yrs) 20 20 15 25 80
Average (11-14Yrs) 30 50 20 40 140
Late (>=14Yrs) 10 10 5 15 40
Total 60 80 40 80 260
Contingency table: When the tabulation is in mutually exclusive and exhaustive
categories. Eg. Mutually exclusive means that one person can belong to only one
category. Or a girl can have timing of menarche early, average or late and not two
together. If the categories are symptoms such as pain in abdomen, vomiting and
constipation, one person can have two or three of these together. They are not
mutually exclusive.
Eg. Exhaustive means that all possible categories are included. If the 4th row in
eg.4 is for birth order 4 and not 4+, the categories would not be exhaustive.
2. In quantitative data: The characteristics and frequency are both variable.
Frequency table or frequency distribution table:
A table in which the value of a variable are classified according to size.
Page | 98
Rules:
a. The class or group interval between the class or group should not too broad or
too narrow.
b. The number of groups should not be too many or to few but ordinarily between 8
and 20.
c. The class interval should be same throughout.
d. Heading must be clear.
e. The rates and proportions are given mentioned in the denominator.
f. Group should be tabulated in order, from lowest to highest value in the range.
g. If certain data are omitted or excluded, reason for the same should be given.
Eg. Number of diagnosed cases of TB by age in Miraj during the year 2003.
Age in (Yrs.) Number of
cases
0 – 04 1242
05 – 14 1081
15 – 24 2482
25 – 44 8153
45 – 64 10916
65 + 7124
Total 30998
Footnote: - Source note: -
Exclusive and Inclusive Methods
Classes Frequency Classes Frequency
0 – 10 2 0 – 9 2
10 – 20 3 10 – 19 3
20 – 30 4 20 – 29 4
30 – 40 2 30 – 39 2
40 – 50 1 40 – 49 1
Total 12 Total 12
It is called exclusive method of classification. It is called exclusive method of
classification. In this case the upper limit is not included in the class. In this case
the upper and lower limit are included in the particular class.
GRAPHICAL PRESENTATION
Page | 99
In day-to-day reading, including newspapers, we all come across various kinds of
diagrams such as a bar diagram, a pie diagram and line diagram. They are used to
readily show the pattern of value. Some people use tricks in drawing such diagrams
to highlight their positive own feature. You should be able to track down such
instances. Also you should be able to make a judicious choice yourself about the
kind of diagram that is most appropriate for a particular kind of data.
Drawing or graphical presentation: After class-wise or group-wise tabulation the
frequency of a characteristic can be presented by two kinds of drawing
a. Graphs b. Diagrams.
Presentation of quantitative data: Continuous or measured data is done by
graphs and those in common use.
a. Histogram:
It is a set of adjoining vertical bars whose areas are proportional to the frequency
represented by the bar. Here by taking class interval on X-axis and the frequency
on Y-axis. Eg. Tuberculin reaction measured in 206 persons.
0
10
20
30
40
50
60
No
of
case
s
Pain in
abdomen
Backache Discharge
PV
Bleeding PV Urinary
complaint
Pelvic
pressure
Symptoms
Bar diagram showing symptoms wise distribution of cases in study
group
b. Frequency polygon:It is easy to construct and simple to interpret. It is a line
chart plotted in the same way as the histogram. Here class mid point on X-axis and
Page | 100
the frequency on Y-axis, joined this points by straight line to give us frequency
polygon.
Scatter diagram showing correlation between days of delirium
and ICU stay in study group
0
5
10
15
20
25
0 2 4 6 8 10 12 14
Days of delirium (Days)
ICU
sta
y (D
ays)
c. Frequency curve:
When the numbers of observations are very large with small class intervals, it gives
smooth curve known as frequency curve. It is slightly modification of frequency
polygon. Here by taking class interval on X-axis and the frequency on Y-axis.
Joined this points by smooth curve instead of straight line.
d. Cumulative frequency curve or Ogive curve:
We are interested to knowing “How many cases attending the hypertension clinic
had cholesterol level less than 200 mg / dl and more than 200 mg / dl”,
“Percentage of students who have failed” etc. To answer these questions, it is
necessary to add the frequencies. When the frequencies are added, they are called
cumulative frequencies. The curve obtained by plotting cumulative frequencies is
called a Cumulative frequency curve or an Ogive curve.
There are two types:
1. Less than type: We start with the upper limits of the classes and go on adding
the frequencies. Plot these points we get a rising curve.
2. More than type: We start with the lower limits of the classes and subtract the
frequencies of each class. Plot these points we get a declining curve.
e. Line chart:
This is a frequency polygon presenting variation by line. It shows an event
occurring over a period of time rising, falling or fluctuations. This kind of diagram
is best used to show trend in a metric measurement over time or over age. Growth
charts, or “Road to Health” card, used for assessment in children, are line diagram.
Eg. This is a temperature chart for a patient of tuberculosis for these consecutive
days. Evening rise in temperature in this case can indicate toxaemia.
Eg. Birth rate, growth rate, IMR, death rate from 1951 to 1991.
Page | 101
f. Scatter or dot diagram:
To show the nature of correlation between two variable in the same persons or
groups. Eg. Height and weight, BMI and BSL in your batch. The points are plotted
on graph paper, one for each observation. Such type of diagram shows how far the
points are scattered. Hence it is called scatter diagram. Draw a line passing
through these points maximum points on a line, half point lie above and half lie
below, to show the nature of correlation at a glance.
Eg. BMI and BSL are recorded for 23 persons; a point can be plotted for each
person with BMI on horizontal axis and BSL on the vertical axis. The vertical axis
in a scatter diagram should be dependent or the outcome variable. Eg. BSL depend
on BMI and BMI does not depend on BSL. Thus BSL should be on vertical axis.
The trend of these points may show that BSL increases when the BMI increases.
Presentation of qualitative data: Continuous or measured data is done by graphs
and those in common use.
a. Bar Diagram:
It is easy and popular method. Length of bars, vertical or horizontal indicates the
frequency of a character to be compared. Bar may be drawn in ascending or
descending order of magnitude or serial order of event. Spacing between two bars
should be equal. There are 3 types of bar diagrams.
-Simple bar diagram
-Multiple bar diagram
-Proportional bar diagram
b. Pie or Sector diagram:
This is another way of presenting discrete data of qualitative characters such as
blood group, Rh group, social group, sex group etc. Pie diagram has circular shape.
A circle is divided into sectors with areas proportional to the frequencies or the
relative frequencies of the categories of the variable.
Eg. Number of episodes of respiratory, digestive, cardiovascular, injuries etc.
attending a clinic are additive. On the other hand, some rates are not added (birth
rate: rural 30/1000 populations, and urban 16/1000 populations). So pie diagram
cannot draw for such rates. Rate will be in between 16 and 30.
Note: Since one patient can and will have more than one sign-symptoms, pie
diagram is not appropriate for this depiction also. All sign-symptoms do not add to
any thing, certainly not to the total number of patients, nor to the total number of
episodes. Thus pie diagram is not applicable.
Page | 102
Pie diagram showing gestational age wise distribution of cases in study
group
32%
64%
4%
28 – 31
32 – 34
35 – 37
c. Pictogram:
It is a popular method to impress the frequency of the occurrence of events to a
common man. Eg. Accidents, attacks, deaths, admitted, discharged etc. The
pictures are drawn on horizontal lines. Each picture indicates a unit of 10, 20, 100,
1000 etc. happenings. The number of pictures in each row gives quick idea of
frequency.
d. Map diagram or Spot Map:
To show the geographical distribution of frequencies of a characteristic. A dot
indicates on unit of occurrence such as attacks or deaths. The number of dots will
indicate the frequency in units.
e. Run chart: to display serial data points over time. Because our minds are
not good at remembering patterns in data, a visual display will allow you to
see the measurement of an entire process. This in turn will enable you to
see trends over time and to make adjustments accordingly.
Page | 103
Use; Run charts are particularly useful in conjunction with simple data from
tally or check sheets. Record tallies of a particular event that you would like
to capture, e.g. a patient's weight or times you are late for a meeting. Plot
the values on the y-axis versus number of measurements on the x-axis.
Note on the chart where any changes in the process are made. Then analyze
the data for trends. Is your patient's weight stabile over time? Did a
decrease coincide with the start of new medication? Since you bought your
electronic organizer, have you made it to more meetings on time. See
examples below.Examples
Example 1: Patient's weight over time.
SAMPLING
Sampling: The process of selecting a sample from a population.
Sample: A finite subset of statistical individuals in a population. Or sample is that
part of the target population, which is actually enquired on or investigated.
Sample size: The number of individuals in the sample. Or the number of sampling
unit included in the sample. Eg. Investigating mineral density in hipbone, you may
like to include 150 persons. In this case, this is the sample size. For ocular ailment,
the sample may be of 80 eyes irrespective of the number of persons. For clinical
trial, the sample size may be 200 cases divided equally to receive treatment and
placebo.
Sampling Unit: The unit of selection in the sampling process. Eg. a person, a
household, or a district. It is not necessarily the unit of observation or study.
Page | 104
In a large scale study that covers, eg. The entire country, it is desirable to select a
sample of states in the first stage, district in second stage, etc. Thus the sampling
unit is state for first stage sampling, district for second stage sampling etc.
The ultimate sampling unit is generally the same as the unit of study.
Eg. In a study on family pattern in hypertension, the sampling unit and the unit of
study is the same i.e. family.
Eg. But in other set up such as for incidence of injuries, the sampling unit could be
a family but the unit of study would be the individual. In this case, the families are
not further sampled and all individuals in the selected family are enquired
regarding the incidence of injury.
Sampling frame: A list of all units in the target population from which a sample is
drawn. If the target population comprises the deliveries in your hospital in one-
year period, the population size is known. Suppose this is N=7000. If you want to
include n=140 of these in your sample, the sampling fraction is n/N = 140/7000
=0.02 = 2%
Parameter: the statistical constant computed from the population value.
TYPES OF DATA COLLECTION DESIGN USED IN HEALTH AND
MEDICINE
Data collection
Objective: Descriptive Analytical
Method: Survey Observational Experimental
Time frame: Prospective Retrospective Cross-Sectional
Setting: Trial Animal
1. Simple random sampling:
Every sample of the same size (Every sampling unit in the sampling frame) has an
equal chance of being selected.
Advantages:
a. The sample is assured of being representative and subject only to sampling
error.
Page | 105
b. Estimates are easy to calculate
Disadvantages:
a. If the sampling frame is large, this method may be impracticable because of
difficulty and expense of constructing or updating it in large-scale surveys.
b. Minority subgroups of interest in the population may be present in the sample
in sufficient numbers for study.
Selection can be done by Lottery method or Random number.
Eg. Suppose you are in a big hospital where nearly 500 cases of Myocardial
infarction (MI) are reported every year. You are interested in their physiological
profile – their blood pressure, cholesterol level, creatinine phosphokinase (CPK)
level, lipoprotein (a) level and Homocysteine level. The objective for the time being is
not to compare these parameters with their healthy counterparts because that
objective will push the study into analytical domain. But the sampling will remain
the same in that case too.
You have resources to do these investigations in not more than 100 of these
patients. The target group still is these 500 cases reporting in one particular year.
That is, the findings should be applicable to all these 500 cases. Naturally you will
like to have a sample of 100 that represents a cross-section of these cases.
Lottery method: Prepare 500 similar cards of same size, mix them in a box, and
draw 1st card at random, note these number. Replace the card drawn again, mix
and draw 2nd card. Repeat the process till 100 cards drawn at random. Reject the
cards that are drawn 2nd time. The patients with these numbers will be in your
sample.
Random number: 500 cases of MI is three-digit figure. Three digit numbers are
chosen from random number table or to take help of computer, generate 100
distinct random numbers “between” 001 to 500. If the selected random number is
more that 500 then divide these number by 500 and reminder taken on your list.
2. Systematic random sampling:
The first unit is selected at random from among the first k – units. The selection of
every Kth unit in the population / list / file. Where K = N / n.
Advantages:
a. The sample is easy to select.
b. A suitable sampling frame can be identified more easily.
c. The sample is evenly spread over the entire reference population.
Disadvantages:
Page | 106
a. The sample may be biased if a hidden periodicity in the population coincides
with that of the selection.
b. It is difficult to assess the precision of the estimate from one survey.
Eg. One hundred cases out of 500 cases of MI are one out of five. Choose one
number at random out of 1st five and add systematically 5 cases each time. If the
1st random number is 3, the others are 8,13,18,23 etc. Patients with these
numbers will be in your sample.
Note: that in simple or systematic method, there is no assurance in this example
that there would be, say, enough obese person or enough females. Thus
physiological profile of patients in different obesity categories or genders may not be
obtained.
3. Stratified random sampling:
First the population is divided into groups or strata according to characteristic of
interest. (eg. Age, Sex, Geographical location etc.)
A simple random sample is then drawn from each stratum using the same
sampling fraction, unless otherwise prescribed for special reasons.
Advantages:
a. Every unit in a stratum has the equal chance of being selected.
b. Using the same sampling fraction for all strata ensures proportionate
representation in the sample of the characteristic being stratified.
Disadvantages:
a. The sampling frame of the entire population has to be prepared separately for
each stratum.
b. Varying the sampling fraction between strata, to ensure selection of sufficient
numbers in minority subgroups in the sample as a whole.
Eg. Out of 100 cases in the sample, 50 should be overweight or obese (BMI >=25)
and the other 50 of normal or low weight. This is because the physiological
parameters could be different in these two groups. This can be achieved when the
MI patients are first grouped into such categories. This grouping is called
stratification, and each group is called stratum. Suppose out of 500 cases of MI,
200 have BMI <25 and 300 have BMI >= 25 kg/m2. The procedures is now to select
50 out of 200 and another 50 out of the other 300 by Simple random sample
separately. This method ensures that a pre-specified number of individuals are in
the sample from different categories and none is under or over represented.
Eg. Select a stratified random sample of 20 patients from 200 patients given below
by stratification of diseases.
Page | 107
Strata I II III IV Total
No. of
patients
100 60 20 20 200
% 50 30 10 10 100
Sample size 10 6 2 2 20
S – I => 50% of 20 =10, S – II => 30% of 20 =6
S – III => 10% of 20 =2, S – IV =>10% of 20 =2
4. Cluster random sampling:
First the population is divided into clusters of homogeneous units, usually based
on geographical contiguity. A sample of such clusters is then selected. All the units
in the selected clusters are then examined or studied.
Advantages:
It reduces the cost of preparing a sampling frame and traveling between selected
units.
Disadvantages:
Sampling error is usually higher than for a simple random sample of the same size.
This procedure is quick and easy to administer. WHO recommends this procedure
to estimate the immunization coverage in developing countries.
Eg. Let us suppose that there are 24000 households in a city and we want to select
a sample of 800 such households. We divide the entire area of the city into clusters
(wards), which are clearly identifiable and ensure the location of each of the city
households in these clusters. Suppose 750 such clusters have been identified. The
average number of households in each cluster is 24000 / 750 = 32 household /
cluster. Cluster to be selected is an 800 / 32 = 25 cluster. Make a random selection
of 25 clusters from among the 750 identified clusters to constitute a sample of 800
households.
Eg. To evaluate vaccination coverage in Expanded Programme of Immunization
(EPI) and Universal Immunization Programme (UIP) where only 210 children, taking
7 from each cluster in the age group of 12 – 17 months are to be examined.
List of all cities, towns, villages and wards of cities with their population
falling in the target area under study for evaluation.
Calculate cumulative population and divide the same by 30, that give the
Sampling Interval (SI)
Select a random number <= SI having same number of digits, this forms the
first cluster.
Page | 108
Random number + SI gives 2nd cluster.
2nd cluster + SI gives 3rd cluster and so on
All houses with population are numbered. The first house should be selected
randomly with the help of random number table or number on currency note.
Before starting house-to-house survey defines the age group and item you wish to
study ie. Children age group is 12 – 17 months, fully vaccinated. They have 3 DPT,
3 Polio, 1 BCG and 1 Measles vaccination. Now survey houses starting with the
selected house till you get 7 children fully vaccinated. Thus 210 such children will
be found in 30 clusters.
5. Multistage random sampling:
-The selection is done in stages until the final sampling units (household or
persons) are arrived at. In the first stage, a list of large sized sampling units is
prepared. These may be towns or villages or schools. A sample of these is selected
at random, with probability of selection proportional to size. For each of the
selected first stage units, a list of smaller sampling units is prepared. (If the 1st
stage units are towns, then 2nd stage units may be households or houses). A
sample of these second stage units is then randomly selected from each of the
selected first stage units. These are then studied. The procedure may contain three
or more stages.
Advantages: Reduced the cost of preparing a sampling frame.
Disadvantages:
Sampling error is increased compared with a simple random sample of the same
size. Eg. Let us consider the prevalence of cataract in elderly population. In this
case to carry out sampling in stages. Select a few but predetermined number of
districts randomly in the 1st stage. Then select a few primary health center (PHC)
areas from rural and a few towns in urban areas again by random method from
each selected district. Now select a few villages from rural and a few census block
from urban. Finally select a few families from each selected village or census block.
Examine all elderly persons in these families for cataract.
If a state has 5 crore population with 3.5million elderlies, a random sample of 4
districts, 3 PHC and 3 urban area from each selected district, 10 villages or 10
census blocks from each selected areas and 20 families from each of these, would
yield a sample of 4x(3+3) x10x20 = 4800 families. These may have only about 1700
elderly persons. This sample of 1700 out of a target population of 3.5 million,
which is just about one in 2000, is an exceedingly small fraction. Yet this sample
could be very representative when random component of selection is faithfully
Page | 109
carried out. However the method requires that the sampling frame (list of sampling
units) be available within each selected unit.
6. Multiphase random sampling:
Part of the information is collected from whole sample and part from sub-sample.
Eg. In a Tuberculosis (TB) survey physical examination or montoux test may be
done in all cases of the sample in the 1st phase, in the second phase X-ray of the
chest may be done in mantoux test positive cases and in those with clinical
symptoms; while sputum may be examined in X-ray positive cases in the 3rd phase
only. Number in the sub-sample of 2nd and 3rd phase will become successively
smaller and smaller. Survey by such procedure will be less costly, less laborious
and more purposeful.
7. Purposive random sampling:
The sampling is purposive when cases that serve specific purpose are chosen for
generating data. Or In which the sample units are selected with definite purpose in
view. Eg. If we want to give the picture that the standard of living has been
increased in the city of New Delhi. We may take individuals in the sample from rich
and posh localities like defense colony, south extension, golf links, jor bagh,
chanakyapuri etc. and ignore the localities where low income group and the middle
class family live.
Measures of Central Tendency
The value of the measures of central tendency is regarded as the most
representative value of the given data. There are measures of central tendency such
as Mean, Median, Mode, Geometric Mean, Weighted Mean, & Harmonic Mean and
locations other than central are Quartiles, Percentiles, Deciles, & Tertiles.
Mean:
It is the sum of all the observations divided by total number of observations and
denoted by X (X bar).
Merits:
1. It is rigidly defined
2. It is It is easy to calculate and simple to understand.
3. It is based on all observations.
4. It is capable of further algebraic treatment.
Demerits:
1. It is vary much affected by extreme values.
2. When extreme class is open Arithmetic Mean cannot be calculated.
Median:
Page | 110
Arrange the data in ascending or descending order of magnitude. The value of the
middle most observation is called median and denoted by Me.
*Ungroup / Discrete series:
a. When the number of observations / subjects is odd then
Me = Middle most observation = ((n+1) / 2)th observation.
b. When the number of observations / subjects is even then
Me = Average of two middle most observation = 1/2((n/2)+((n/2)+1))
Merits:
1. It is rigidly defined
2. It is easy to calculate and simple to understand.
3. It is not affected by extreme values.
Demerits:
1. It is not based on all observations.
2. It is not an exact value when observations are even.
Mode:
The most common or most frequently occurring value is called mode and denoted
by Mo.
Ungroup / Discrete series:
Mo – Most common or frequently occurring value.
Merits:
1. It is easy to calculate and simple to understand.
2. It is not affected by extreme values.
Demerits:
1. It is not based on all observations.
2. It is not capable of further algebraic treatment.
Mean, Median and Mode in the symmetric and skewed distributions:
If the women are generally under-nourished, the lower values would be quite
common and the distribution would be left skewed. Fig. (a) shows a left skewed
distribution of Hb. Level in under-nourished women. A skewed distribution (left /
right) will have different Mean, Median and Mode. Very different values give a clear
indication of skewness. If left skewness Mean < Median < Mode and right skewness
Mean > Median > Mode.
If women are well nourished or in good health, maximum number may have Hb
around 14 g/dl and there will be as many on the lower side as on the upper side. If
Page | 111
all the three averages ie. Mean, median and mode are nearly equal; this is a good
indication that the frequency curve is symmetric, possibly Gaussian.
Where to use mean, where median and where mode to represent central value?
Guideline is as follows. Always use mean as the central value because the average
is so easy to comprehend, except when
a. Outliers are present in the data, then use median.
b. Interest specifically is in the most common value, then use mode.
Geometric mean:
It is the nth root of the products of n observation and denoted by GM.
GM =n√X1, X2, --- Xn g = antilog {logx / n}
Merits:
1. It is rigidly defined
2. It is based on all observations.
3. It is capable of further algebraic treatment.
Demerits:
1. It is neither easy to calculate nor simple to understand.
2. If any value in a series is zero then GM is also zero.
Weighted mean:
A mean for which individual values in the set are weighted, very often by their
respective frequencies.
Harmonic mean:
It is the reciprocal of the mean of the reciprocal of value/ observations and denoted
by HM.
HM = 1 / (1/n)* (1/Xi) = n / (1/Xi)
Merits:
1. It is rigidly defined
2. It is based on all observations.
3. It is capable of further algebraic treatment.
Demerits:
1. It is neither easy to calculate nor simple to understand.
2. If any value in a series is zero then HM is zero.
Location other than central:
You may have heard of a very difficult examination and the topper scoring only
72% marks. In medicine too, it is sometimes important how a patient appears
relative to the others, although he may not be optimal. In growth assessment of
Page | 112
children, it is common to say that the height of a particular child of age four years
is better than 80% children of his age. The interest is not in the central value but is
in other locations. Most popular measure of such location is percentile.
*Percentiles:
It divides the subject in 100 equal parts, each part containing n/100 subjects. If
n=400, then each part will have 4 subjects. The parts are identified by 99 cut
points of the measurement under consideration i.e. P1, P2, P3, - - - P99.
Ungroup / Discrete series:
Kth percentile = (k*n/100)th value after arranging in ascending order from lowest
to highest.
Group / Continuous series:
Pi = L1+(((iN/100)-c)*h/fi) i=1,2,3, - - - 99
P10=D1, P20=D2, - - - P90=D9
P25=Q1, P50=Q2=D5=Me, P75=Q3
Percentile curve is a cumulative curve drawn on a percentage basis (<type)
Deciles, Quartiles and Tertiles:
Deciles divide the group of subjects into 10 equal parts, quartiles into 4 equal parts
and tertiles into 3 equal parts. The procedures are same as above. The
denominator, which was 100 in the case of percentiles in formula Kth percentiles
mentioned above, would be 10 for deciles, 4 for quartiles and 3 for tertiles.
Measures of Variation (Dispersion)
The deviation of each and every observation from any measures of central tendency
is called measures of variation or dispersion.
When the mean value of a series of measurements has been obtained, it is usually
a matter of considerable interest to express the degree of variation or scatter
around this mean. Are the reading all rather closed to the mean or are some of
them scattered widely in each direction?
Eg. The daily Calorie requirement for a man of 25 years is given as 3200. This
requirement must very from one person to another, how large is the variation?
Types of measures of variation:
Range, Variation, Standard deviation and coefficient of variation.
Range:
The difference between the maximum value and the minimum value and denoted
by R.
R= Xmax. – Xmin.
Eg. Systolic blood pressure
Page | 113
Group I: 128, 132, 129, 130, 131
Group II: 140, 150, 120, 130, 110
Systolic level in-group I is 128 to 132, so that the range is 4 mmHg. In-group II,
this is from 110 to 150 and the range is 40 mmHg. The difficulty in measuring
dispersion by range is that an alteration in just one value to an extremely high or
extremely low value, drastically changes the range. If the last BP in-group I is 161
instead of 131, the range shoots to 33 mmHg. Although the other 4 values are still
closed to one another and the dispersion is not high, but the range unnecessarily
will indicate a very high dispersion because of one extremely high value. Because of
this demerit, we look for a measure that considers all the values, and not just the
minimum and the maximum value.
Standard deviation:
It is the square root of the mean of the square deviation from their mean and
denoted by sd/ (sigma)
Ungroup / Discrete series:
= √ (Xi-X)2/n =√(1/n){X2 –(X)2/n}
The square of sd is called variance and denoted by var. / sd2 / 2.
Steps:
1. Calculate the mean ie. Xbar
2. Find the difference of each observation from the mean ie. (Xi-X)
3. Square the difference of each observation from the mean ie. (Xi-X)2
4. Add the square value to get sums of square ie. (Xi-X)2
5. Divide this sums of square by number of observation –1 to get variance ie.
2 = (Xi-X)2/n-1
6.find the square root of the variance to get sd.
Coefficient of variance:
The ratio of sd to the mean and denoted by CV.
CV = sd/x
It is used to compare the variability of a characters in two group or two characters
in the same group. OR When measurements are for different persons and
parameters are different.
NORMAL DISTRIBUTION AND NORMAL CURVE
When the number of observation is very large of any variable characteristics are
taken at random to make it a representative sample eg. Height, Weight, Blood
Page | 114
Pressure, Pulse Rate etc. Prepared a frequency distribution table by keeping small
class interval then it will be seen that:
-Some observations are above the mean and other are below the mean.
-If they are arranged in order, deviating towards the extremes from the mean, one +
or – side, maximum number of observations will be seen in the middle around the
mean and fever at the extremes, decreasing smoothly on both sides.
-Normally half of the observations lie above and half lie below the mean
and all observations are symmetrically distributed on each side of the mean.
A nature or shape of this distribution is called Normal Distribution or Gaussion
Distribution.
If mean and standard deviation are known:
a. Mean ± 1 SD covers 68.27% observations. Remaining 32% observations lie
outside the range mean ± 1 SD.
b. Mean ± 2 SD covers 95.45% observations. Remaining 4.55% observations lie
outside the range mean ± 2 SD. (Mean ± 1.96 SD covers exactly 95% observations )
c. Mean ± 3 SD covers 99.73% observations. Remaining 0.27% observations lie
outside the range mean ± 3 SD. (Mean ± 2.58 SD covers exactly 99% observations )
You know that the normal range of fasting blood glucose level is 80 to 110 mg/dI.
Do you know how is this range obtained?
Opposed to a range for fasting blood glucose level, the normal body temperature is
a single value 98.6F. Why is this not a range?
Normal Range:
Most medical measurements show a substantial variation even in healthy subjects.
Thus a range of normal values is obtained. In above case, fasting blood glucose
level, the normal range may be 80 to 110 mg/dI but there will be people with 79 or
75 mg/dI yet absolutely healthy and other healthy people with 115 or 118 mg/dI
level. In other words, for any parameter, there will be healthy people with very low
Page | 115
or very high values. If such values are also included in the normal range, the
difficulty is that many diabetics would have overlapping levels such as 115 or 118
mg/dI. Similarly there will be hypoglycemic with levels 79 or 75 mg/dI.
No matter how a normal range is chosen, there is always a risk of exclusion of
healthy subjects and inclusion of non-healthy subjects. The best course, of course,
is to find levels beyond which persons or patients start feeling sick, or the levels
that have increased risk of early mortality. Such level can indeed be considered
pathologic. But this procedure is highly nonspecific and too difficult to adopt. Also,
even in such delineation there would still be a chance of exclusion of healthy and
inclusion of nonhealthy subjects. In view of these difficulties, it is considered
convenient to use statistical principles to determine the normal range.
Normal value:
Any biological measurement must very from person to person, and even in a person
from time to time. The variation could be small or large. Whenever the variation is
small, a single value is obtained as the representative value. There is no need to
worry about SD of such measurement. This is true for body temperature.
Although there would be some healthy people with temperature 98.5 F or 98.7 F,
even with 98F but those will be few. A 5% rise in body temperature has an
enormous clinical significance whereas 5% rise in cholesterol level may not be
much consequence. When the variation is small, the mean is generally chosen as
the reference value.
If you want to establish normal body temperature of adolescent boys in your area,
select a random sample of at least 300 apparently healthy boys, measure their
body temperature and calculate the mean. That will be the normal level for these
boys. No such exercise has ever been undertaken for Indian boys, girls, children‟s
or adults on a large scale. Thus our normal body temperature is not known.
However the internationally recognized level of 98.6 F, which actually was
established for Swedish adults, seems to work for Indian as well.
Characteristics of Normal Curve:
1. It is bell shaped curve.
2. It is symmetrical.
3. Mean, Median and Mode coincide.
4. It has two inflections never touches to the baseline.
5. Area under curve is one.
6. If standardized then mean is zero and SD is one.
Standard Normal deviate/ Standard Normal viriate/ Standard Normal curve:
Page | 116
This process of subtracting mean and dividing by SD is standardization or
sometime normalization and denoted by Z
Z =(X-)/SD
Inferential statistics: is the branch of statistics that is used to make inference
about the characteristics of a population based on sample data.
Sampling variation:
The sample after all the part of the population and they may or may not truly
represent all the features of the population.
One sample would differ from the other even if both were taken from the same
target population. Mean and SD obtained from one sample would be different from
the other sample. This is called sampling variation / sampling fluctuation /
sampling error.
Standard error:
The point estimate is simply the corresponding sample statistics of the population
parameter. The forced expiratory volume in one second (FEV1) in students of age
18 to 22 years. Take a random sample 40 students and found that mean FEV1 is
2.71 l. He was somehow not happy with this mean. He took another sample of 40
and found mean FEV1is 2.50 l. He repeatedly took sample of 40 students another
eight times. Considering the variation in various sample mean, he was not sure
how to express this uncertainty. He was then advised to find the SD of these 10
means obtained in 10 different samples. He then understood that the SD of these
10 sample means is the measures of variability in sample means. This SD of
sample mean is called the standard error (SE) of mean.
Estimation: There are two types estimation.
Point Estimation: is the value of your sample statistic (Sample mean or sample
correlation) and it is used to estimate the population parameter (Population mean
or Population correlation) eg. If you take a sample of Dr. living in pune city and you
find that the average income of Dr. in your sample is Rs 25000 then your point
estimate of Dr in pune city will be Rs 25000.
The value of the Sample Mean / Median is an estimate of the population Mean /
Median. Similarly, Sample Proportion is an estimate of the population Proportion.
These are called point estimates.
Point estimate for is Xbar and for P is p.
Interval estimation: is a range of numbers inferred from the sample that
has a known probability of capturing the population parameter over the long
run.
Page | 117
Consider the average haemoglobin (Hb) level in women going into hypertension
during labour. This enquiry for Hb level can possibly lead to some etiology of
hypertension during labour. In a random sample of 300 such women, this average
is 10.6 g/dl. Would you accept this 10.6 g/dl as the absolute truth for these
women or you would allow for some sampling fluctuation and say that it is most
likely somewhere between 10 and 11 g/dl? Obtaining such interval for any
parameter is called interval estimation.
It is used in patient care when you inform the relatives of a cancer patient that the
survival duration is somewhere between 2 and 7 months at that stage of disease.
Point estimates have reliability only when SE is small. If SE is large, interval
estimates are obtained.
Confidence level:
The 95% or any other level that is fixed as a measure of hope or expectation is
called the confidence level.
It must be very clear that uncertainties in medical practice can only be minimized
but not eliminated. You can never be 100% confident about the outcome. This is all
the more true while dealing with the samples. Thus a confidence level is fixed at a
sufficiently high level to ensure reasonable reliability. There is a tendency around
the world to consider 95% confidence as adequate while dealing with samples – be
it sample of patients, blood samples or sample of healthy people.
Confidence Interval:
The interval within which a parameter values expected to lie with a certain
confidence level.
* Specifically, if you have the computer provide you with 95 percent confidence
interval then you will be able to be “95% confident” that it will include the
population parameter. That is “level of confidence” is 95%.
Eg. The point estimate of annual income of Dr. in pune city is Rs 25000 and
surround it by a 95% confidence interval. You might find that the confidence
interval is Rs. 22000 to Rs. 27000. In this case, you can be 95% confident that the
average income is somewhere between Rs. 22000 and Rs. 27000.
TEST OF SIGNIFICANCE
A statistical procedure to test whether or not the observations fall into a specified
pattern, such as equal mean of two or more groups or following a linear trend. If
they do no, the result is called statistically significant. This requires prior fixing of
the level of significance that specifies the maximum tolerable probability of type I
error.
Page | 118
Hypothesis: Any statement regarding population parameter is called as
hypothesis. Or A statement of belief that is made before the investigation regarding
the status of parameters under study, including those that measure relationship.
Null Hypothesis: A hypothesis that says that there is no difference. The initial
assumption will be that the new regimen is not better. This kind of assumption is
called null hypothesis and denoted by Ho.
Eg. 1. The two regimens or two groups have no difference.
2. The incidence of leukemia in the four blood groups is same.
2. There is no difference in the mean aspartate amino-transferase (AST) level in
the case of hepatitis, cirrhosis and liver malignancy.
Alternative Hypothesis: A hypothesis, which is accepted by default when the null
hypothesis is rejected and denoted by H1.
Note: The null hypothesis is never completely right or wrong, or true or false, but is
only rejected or not rejected at the probability level of significance concerned.
Type I error: The probability of rejecting a null hypothesis (Ho) when it is in fact
true and denoted by alpha ()
= P(Rejecting Ho/Ho is true)
Type II error: The probability of not-rejecting i.e. accepting a null hypothesis (Ho)
when it is false and denoted by beta ()
= P (Accepting Ho/Ho is false)
Level of significance: Size of the type I error (). Or the distance from the mean at
which Ho is rejected. Or the maximum tolerable probability of type I error that is
fixed in advance, such as 5%, denoted by alpha ().
P-value: The probability of committing type I error is called the P-value. Or P-value
is the chance that the presence of difference is concluded when actually there is
none.
One tail Test: Checks only one of the tail (upper or lower of the normal
distribution curve).
Eg. 1. Comparing the rate of cancer between a population exposed to known
carcinogen and a control population. This test can be used because the only
alternative hypothesis of interest is that the exposure was harmful. It is assume
that the exposure was not harmful.
2. Consider a new haematinic that is supposed to increase the Hb. level among
anaemics. In a trial for this preparation, the null hypothesis again is that it is not
effective. This hypothesis implies that the average Hb. level will not alter taking this
preparation. What is the alternative hypothesis?
Page | 119
If the Ho is rejected, the conclusion will be that the new haematinic is effective or
that the average Hb level has increased. In this situation, the possibility of
reduction in Hb level is ruled out. Thus, H1 is one sided and is called one tail test.
3. While comparing a test regimen with placebo, if there is an assurance that test
regimen cannot be worse than placebo. This requires one tail test.
Two tail Test: Checks the upper and lower tails of the normal distribution curve.
Eg. 1. Comparing the rates of death between two neighboring communities. This
test is used to look for significant differences because no assumption is made about
the H1.
2. For dilating cervix by Misoprostol Vs the existing ethinyl estradiol, there is a
possibility that the efficacy of new regimen is even less than the existing regimen.
The efficacy can be higher or can be lower who know! The null hypothesis in this
case would be usual saying that the two regimens are equally effective.
When null hypothesis is rejected, what is accepted is called alternative hypothesis.
For Misoprostol efficacy, the alternative hypothesis is that it is either lower or
higher than the efficacy of the existing procedure. This type of alternative is called
two-sided because both possibilities are envisaged. A test in a situation where the
alternative hypothesis is two sided is called a two-tail test.
3. While testing equality of two groups. This happen, when a test regimen is being
compared with the existing regimen. This requires two-tail test.
Commonly used test of significance as per type and size of data:
Type of data
Size ↴
Quantitative Qualitative
Large sample
(n >= 30)
Z-test (SEx, SEx1-x2) Z-test (SEp, SEp1-p2)
χ2 test
Small sample
(n < 30)
t-test (Paired & unpaired)
N-P test
χ2 test with Yates correction
N-P test
Procedure and steps:
Find the type of problem and the question to be answered.
To state null (Ho) and alternative (H1) hypothesis.
Selection of a appropriate test.
Fixation of LOS (a), minimum desirable level is 5%.
Calculation of the test criterion based on the type of test.
Page | 120
Comparison of calculated value with theoretical value.
Drawing the conclusion. P-value.
* If the results of a sample fall within the Mean 1.96 SE, the null hypothesis is
accepted, hence this area is called zone of acceptance for null hypothesis.
* If the results of a sample fall outside the Mean 1.96 SE, the null hypothesis is
rejected, hence this area is called zone of rejection for null hypothesis.
Now let us discuss the various situations where we have to apply different test of
significance. For the sake of convenience and clarity these situation may be
summed up under the following three heads:
1. Test of significance for Attributes.
2. Test of significance for Variables (large sample).
3. Test of significance for Variables (small sample).
1.Test of significance for Attributes.
As distinguished from variables where quantitative measurement of a phenomenon
is possible, in case of attributes we can only find out the presence or absence of a
particular characteristic.
Eg. In a study of attribute „Literacy‟ a sample may be taken and people classified as
literates and illiterates. With such data the binomial type of problem may be
formed. The selection of an individual on sampling may be called „event‟ the
appearance of an attribute A may be taken „Success‟ and its non-appearances as
„Failure‟.
a. Test for number of success:
The sampling distribution of the number of success follows a binomial probability
distribution.
SE of number of success = √npq
Where n = Size of sample, p = Probability of success in each trial and q = 1-p =
Probability of failure.
Eg. In a hospital 480 female and 520 male babies were born in a week. Do this
figure; confirm the hypothesis that males and females are born in equal number?
Ho: The male and female babies are born in equal number. i.e. p = ½
H1: p =/ ½
n =1000, p = ½, q = ½
SE = √npq = 15.81
Difference between observed and expected number of female babies =520 – 500 =
20
Z = Observed difference / SE = 20 / 15.81 = 1.265
Page | 121
Since the difference is less than 1.96 SE at 5% LOS. Hence the male and female
babies are born in equal number. Or
Z < 1.96, Accept Ho, hence the male and female babies are born in equal number.
2.Test of significance for Variables (large sample). Z – test.
The Z-test for mean has two applications
a. To test the significance of difference between a sample mean (x) and a
known value of population mean (u)
Z = observed difference between x and u / SEx
b. To test the significance of difference between two sample means or
between experimental sample mean and control sample mean.
Z = Observed difference / SE
Standard error of mean (SEx):
When?
-Sample size should be large and drawn randomly.
-Data should be quantitative.
-The variable under study is assumed to follow Normal distribution in the
population.
Why?
-To calculate size of the sample.
-To estimate population parameter, when u is not known.
-To test whether the sample is drawn from representative of population or not,
when u is given.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data
-Calculate SEx = sd / √n
-Calculate „Z‟ value
Z = Observed difference / SEx
-Fixation of LOS, minimum desirable is 5%.
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
Standard error of Proportion (SEp):
Page | 122
When?
-Sample size should be large and drawn randomly.
-Data should be qualitative.
-The variable under study is assumed to follow Normal distribution in the
population.
Why?
-To calculate size of the sample.
-To estimate population proportion.
-To test whether the sample proportion is drawn from representative of population
proportion or not.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data
-Calculate SEp =√ pq/n
-Calculate „Z‟ value
Z = Observed difference / SEp
-Fixation of LOS, minimum desirable is 5%.
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
Standard error of difference between two mean (SEx1-x2):
When?
-Sample size should be large and drawn randomly.
-Data should be quantitative.
-The variable under study is assumed to follow Normal distribution in the
population.
Why?
-To compare the efficacy of two therapies.
-To test whether the sample is drawn from the representative of population or not.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data
Page | 123
-Calculate SEx1-x2 =√ {(sd12/n1)+(sd22/n2)} = x √ {(1/n1)+(1/n2)}
Where =√ {[(X1-x1)2 +(X2-x2)2]/(n1+n2-2)}
-Calculate „Z‟ value
Z = Observed difference / SEx
-Fixation of LOS, minimum desirable is 5%.
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
Standard error of difference between two proportion (SEp1-p2):
When?
-Sample size should be large and drawn randomly.
-Data should be qualitative.
-The variable under study is assumed to follow Normal distribution in the
population.
Why?
-To test whether the sample proportions are drawn from the representative of
population proportion or not.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data
-Calculate SEp1-p2 =√{(p1q1/n1)+(p2q2/n2)}
-Calculate „Z‟ value
Z = Observed difference / SEp1-p2
-Fixation of LOS, minimum desirable is 5%.
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
3.Test of significance for Variables (small sample). t-test.
Small samples or their Z values do not follow normal distribution, because the
samples SD do not adequately represent the population SD, even if the sample is
Page | 124
drawn from the normal distribution. So the Z value, in these cases will not give the
correct level of significance or probability.
In case of small sample, „t‟ test is applied instead of Z test. This test is design by
W.S. Gossett whose pen name was Student. Hence, this test is also called Student
„t‟ test.
There are two types of „t‟ tests.
1. Paired „t‟ test and 2. Unpaired „t‟ test.
Paired „t‟ test:
When?
-Sample size should be small and drawn randomly.
-Data should be quantitative.
Why?
To paired data of independent observations from one sample only when each
individual give a pair of observations.
-To study the role of a factor or cause when the observations are made before and
after its play. Eg. Of exertion on pulse rate; of a drug on blood pressure‟ of a anti-
depressive drug on the sleep of the patients; of meals on leucocytes count; of
Bengal gram, garlic, onion etc. on cholesterol level in the blood.
-To compare the effect of two drugs, given to same individuals in the sample at two
different occasions. Eg. Adrenaline and non-adrenaline on pulse rate.
-To study the comparative accuracy of two different instruments.
-To compare result of two different laboratory techniques.
-To compare observations made at two different sites in the same body. Eg.
Compare temperature between toes and between fingers or in axilla and mouth.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data: paired observations
-Calculate difference for each pair of values i.e. d=x1-x2
-Calculate mean of difference i.e. d, SDd and SEd
Where SEd = SDd/√ (n-1)
-Calculate „t‟ value
t = Observed difference / SEd
-Degree of freedom = (n-1), refer „t‟ table and find out the probability of the
calculated „t‟ value to the degree of freedom (n-1)
Page | 125
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
OR
Wilcoxon Sign rank sum test is for paired or matched data.
Unpaired „t‟ test:
When?
-Sample size should be small and drawn randomly.
-Data should be quantitative.
Why?
To unpaired data of independent observations made on individual of two different
or separate groups or samples drawn from two populations.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data:
* If one sample „t‟ test:
-Find the difference between the actually observed mean and the claimed mean.
The claimed mean is the Ho. In terms of notations, this difference is x-uo
-Estimate SEx = s/√n, Where s is the standard deviation and s is the number of
subjects in the actually studied sample. This SE measures the inter sample
variability of mean.
-Check the difference obtained in @ is sufficiently large relative to the SE. Calculate
student „t‟ test.
T = (x-uo)/SEx this follows with (n-1) df
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
* Two-sample „t‟ test:
-Calculate SE
Where SE = x√{(1/n1)+(1/n2)}
Where =√{[(X1-x1)2 +(X2-x2)2]/(n1+n2-2)}
Page | 126
-Calculate „t‟ value
t = Observed difference / SE
-Degree of freedom = (n1+n1-2), refer „t‟ table and find out the probability of the
calculated „t‟ value to the degree of freedom (n1+n2-2)
-Comparison
If Z > 1.96, reject Ho, hence observed difference is significant.
If Z < 1.96, Accept Ho, hence observed difference is insignificant.
-Conclusion
OR
Wilcoxon two-sample test is for unpaired data.
CHI-SQUARE TEST (χ2)
When?
-Sample size should be large & small and drawn randomly.
-Data should be qualitative.
Why?
-To test of goodness of fit.
-To test association between two events.
-To test the significance difference between two or more than two proportions.
How?
-See the type of problem and the question to be answered.
-State Ho and H1
-Given data: Observed frequency (O)
-Calculate Expected frequency (E)=(RTxCT)/N Where RT=Row total, CT=Column
total, N =Total number of observations.
-Calculate „χ2‟ value
χ2 =Σ (O-E) 2/E
-Degree of freedom (df)=(r-1)(c-1); (r=row and c=column.)
-Refer χ2 table and find out table value with (r-1)(c-1) df.
-Comparison
If χ2 > table value, reject Ho, hence observed difference is significant.
If χ2 < table value, Accept Ho, hence observed difference is insignificant.
-Conclusion
Association: They are either independent of each other or they are
dependent of each other.
χ2 test of goodness of fir:
Page | 127
It is applied as a test of “goodness of fit” to determine if actual numbers are
similar to expected or theoretical number. K- is the number of classes for χ2
in goodness of fit test. We can find whether the observed frequency
distribution fits to a theoretical distribution of qualitative data. Df=K-1
χ2 test of association between two events:
The simplest setup is when two characteristics under investigation for
association are Yes/No type, or any other two categories such as
Male/Female, Child/Adult, Blood group B/Other than B, Systolic
BP<140/Systolic BP>= 140 etc. The null hypothesis is that there is no
association. The sample data are then examined whether they provide
sufficient evidence against the null hypothesis. Two methods are available
for this setup:
1) Proportion test based on Gaussian distribution and
2) Chi-square test based on contingency table.
Both are equivalent and give same result. They both are applicable only
when „n‟ is large.
The association between two sets of events, this table is also called
association table because they are only two samples and each divided into
two classes, it is called 2x2 contingency table or fourfold table.
Eg. Cataract prevalence in males and females.
Gender Cataract Total
Yes No
Male a b a+b (R1)
Female c d c+d (R2)
Total a+c (C1) b+d (C2) N
The objective of the survey is to find whether or not the prevalence is
associated with the gender. The null hypothesis that there is no association.
If that is true, the prevalence in both the groups should be same. Can you
guess what should be the prevalence if it is same in males as in females?
Alternative formula for χ2 is
χ2= ((ad-bc) 2xN)/{(a+b)(c+d)(a+c)(b+d)}
Page | 128
Note:
-For any χ2 to be valid, it is necessary that the expected frequency in each
cell is at least 5.
-If any cell frequency is less than 5, a different procedure called Fisher‟s
exact test or yate‟s correction / continuity correction.
-Cochran (1954) recommends the use of the exact test, in preference to the
χ2 test with yate‟s correction, (1) if N<20 or (2) if 20<N<40 and the smallest
expected value is less than 5.
-Yate‟s correction is applicable only 2x2 contingency table.
-If the theoretical / Expected frequencies are smaller i.e. <5 then adjoining
classes should be merged together.
Fisher exact test= (R1! R2! C1! C2!)/(N! a! b! c! d!)
Yates correction (χ2)={(|ad-bc|-(N/2)) 2xN}/{R1R2C1C2}
Or. χ2= Σ (| O-E| - ½) 2/E)
Bigger contingency tables:
Now consider duration of AIDS development in various blood groups. The
duration is divided into three groups, viz; <5years, 5-8 years and >=8 years.
This is counted from the time of HIV infection to the appearance of clinical
symptoms of AIDS. The objective is
-To find whether the duration is associated with blood group or not i.e.
whether the duration is different in different blood groups.
-There is no association between blood group and duration of developing
AIDS.
Eg. Duration of developing AIDS in HIV positive of different blood groups of
are as follows:
Duration of
deve-loping
AIDS (Yrs)
Blood group Total
O A B AB
< 5 20 30 48 7 105
5 - 8 21 27 104 19 171
>=8 59 43 98 24 224
Page | 129
Total 100 100 250 50 500
Ho: there is no association between blood groups and the time taken for
development of AIDS in HIV infected cases.
H1: there is a association between blood groups and the time taken for
development of AIDS in HIV infected cases.
Calculate expected frequency of each cell. (12 expected frequency)
Apply χ2 test χ2 = 22.72. ,
df = (r-1)(c-1) =6 Table value χ26,0.05 = 12.59
Comparison: χ2 > χ26,0.05 , R eject Ho.
Conclusion: Hence there is a association between blood groups and the time
taken for development of AIDS in HIV infected cases.
Page | 130
Lesson IX: Objectives: 1. Understand the areas in Education and Learning requiring research 2. Delineate the Priority areas of educational research in India 3. Describe the Ethical problems involved in Educational Research Lesson Outline:
Understand the areas in Education and Learning requiring research including Teaching and Learning methods, Assessment, Evaluation and interventions. Able to formulate priority areas based on the need of the institution.
• The barriers and opportunities of initiating the medical education research • Ways to strengthen the research capabilities in medical education • Describe the specific Ethical problems involved in Educational Research
Introduction:
In India, medical institutions are established with an objective of three legged stool
consisting of research, education and service. Medical teachers hence referred as
„triple threat academicians‟. Medical teachers are thus original and productive
investigators, committed teachers and compassionate practicing physicians. Fourth
obligation recently emphasized is „social responsiveness‟. Medical schools are
confronted with the challenge of making their curricula relevant to the needs of the
times. One response to this challenge is increased interest in research in medical
education.
It has been under fire that Medical education in Asia has colonial-biased, subject-
oriented, teacher-centred, discipline-based, lecture-focused and hospital-based
traditions, which failed “to train medical students appropriately for national health
needs and for medical schools to assume leadership role in shaping services
oriented to the needs of the community”. However there are signs of positive winds
in medical education from government of India and Medical Council of India. The
recent trends in Medical Educational Research suggest that Research is either
quantitative or qualitative. Biomedical or objectifying and holistic or humanizing
research confined to more quantitative, experimental or quasi-experimental
approaches. There is a need to change the direction of research to shift towards
qualitative and descriptive methods
The priority areas in medical education are many and depend on the needs and the
mission of individual institutions. Need based research is a pragmatic approach for
Page | 131
budding medical education researcher or for a newly established medical education
unit. Need based research directly answers the questions related to individual or
institutional needs and is of immediate interest to the faculty and administrators.
The research in medical education has contributed significantly in our
understandings of teaching and learning medicine. Medical education research is
not merely academic and esoteric in nature. On the contrary, the vast majority of
the studies and publications address issues that are practical and of immediate
interest to medical teachers.
Hardens Approaches to Medical Education Research: Experimental Fact-finding Action-research Open ended research Creative research
*Due to the practical and problem solving nature, action research is becoming popular among teacher-researchers Areas in Education & Learning Requiring Research: Outcomes Interventions Teaching & Learning Methods Assessment Evaluation
Exercise: Suggest areas in medical education which need research Prioritize the areas of research by giving justification
Barriers to Medical Education Research: Poor socio-economic conditions Cultural and religious conservatism Lack of relevance Leadership crisis Faculty development Information poverty Unforeseeable short-term research outcomes
Ways to Strengthen Research Capabilities in Medical Education:
Leadership and commitment Relevance Establishment of centre for medical education research Availability of financial resources Research methodology Access to information
Ethical Problems involved in Research: Ethics deals with values and morals. It is based on people's personal value
systems. What one person or group considers to be good or right might be
Page | 132
considered bad or wrong by another person or group. There are three major
approaches to ethics.
1. Deontological Approach - This approach states that we should identify and use a
Universal code when making ethical decisions. An action is either ethical or not
ethical, without exception.
2. Ethical skepticism - Concrete and inviolate ethical or moral standards cannot be
formulated. In this view, ethical standards are not universal but are relative to
one's particular culture, time, and even individual.
3. Utilitarianism - Decisions about the ethics should be based on an examination
and comparison of the costs and benefits that may arise from an action. Note that
the utilitarian approach is used by most people in academia (such as Institutional
Review Boards) when making decisions about research studies.
Ethical Concerns
There are three primary areas of ethical concern for researchers:
1. The relationship between society and science.
• Should researchers study what is considered important in society at a given time?
• Should the government and other funding agencies use grants to affect the areas
researched in a society?
• Should researchers ignore societal concerns?
2. Professional issues.
• The primary ethical concern here is fraudulent activity (fabrication or alteration of
results) by scientists. Obviously, cheating or lying are never defensible.
• Duplicate publication (publishing the same data and results in more than one
journal or other publication) should be avoided.
• Partial publication (publishing several articles from the data collected in one
study). This is allowable as long as the different publications involve different
research questions and different data, and as long as it facilitates scientific
communication. Otherwise, it should be avoided.
3. Treatment of Research Participants
• This is the most fundamental ethical issue in the field of empirical research.
• It is essential that one insures that research participants are not harmed
physically or psychologically during the conduct of research.
Ethical Guidelines for Research with Humans
One set of guidelines specifically developed to guide research conducted by
educational researchers is the ICMR Guidelines. The ICMR is the largest
Page | 133
professional association in the field of Medical Research. Ethical guidelines are
based on the following general principles
1 Essentiality
2 Voluntariness, informed consent and community agreement
3 Non-exploitation
4 Privacy and confidentiality
5 Precaution and risk minimisation
6 Professional competence
7 Accountability and transparency
8 Maximisation of the public interest and of distributive Justice
9 Institutional arrangements
10 Public domain
11 Totality of responsibility
12 Compliance
British Educational Research Association (BERA) has issued Ethical Guidelines for
Educational Research in 2004. It emphasizes that research in education should be
conducted within an ethic of respect of Person, Knowledge, Democratic Values,
Quality of educational research and Academic Freedom. The guidelines framed
under the broad headings of Responsibilities of Participants, Sponsors of research
and community of educational researchers.
Institutional Review Board
The IRB is a committee consisting of professionals and lay people who review
research proposals to insure that the researcher adheres to federal and local ethical
standards in the conduct of the research. Virtually every medical college in
maharashtra has an IRB.
• Researchers must submit a Research Protocol to the IRB for review.
• Three of the most important categories of review are exempt studies (i.e.,
studies involving no risk to participants and not requiring full IRB review),
expedited review (i.e., the process by which a study is rapidly reviewed by
fewer members than constitute the full IRB board), and full board review
(i.e., review by all members of the IRB).
• Although many educational studies are fall into the exempt category, it is
essential that you understand that it is the IRB staff and not the researcher
that makes the decision as to whether a research protocol is exempt. The
IRB will provide the formal documentation of this status for your study.
Page | 134
Research Project Proposals Submitted Proposal I:
Title: To Study the effectiveness of Objective Structured Practical Examination (OSPE) in Assessing I MBBS students Type of Study:
Mixed Type Study Design:
50 students of I MBBS are included in this study. All the students are assessed twice, once with the traditional method and then with OSPE method
Objectives: 1 To conduct practical examination of I MBBS using traditional method
2 To conduct practical examination of I MBBS using OSPE method 3 To compare the performance of students by two methods and analyse
statistically
4 To analyse students satisfaction in both methods
OSPE: 1 Skill 2 Knowledge
3 Attitude(communicating skills) CONSENT CONVINCE
Members of the group:
Dr SS Pandey PDMC, Amaravati Dr SR Pandey PDMC, Amravati
Dr CD Dange SBH Govt Medical College, Dhule Dr Anita Jadhav SBH Govt Medical College, Dhule Dr Prashant Patil SBH Govt Medical College, Dhule
Dr Anita Kale SBH Govt Medical College, Dhule Dr Navid Shah ACPM Medical College, Dhule
Page | 135
Proposal II:
Title: Advantages of PBL over routine methods of teaching Type of Study:
Study Design: PBL for small group of 8 – 10 students. Over a period of 2-3 weeks
Initial disclosure of patients complaints Session I---- Learning issues
Swelling in the neck Swelling in the midline Progress of disease
Change in symptoms Facilitators- 1-2
What are resources to satisfy the knowledge seeking Faculty Internet
Library 3-5 days for study Session II:
Division of students------ Chairman, scribe, topic presenter for each learning issue, 2-3 students as observers
Disclosure no2 Symptoms of thyrotoxicosis/ myxedema Biochemical investigations---TFT, S.Cal, S Cholesterol, Bl.
Sugar, Pulse, BP, Biopsy, ECG, Weight, imaging investigations: X-ray neck, USG,
Radioactive uptake scan
Learning Issues 3-4 days for learning
Session III: Reassemble Regroup Missing investigations FNAC
Expert Common: Ca. Thyroid, other thyroid diseases PBL as topic for research project
1 Control group: Usual way of teaching inpatients 2 Test Group: Those who underwent PBL training
Assessment: Summative for control group and test group Test group may be exposed to formative assessment
Page | 136
Testing by questionnaire on topic of thyroid Validation of questionnaire
Objectives:
Members of the group: Dr Suvarna Joshi BJ Medical College, Pune
Dr Anita Kavatkar BJ Medical College, Pune Dr NK Wani ACPM Medical College, Dhule Dr SS Date ACPM Medical College, Dhule
Dr Mrs AB Patil ACPM Medical College, Dhule Dr Mrs AS Gadre ACPM Medical College, Dhule
Page | 137
Proposal III:
Title: Development of Newer Teaching Tools Type of Study:
Study Design: Qualitative Research Focus Group Discussion
Objectives: To create better physician
Methods: Using tools such as CDs, Photographs and Case studies
Equipment Required:
Members of the group: Residents of Paediatrics, Medicine, Obstetrics & Gynecology and Physiology
Page | 138
Proposal IV:
Title: Assessment of perceptions of third year medical students regarding curriculum in community medicine at Shri. Bhausaheb Hire Govt. Medical College, Dhule .
Type of Study: Cross sectional Study Design: Qualitative Research utilising Focus Group Discussion
Objectives: 1 To explore the present status and deficiencies if any in the teaching
and learning of community medicine 2 To suggest improvements in the teaching process
Participants: VI, VII & VIII Semester students.
As you know, important objective of medical education at graduate level is preparing basic doctor, who should be able to tackle common health problems & fulfilling the responsibilities of first contact public health expert
. In Medical Colleges, Dept. of Community Medicine carries the responsibility of educating & training undergraduate students in the context of above objective. But it is observed that attendance & involvement
of the students in the teaching learning process in community medicine subject is becoming less & less. After passing out, students feel that this
subject is useless subject. This is harmful to the important objective of the medical education. That is why the present study has been planned to explore the deficiencies
in the curriculum & lacunae in teaching learning process at departmental level from the students view. Perceptions of the students will better be appreciated through focus
group discussion.
Members of the Group: Dr RT Ankushe Dr Sarika P Patil Dr Vijay Singh Dr Dr Amol Patil
Dr Samir Sheik
Page | 139
Workshop on " Medical Educational research: Concepts & Methodologies" held at Medical Education Unit, SBH Government Medical College,
on 29 & 30 July, 2009 Sn Name Designation College
1 Dr S S Date Professor ACPM Medical College
2 Dr Mrs Alka Patil Professor ACPM Medical College
3 Dr Wasim Sheik Assistant Professor Pramukhswami Medical College
4 Dr Mrs Jadhav Assistant Professor SBH Government Medical College
5 Dr S N Wanjari Assistant Professor SBH Government Medical College
6 Dr Sarika Patil Assistant Professor SBH Government Medical College
7 Dr Arundhati Gadre Assistant Professor ACPM Medical College
8 Dr Rajashri damle PG Student ACPM Medical College
9 Dr Priya Bagle PG Student ACPM Medical College
10 Mr Sunil Kumar Patil Statistician ACPM Medical College
11 Dr Ajit Pathak Associate Professor SBH Government Medical College
12 Dr Naveenchandra Wani Professor ACPM Medical College
13 Dr Suvarna Joshi Associate Professor BJ Medical College
14 Dr Anita Kavatkar Associate Professor BJ Medical College
15 Dr Mrs Sushma Pandey Professor Dr PDM Medical College
16 Dr Santosh Pandey Associate Professor Dr PDM Medical College
17 Dr Anita Kale Assistant Professor SBH Government Medical College
18 Dr Danish Memon PG Student ACPM Medical College
19 Dr Shree deshmukh PG Student ACPM Medical College
20 Dr Jasleen Mavi PG Student ACPM Medical College
21 Dr Prabhneet Kahlon PG Student ACPM Medical College
22 Dr Asma Kahn PG Student ACPM Medical College
23 Dr AN Borde Associate Professor SBH Government Medical College
24 Dr RT Ankushe Associate Professor SBH Government Medical College
25 Dr CD Dange Assistant Professor SBH Government Medical College
26 Dr Mrs Kulkarni Assistant Professor SBH Government Medical College
27 Ms Shilpa Tyagi Student SBH Government Medical College
28 Dr Vujay Singh Assistant Professor LTM Medical College
29 Dr Santosh Suryawanshi Assistant Professor LTM Medical College
30 Dr Prayag Makwana PG Student ACPM Medical College
31 Dr Sandeep Gaidhani PG Student ACPM Medical College
32 Dr Puneet Patil PG Student ACPM Medical College
33 Dr Vishal Gaeikwad PG Student ACPM Medical College
34 Dr Kishore Suryawanshi PG Student ACPM Medical College
35 Dr Mayur Kahate PG Student ACPM Medical College
36 Dr Vaibhav Jain Assistant Professor ACPM Medical College
37 Dr K K Borgaonkar Assistant Professor SBH Government Medical College
38 Dr Naveed Agha PG Student ACPM Medical College
Page | 140
39 Dr Prakash Humlekar Assistant Professor ACPM Medical College
40 Mr Amitesh Khare Student SBH Government Medical College
41 Dr N N Shah Assistant Professor ACPM Medical College
42 Dr Prashant Patil Assistant Professor SBH Government Medical College
43 Dr Arun More Associate Professor SBH Government Medical College
44 Dr Amol Patil Assistant Professor SBH Government Medical College
45 Dr Samir Sheik PG Student SBH Government Medical College
Page | 141
Teaching & Learning Methods Suggested for the Revised MBBS Curriculum
Method
First MBBS Second MBBS Final MBBS
Anatomy
Physiology
Biochemistry
Pathology
Microbiology
Forensic
Medicine
Pharmacology
Community
Medicine
Medicine
Paediatrics
Dermatology
Psychiatry
TB &
Chest
Surgery
Orthopaedics
ENT
Ophthalmology
Obstetrics &
Gynecology
Anesthesia
Lectures
Structured
interactive
sessions
Small group
discussion
a)
Demonstrati
ons.
b) Tutorials.
c) Seminars.
d) Problem
Based
Learning.
Focused
group
discussion
(FGD)
Projects Participator
y learning
appraisal
(PLA)
Family and
community
visits
Institutional
visits Practical
including
demonstrati
ons
Problem
based
exercises
Video clips
Page | 142
Written case
scenario Self
learning
tools
Interactive
learning
e-modules Dissection /
Prosected
parts
demonstrati
ons /
Instructions
on
mannequins
.
Skills Lab
with CDs of
various
stages of
dissection.
Histology
Lab.
Surface
marking.
Imaging
anatomy
Lab.
Visit to the
museum.
Preparation
of scientific
article.
Preparation
of practical
drawing
book
Role Play
Seminars
Algorithms
Integrated
Page | 143
teaching
Field visits
Problem based paper & real cases
Simulated Patient Management Problems
Case Studies
Tutorials
Workshops
One to one teaching in theatre
Departmental Morbidity, Audit, Journal Club
Self Assignments
![Volta Inception Workshop Final Report[1]](https://img.pdfslide.net/doc/110x75/577d26741a28ab4e1ea142a9/volta-inception-workshop-final-report1.jpg)
