By: Omar AL-Rawajfah, RN, PhDy jf , ,

Measurement and the assessment of quality of dataAdvantage of MeasurementAdvantage of MeasurementFactors Contribute to Measurement ErrorCriteria for Selection of Data-Collection IInstrument◦ Practicality◦ Reliability

Validity◦ Validity Assessment of qualitative data◦ Credibility◦ Dependability◦ Dependability◦ Confirmability◦ Transferability

Rules for assigning numeric values to qualities of objects to designate thequalities of objects to designate the quantity of attribute.Quantification: assignment of numbersQuantification: assignment of numbers to certain attribute◦ Temp., weight, BP, etc are physical attributesp , g , , p y◦ Non-physical attributes such as pain,

depression, fatigue, anxiety, etc

Remove much of the guesswork in gatheringRemove much of the guesswork in gathering info.Info obtained by measurement is objectiveInfo. obtained by measurement is objective and almost identicalProvide precise infoProvide precise info.Constitutes a language of communication. Numbers are less vague than wordsNumbers are less vague than words

Even with the best measuring instrumentsEven with the best measuring instruments there is certain margin of errorObtained score = True score ± ErrorObtained score = True score ± ErrorTrue score is a hypothetical entity, it can never be knownnever be knownError of measurement is the difference between the observed score and the truebetween the observed score and the true score

Situational contaminantsSituational contaminantsResponse-set biasTransitory personal factorsTransitory personal factorsAdministration variationI l iInstrument clarityItem sampling

What data will be collected?What data will be collected?How will the data be collected?Who will collect the data?Who will collect the data?Where will the data be collected?Wh ill h d b ll d?When will the data be collected?

Cost and appropriateness to the studyCost and appropriateness to the study purposeTime to administer the instrumentTime to administer the instrumentAny special motor skills or language abilities needed to administer the instrumentneeded to administer the instrument

Degree of consistency with which theDegree of consistency with which the instrument measures the attributeThree aspects of reliability:Three aspects of reliability:◦ Stability: same scores are obtained when the

instrument is used with the same people on different occasionsdifferent occasions

Test-retest reliability is used to indicate the stability of the instrumentReliability coefficient: ranged from .00 – 1.00. satisfactory scores are 0.70 and above

◦ Internal consistency: to which extent all of the instrument subpart are measuring thethe instrument subpart are measuring the same characteristics

Split-half technique: alpha 0.00 – 1.00.Cronbach’s alpha or coeficient alpha 0.00 – 1.00.p p

◦ Equivalence: used when different observer are using the same character for the same subjects

l b lInterrater reliability: .00 – 1.00.Interpretation of reliability coefficients◦ Test-retest tend to decline as the time

b h 2 ibetween the 2 measures increase◦ More homogeneous sample, the lower

reliability coefficientTh hi h h li bili ffi i h◦ The higher the reliability coefficients, the better instrument

The degree to which an instrument measureThe degree to which an instrument measure what it is supposed to be measuringA measurement device that is not reliableA measurement device that is not reliable can’t possibly be validAn instrument can be reliable howeverAn instrument can be reliable, however, without being validFace validity: the instrument looks as thoughFace validity: the instrument looks as though it is measuring the appropriate character

Three aspects of validity:f f◦ Content validity: is of particular relevance for

instrument that test knowledge in a specific areaNo totally objective methods for measuring the

t t liditcontent validityExperts in the concern area are often called to analyze the items

◦ Criterion related validity: relationship between◦ Criterion-related validity: relationship between the scores on the instrument and some external criterion

Validity coefficient: 00 – 1 00Validity coefficient: .00 – 1.00◦ Construct validity: concern with degree to which

the instrument measure the character of interestKnown-group techniqueKnown-group techniqueFactor analysis

Trustworthiness: terms used to reflect the lit f d t i th lit ti hquality of data in the qualitative research

Four aspects used to evaluate the quality of the qualitative dataqualitative data◦ Credibility:

Degree of the confidence in the truth of the dataParallel to the validity in the quantitative researchProlonged engagement and persistent observationTriangulation:Triangulation:

Data source triangulationInvestigator triangulationTheory triangulationTheory triangulationMethod triangulation

◦ Dependability: Stability of data over time and over conditions, parallel to reliability

Stepwise replication: two teams deal with dataStepwise replication: two teams deal with data sources separately & conduct independent inquires

◦ Confirmability: objectivity and neutrality of th d tthe data

Agreement between two or more independent people about certain relevance meaningInquiry audit and audit trail

◦ Transferability: generalizability of the dataThick description: detail description of researchThick description: detail description of research settings and context and the process of inquiry

Quantitative researchQuantitative research◦ Does the framework appear to be congruent to the

research questionresearch question◦ Do the instruments used to measure the study

variables are appropriate◦ Any evidence of validity are reliability◦ Are the validity and reliability measures were

appropriate and ithin the acceptable rangappropriate and within the acceptable rang

Qualitative researchQualitative research◦ Which techniques did the researcher use to enhance

the data credibilitythe data credibility◦ Which techniques did the researcher use to enhance

the data dependability ◦ Which techniques did the researcher use to enhance

the data confirmability

BY: Omar AL-Rawajfah, RN, PhD

Involves a miniature trial version of the actualInvolves a miniature trial version of the actual studyPeople selected for pilot testing are of thePeople selected for pilot testing are of the same characteristics of the main study samplesampleAll planned procedure, protocols and data collection instruments are usedcollection instruments are used

Could prevent conducting a large-scale studyCould prevent conducting a large-scale study that might be an too expensive to doExamine issues related to the design sampleExamine issues related to the design, sample size, data collection procedure, and analysis approachesapproachesTest new instrument

Estimate how long it will take to conduct theEstimate how long it will take to conduct the data collectionAfter the pilot study necessary revisionsAfter the pilot study, necessary revisions should be madeAnother pilot study may be needed if manyAnother pilot study may be needed if many changes are implemented

Usually begins before even data collectionUsually, begins before even data collectionPrepare dummy tableMake sure that questionnaire is completedMake sure that questionnaire is completed appropriatelyDecision about missing datagAppropriate data coding and enteringAppropriate procedure to retrieve data pp p pcollection forms