Upload
gizzmo310
View
217
Download
0
Embed Size (px)
Citation preview
8/9/2019 Summit 2010 Measures Aidala
1/25
Wheres the Evidence?Wheres the Evidence?Measurement ChallengesMeasurement Challenges
Angela A.Angela A. AidalaAidala
Columbia University Mailman School of Public HealthColumbia University Mailman School of Public Health
Columbia Center for Homelessness Prevention StudiesColumbia Center for Homelessness Prevention Studies
HOUSING AND HIV/AIDS RESEARCH SUMMIT VHOUSING AND HIV/AIDS RESEARCH SUMMIT V
June 2, 2010June 2, 2010 Toronto,OntarioToronto,Ontario
8/9/2019 Summit 2010 Measures Aidala
2/25
Presentation GoalsPresentation Goals
Discuss data collection methods and tools for:Discuss data collection methods and tools for:
Describing/ documenting programsDescribing/ documenting programs
What is the program about? What does it do?What is the program about? What does it do?
Describing/ documenting contextsDescribing/ documenting contexts
What are risk promoting or protective contexts?What are risk promoting or protective contexts?
Answering process questionsAnswering process questions
How did it happen? How does it operate?How did it happen? How does it operate?
Answering outcomes questionsAnswering outcomes questions
So what? What difference did it make in the lives ofSo what? What difference did it make in the lives of
individuals, families, communities?individuals, families, communities?
8/9/2019 Summit 2010 Measures Aidala
3/25
Are you just pissing and moaning, or can you verify what youre saying with data?
Credible EvidenceCredible Evidence
8/9/2019 Summit 2010 Measures Aidala
4/25
Collecting DataCollecting Data
Different data collection strategies areappropriate for answering research questions
Surveys / interviews - structured conversation using
a questionnaire or interview guide
Focus groups (group interviews)
Records review (staff reports)
Administrative data (surveillance indicators)
Biomedical assessment (x-rays, blood tests)
Ethnography, observational analysis (being there)
Environmental assessments (material, social context)
8/9/2019 Summit 2010 Measures Aidala
5/25
MeasuresMeasures
Most of what we want to study cannot be directly seen,heard, touched, etc.
- Racism, stable housing, good health
Need procedures to define concepts so that they can be
measured
- Stable housing = same address 6+ months
Need to be RELIABLE (consistent result when use tool)
Need to be VALID ( is true measure of phenomena)
Need to be appropriate to answer research question
Need to be appropriate for different populations
8/9/2019 Summit 2010 Measures Aidala
6/25
What Should be Measured?What Should be Measured?
Contexts: Social, Economic, Political, Cultural, ServiceContexts: Social, Economic, Political, Cultural, Service
Census dataCensus data
City/county dataCity/county data
HIV surveillanceHIV surveillance
Service directoriesService directories
Newspaper archivesNewspaper archives
Informant interviewsInformant interviews
Need to consider: Peoples perceptions and experiencesNeed to consider: Peoples perceptions and experiences
Useful tools: HUDuser.gov, AmericanUseful tools: HUDuser.gov, American FactFinderFactFinder
8/9/2019 Summit 2010 Measures Aidala
7/25
Ex: Distressed NeighborhoodEx: Distressed Neighborhood
Neighborhood operationally defined as blockgroup distressed indicated by summary %
households below 75% of the poverty line
vacant housing units
female-headed households with children < 6yrs
unemployment rate
public assistance income
crimes against persons per 1,000 population
property crime per 1,000 population
Lee (2009). Factors Associated With Departure From Supported Independent Living Programs
8/9/2019 Summit 2010 Measures Aidala
8/25
What Should be Measured?What Should be Measured?
Process data: Information about how a programProcess data: Information about how a programworks and lessons learned implementing the programworks and lessons learned implementing the programthat could guide replication or expansionthat could guide replication or expansion
Routine agency documentsRoutine agency documents
Informant interviewsInformant interviews
Meeting notesMeeting notes
ObservationsObservations
Useful strategy: MonthUseful strategy: Month--byby--month chronologymonth chronology
Useful tool: Standard form for meeting notesUseful tool: Standard form for meeting notes
8/9/2019 Summit 2010 Measures Aidala
9/25
Ex: Record of Issues and DecisionsEx: Record of Issues and Decisions
8/9/2019 Summit 2010 Measures Aidala
10/25
Outcome Indicators: Specific items of informationOutcome Indicators: Specific items of informationthat track a programs success on outcomesthat track a programs success on outcomes
Levels of outcomes can focus on any of:Levels of outcomes can focus on any of:
---- Individual (client)Individual (client)---- Agency, organization, programAgency, organization, program
---- Group, social network (family, local club scene)Group, social network (family, local club scene)
---- Service SystemService System---- BroaderCommunityBroaderCommunity
What Should be Measured?What Should be Measured?
10
8/9/2019 Summit 2010 Measures Aidala
11/25
Outcome indicators can focus on:Outcome indicators can focus on:
What Should be Measured?What Should be Measured?
11
Desired Change: Types ofOutcomes:
Attitudes Improved self-efficacy
Perceptions Increased perception of need
Knowledge Greater awareness of services
Skills Improved problem solving
Behaviors Increased condom use
Conditions Reduction of housing instability
8/9/2019 Summit 2010 Measures Aidala
12/25
12
Interviewing Clients forInterviewing Clients forService Delivery AND ResearchService Delivery AND Research
Does Not Mean:Does Not Mean:
Staff cannot have theirStaff cannot have their
own styleown style Information restricted toInformation restricted to
check listscheck lists
All information must beAll information must be
obtained at one timeobtained at one time Staff cannot establishStaff cannot establish
rapport with clientsrapport with clients
Does Mean:Does Mean:
Intent of questions must beIntent of questions must be
understoodunderstood
Systematic way to gatherSystematic way to gatherinformationinformation
Information about the sameInformation about the same
client must be linkedclient must be linked
Interviewers cannot leadInterviewers cannot leadclients to answerclients to answer
8/9/2019 Summit 2010 Measures Aidala
13/25
8/9/2019 Summit 2010 Measures Aidala
14/25
14
Some RulesSome Rules
There should be good reason for asking eachThere should be good reason for asking eachquestionquestion
Use established questions and scales whereUse established questions and scales whereappropriateappropriate e.g. SFe.g. SF--12, CDQ12, CDQ
Begin with topics the client might want to discussBegin with topics the client might want to discussNOT demographicsNOT demographics
Provide interviewer instructions on formProvide interviewer instructions on formincluding skip directionsincluding skip directions
For closed questions make sure all alternativesFor closed questions make sure all alternativesare presentedare presented
No double barrel questionsNo double barrel questions
Shortest is not always best !Shortest is not always best !
8/9/2019 Summit 2010 Measures Aidala
15/25
HOPWA INTAKE FORMDate:_______________________ Intake Worker:____________________________________
Referral Source:_______________________________________________________________
Legal Name:________________________________________ Other Name_______________
Phone:_________________________________Beeper:_______________________________
Current Address:______________________________________________________________
____________________________________________________________________________
Last Known Address___________________________________________________________
____________________________________________________________________________
Date of Birth:_________________ Age:_______ Place of Birth: City____________________
State:________________ Country:__________________ U. S. Citizen: Y N
Sex:________________ Sexual Orientation:_______________ SS#____________________
Race:_____________________ Ethnicity:___________________ Religion:_______________
Emergency Contact:
Name/Relationship: _____________________________________ Phone:________________
PRESENTING PROBLEM AND SERVICES REQUEST:
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
Other agency involvement:______________________________________________________
____________________________________________________________________________
Services Requested (circle)
shower food clothes shelter medical legal birth control/ condoms
counseling help with family problems GED job training identification
public assistance Medicaid permanent housing HIV testing HIV education
8/9/2019 Summit 2010 Measures Aidala
16/25
Client ID # |___|___|___|___|
HARBOR OF HOPE PROGRAM ASSESSMENT FORM
Todays Date:____ /____/____ Interviewer:________________________
mm dd yyyy
1. What is your name________________________________________________
1a. Preferred name________________________________________________
2. What brings you here today? ( State in clients own words):
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
Services Requested (circle all that apply)
01...Shower 06...Birth control 11Job Training 02...Food 07Condoms 12Public Assistance
03...Clothes 08... Counseling, MH 13...Medical services
04...Emergy Shelter 09AOD services 14...Medicaid
05...Permt Housing 10...GED, education 15HIV testing
Well talk more about these issues later in the questionnaire but first some
background questions .
8/9/2019 Summit 2010 Measures Aidala
17/25
17
TYPICAL QUESTION ABOUT CLIENT HEALTH:
Health:______________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
STANDARDIZED HEALTH MEASURE:
Next are some questions about your health at the present time.
1. In general would you say your health is. . .
Excellent. . . . . . . . . . . 1
Very Good. . . . . . . . . . 2
Good. . . . . . . . . . . . . . 3
Fair . . . . . . . . . . . . . . . 4
Poor. . . . . . . . . . . . . . . 5
2. The following items are about activities you might do during a typical day. Does your health
now limit you in these activities . . .
8/9/2019 Summit 2010 Measures Aidala
18/25
18
Indicators: Physical HealthIndicators: Physical Health
CD4Count provide categories (i.e. belowCD4Count provide categories (i.e. below200, above 500)200, above 500)
Viral loadViral load undetectableundetectable v.v. detectabledetectable
Score on health functioning measure:Score on health functioning measure:MOSMOS SF12SF12
General health question: Health nowGeneral health question: Health nowexcellent, very good, good, fair, or poor.excellent, very good, good, fair, or poor.
8/9/2019 Summit 2010 Measures Aidala
19/25
19
Indicators: Mental HealthIndicators: Mental Health
Client underClient under--recognition of need for mentalrecognition of need for mentalhealth serviceshealth services
Systematic screening v. seems depressedSystematic screening v. seems depressed
Use standardized measure as well as selfUse standardized measure as well as self--report of problemsreport of problems
Diagnostic screeners (Diagnostic screeners (e.ge.g CDQ) provide moreCDQ) provide more
information than symptom checklistsinformation than symptom checklists
8/9/2019 Summit 2010 Measures Aidala
20/25
20
Engagement with Medical CareEngagement with Medical Care
Usual source of medical care can beUsual source of medical care can beambiguousambiguous
Need also ask: How many visits for HIV careNeed also ask: How many visits for HIV care
from a doctor or other medical provider didfrom a doctor or other medical provider didyou have in the past 6 months?you have in the past 6 months?
ER visit, hospitalized over night or longer inER visit, hospitalized over night or longer inthe past 6 monthsthe past 6 months
Stopped going to the doctor for any reasonStopped going to the doctor for any reasonfor 6 months or longerfor 6 months or longer
8/9/2019 Summit 2010 Measures Aidala
21/25
21
ASKING ABOUT SENSITIVE TOPICSASKING ABOUT SENSITIVE TOPICS
Provide brief introduction to question and giveProvide brief introduction to question and giveassurance of confidentialityassurance of confidentiality
Ask whether client hasAsk whether client has everever engaged inengaged inbehavior before asking about current behaviorbehavior before asking about current behavior
Ask how often or when was the first ratherAsk how often or when was the first ratherthan did you everthan did you ever
Staff discomfort with questions will make clientStaff discomfort with questions will make clientuncomfortableuncomfortable
8/9/2019 Summit 2010 Measures Aidala
22/25
22
5. Have yo u e ver use d any o f the follow ing, eve n one tim e...Go dow n entire list, then go back a nd for a ny YES ask : Did you u se (drug) in the pa st six m onth s?
Ever usedUse d pas t sixmonths
SUBSTA NCE Yes No Yes No
1. M arijua na, has his h 1 0 1 0
2. Powd ered C ocain e 1 0 1 0
3. C rac k (roc k ) 1 0 1 0
4. H eroin or s peedb all 1 0 1 0
5. A m phetem ines o r m etha nphetem ines 1 0 1 0
6. Inhala nts , p oppe rs 1 0 1 0
7. A ny other dru g us ed regular ly,3+ t imes wk o r more often:
Drug: _____________________
1 0 1 0
5. Have yo u e ver use d any o f the follow ing, eve n one tim e...Go dow n entire list, then go back a nd for a ny YES ask : Did you u se (drug) in the pa st six m onth s?
Ever usedUse d pas t sixmonths
SUBSTA NCE Yes No Yes No
1. M arijua na, has his h 1 0 1 0
2. Powd ered C ocain e 1 0 1 0
3. C rac k (roc k ) 1 0 1 0
4. H eroin or s peedb all 1 0 1 0
5. A m phetem ines o r m etha nphetem ines 1 0 1 0
6. Inhala nts , p oppe rs 1 0 1 0
7. A ny other dru g us ed regular ly,3+ t imes wk o r more often:
Drug: _____________________
1 0 1 0
E . S U B S T A N C E U S E
M a n y p e o p le h a v e e x p e r ie n c e s w i h a lc o h o l a n d u s e o f d if f e r e n t s u b s t a n c e s . W e w o u ld l i e
to k n o w a b o u t yo u r e x p e r ie n c e a s p a r t o f y o u r h e a l t h p r o f i le . I d li k e to r e m in d y o u t h a t
e v e r y t h in g y o u s a y is s t r ic t ly c o n f id e n t ia l a n d p r o t e c t ed
E . S U B S T A N C E U S E
8/9/2019 Summit 2010 Measures Aidala
23/25
23
DATA QUALITY ASSURANCE
All persons using forms must be trainedAll persons using forms must be trained
Dont assume knowledge or comfort level ofDont assume knowledge or comfort level ofstaffstaff
Use training documents e.g.Use training documents e.g. QxQQxQ
Pilot test data collection with consumersPilot test data collection with consumers
Review completed forms for clarity andReview completed forms for clarity andcompletenesscompleteness
8/9/2019 Summit 2010 Measures Aidala
24/25
My question is, Are we making a difference?
8/9/2019 Summit 2010 Measures Aidala
25/25
25
RESOURCES FOR MEASURESRESOURCES FOR MEASURES
Tools for HOPWA Outcome Based EvaluationTools for HOPWA Outcome Based Evaluationhttp://www.aidshousing.org
Community ToolboxCommunity Toolbox -- Resources for HealthyResources for HealthyCommunitiesCommunities http://ctb.ku.edu/en/Default.htm
Columbia Center for Homelessness PreventionColumbia Center for Homelessness PreventionStudiesStudies -- Measures LibraryMeasures Library http://www.columbia-chps.org/resources/Homelessness-and-Housing-Measures
MOSMOS--SF12 v2SF12 v2 http://www.sfhttp://www.sf--36.org/tools/sf12.shtml36.org/tools/sf12.shtml
CDQ & Training ManualCDQ & Training Manual [email protected]@columbia.edu
HRSA, CDC, SAMHSA online guidesHRSA, CDC, SAMHSA online guides