Seth M. Noar, Ph.D.Seth M. Noar, Ph.D.Associate ProfessorAssociate Professor

Department of CommunicationDepartment of CommunicationUniversity of KentuckyUniversity of Kentucky

Efficacy of Computer-based Efficacy of Computer-based Interventions Interventions

in HIV Prevention:in HIV Prevention:Evidence from Meta-AnalysisEvidence from Meta-Analysis

AcknowledgementsAcknowledgements Collaborators on meta-analyses: Collaborators on meta-analyses:

Hulda Black, M.B.A, Larson Pierce, M.Ed.Hulda Black, M.B.A, Larson Pierce, M.Ed.

Computer-based Computer-based InterventionsInterventions

Computer-based interventions have many advantagesComputer-based interventions have many advantages

Once developed, cost to deliver is minimalOnce developed, cost to deliver is minimal

Intervention fidelity is maintainedIntervention fidelity is maintained

Content can be customized at the Content can be customized at the individualindividual level level

Interactivity and multimedia offer engaging user Interactivity and multimedia offer engaging user features (may foster skills acquisition and behavioral features (may foster skills acquisition and behavioral change)change)

Flexible disseminations channels – clinics, community Flexible disseminations channels – clinics, community agencies, prisons, Internet, etc.agencies, prisons, Internet, etc.

Computer-based Computer-based InterventionsInterventions

Definition: Use computer technology as primary or sole Definition: Use computer technology as primary or sole medium to deliver intervention medium to deliver intervention

There are several different types of CBI’sThere are several different types of CBI’s

Group targetedGroup targeted

Individually tailoredIndividually tailored

Virtual decision-makingVirtual decision-making

Group TargetedGroup Targeted

Intervention designed for a particular target audienceIntervention designed for a particular target audience May include a variety of content, activities, and/or May include a variety of content, activities, and/or

multimedia designed with that particular audience in mindmultimedia designed with that particular audience in mind Typically delivered on screen, using a local computer Typically delivered on screen, using a local computer

(e.g., CD-ROM) or via the Internet(e.g., CD-ROM) or via the Internet

Lightfoot, M., Comulada, W. S., & Stover, G. (2007). Computerized HIV preventive intervention for adolescents: Indications of efficacy. American Journal of Public Health, 97(6), 1027-1030.

Individually TailoredIndividually Tailored

Intervention which tailors content to each individual, based Intervention which tailors content to each individual, based on an assessment of characteristics of that individualon an assessment of characteristics of that individual

Traditionally produced tailored print materials; now Traditionally produced tailored print materials; now increasingly delivered on screen using local computers or increasingly delivered on screen using local computers or via the Internetvia the Internet

Redding, C. A., Morokoff, P. J., Rossi, J. S., & Meier, K. S. (2008). A TTM-tailored condom use intervention for at-risk men and women. In T. Edgar, S. M. Noar & V. S. Freimuth (Eds.), Communication perspectives on HIV/AIDS for the 21st century (pp. 423-428). New York: Lawrence Erlbaum.

Noar, S. M., Webb, E. M., Van Stee, S. K., Redding, C. A., Feist-Price, S., Crosby, R., & Troutman, A. (2010). Using computer technology for HIV prevention among African Americans: Development of a tailored information program for safer sex (TIPSS). Manuscript submitted for publication.

Noar, S. M., Webb, E. M., Van Stee, S. K., Redding, C. A., Feist-Price, S., Crosby, R., & Troutman, A. (2010). Using computer technology for HIV prevention among African Americans: Development of a tailored information program for safer sex (TIPSS). Manuscript submitted for publication.

Virtual Decision-MakingVirtual Decision-Making Also referred to as ‘interactive video’Also referred to as ‘interactive video’ Intervention that simulates dating and sexual situations and Intervention that simulates dating and sexual situations and

allows the user to make choices at various decision points allows the user to make choices at various decision points and witness the consequences of those decisionsand witness the consequences of those decisions

Typically delivered on screen using local computerTypically delivered on screen using local computer

Roberto, A. J., Zimmerman, R. S., Carlyle, K. E., Abner, E. L., Cupp, P. K., & Hansen, G. L. (2007). The effects of a computer-based pregnancy, STD, and HIV prevention intervention: A nine-school trial. Health Communication, 21(2), 115-124.

Read, S. J., Miller, L. C., Appleby, P. R., Nwosu, M. E., Reynaldo, S., Lauren, A., et al. (2006). Socially optimized learning in a virtual environment: Reducing risky sexual behavior among men who have sex with men. Human Communication Research, 32(1), 1-34.

Research QuestionsResearch Questions

Purpose: Meta-analysis of computer technology-based HIV Purpose: Meta-analysis of computer technology-based HIV prevention interventionsprevention interventions

Research QuestionsResearch Questions

Are CBIs efficacious in changing theoretical mediators Are CBIs efficacious in changing theoretical mediators of safer sex and sexual risk behaviors?of safer sex and sexual risk behaviors?

How does the effect size compare to human-delivered How does the effect size compare to human-delivered interventions?interventions?

What factors moderate the efficacy of interventions?What factors moderate the efficacy of interventions?

MethodsMethods

MethodMethod Exhaustive search of published and unpublished Exhaustive search of published and unpublished

studies undertaken (through March, 2008)studies undertaken (through March, 2008)

Numerous keywords used to search Medline, Numerous keywords used to search Medline, PsycINFOPsycINFO

Forward citation searches conducted on all articles Forward citation searches conducted on all articles foundfound

Each article reference list examinedEach article reference list examined

Several review articles examinedSeveral review articles examined

Email sent out to 44 researchers in field; 17 repliesEmail sent out to 44 researchers in field; 17 replies

Method (cont’d)Method (cont’d) During search it became clear that the literature should be During search it became clear that the literature should be

divided into 2 meta-analyses:divided into 2 meta-analyses:

1.1. Meta-analysis focusing on CBI’s to change theoretical Meta-analysis focusing on CBI’s to change theoretical mediators of safer sex (e.g., knowledge, attitudes, self-mediators of safer sex (e.g., knowledge, attitudes, self-efficacy, etc.)efficacy, etc.)

To be published in 2010 in To be published in 2010 in Human Communication Research (A Human Communication Research (A Journal of the International Communication Association)Journal of the International Communication Association)

During publication process, this paper was updated through August, During publication process, this paper was updated through August, 20092009

2.2. Meta-analysis focusing on CBI’s to change sexual risk Meta-analysis focusing on CBI’s to change sexual risk behaviors (condom use, unprotected sex, number of partners, behaviors (condom use, unprotected sex, number of partners, etc.)etc.)

Published in 2009 in Published in 2009 in AIDS (Official Journal of the International AIDS AIDS (Official Journal of the International AIDS Society)Society)

Inclusion CriteriaInclusion Criteria Computer technology-based HIV prevention Computer technology-based HIV prevention

interventionintervention

Directed at individuals of HIV-negative or unknown Directed at individuals of HIV-negative or unknown serostatusserostatus

Experimental design in which individuals were Experimental design in which individuals were randomized to conditionsrandomized to conditions

Also:Also:

1.1. Measured a theoretically-oriented STD or HIV-Measured a theoretically-oriented STD or HIV-related outcome measurerelated outcome measure

2.2. Measured condom use or unprotected sex as Measured condom use or unprotected sex as dependent variabledependent variable

Study 1 - MediatorsStudy 1 - Mediators Total Total NN = 61 = 61

Reasons for exclusion:Reasons for exclusion:

No theoretically-oriented outcome measure (pilot No theoretically-oriented outcome measure (pilot or behavioral study) (or behavioral study) (kk = 21) = 21)

Not a randomized design (no control) (Not a randomized design (no control) (kk = 8) = 8)

No computer technology (No computer technology (kk = 8) = 8)

Not an intervention study (Not an intervention study (kk = 5) = 5)

Redundant publication (Redundant publication (kk = 1) = 1)

A final set of 18 articles containing 20 studies were A final set of 18 articles containing 20 studies were included in the review (2 were unpublished)included in the review (2 were unpublished)

Study 2 - BehaviorStudy 2 - Behavior Total Total NN = 49 = 49

Reasons for exclusion:Reasons for exclusion:

No measure of condom use or UPS (No measure of condom use or UPS (kk = 18) = 18)

Not a randomized design (no control) (Not a randomized design (no control) (kk = 7) = 7)

No computer technology (No computer technology (kk = 6) = 6)

Not an intervention study (Not an intervention study (kk = 5) = 5)

Intervention with HIV positive persons (Intervention with HIV positive persons (kk = 1) = 1)

A final set of 12 studies were included in the review (7 A final set of 12 studies were included in the review (7 published, 5 unpublished)published, 5 unpublished)

CodingCoding

Studies coded by 2 independent codersStudies coded by 2 independent coders Coded participant, intervention, and Coded participant, intervention, and

methodological moderatorsmethodological moderators Coding reliabilities very good in both studiesCoding reliabilities very good in both studies Most coding categories had 100% agreement Most coding categories had 100% agreement

and kappa = 1.0and kappa = 1.0

Effect Size (ES)Effect Size (ES)

Standardized mean difference, Standardized mean difference, d d (difference in (difference in treatment and control means divided by pooled treatment and control means divided by pooled SD) used as ES indicatorSD) used as ES indicator

ES’s calculated from data provided in study ES’s calculated from data provided in study reports – typically means, standard deviationsreports – typically means, standard deviations

ES outcomes coded on all relevant variables in ES outcomes coded on all relevant variables in reportreport

Longest follow-up period codedLongest follow-up period coded Positive effect indicated by + sign, negative effect Positive effect indicated by + sign, negative effect

indicated by – signindicated by – sign

Meta-AnalysisMeta-Analysis

ES’s weighted by their inverse variance and ES’s weighted by their inverse variance and combined using fixed and random effects combined using fixed and random effects procedures (Cooper & Hedges, 1994; Lipsey procedures (Cooper & Hedges, 1994; Lipsey & Wilson, 2001)& Wilson, 2001)

Heterogeneity of effect sizes examined using Heterogeneity of effect sizes examined using QQ statistic statistic

Categorical moderators statistically compared Categorical moderators statistically compared ((QBQB statistic) statistic)

Continuous moderators examined – Continuous moderators examined – correlation with effect sizecorrelation with effect size

ResultsResults

Characteristics – Study 1Characteristics – Study 1

KK = 20 studies, cumulative N = 12,383 = 20 studies, cumulative N = 12,383

Published (or presented) between 1997 – 2008 Published (or presented) between 1997 – 2008 (median = 2006)(median = 2006)

11 US studies, 9 non-US studies (China, Hong 11 US studies, 9 non-US studies (China, Hong Kong, Kenya, Brazil, Great Britain, Netherlands, Kong, Kenya, Brazil, Great Britain, Netherlands, Peru)Peru)

17/20 heterosexually active samples; remainder 17/20 heterosexually active samples; remainder MSMMSM

Mean age = 20.95 (SD = 7.73)Mean age = 20.95 (SD = 7.73)

Study 1 (cont’d)Study 1 (cont’d) Types of interventionsTypes of interventions

Group targeted (Group targeted (kk = 13) = 13)

Individually tailored (Individually tailored (kk = 3) = 3)

Virtual decision-making (Virtual decision-making (kk = 2), Multiple type ( = 2), Multiple type (kk = 2) = 2)

Delivery modesDelivery modes

Internet (Internet (kk = 11) = 11)

On screen, on site (On screen, on site (kk = 8) = 8)

Print materials (Print materials (kk = 1) = 1)

Characteristics – Study 2Characteristics – Study 2

KK = 12 studies, cumulative N = 4,639 = 12 studies, cumulative N = 4,639

Published (or presented) between 2002 – Published (or presented) between 2002 – 2008 (median = 2006)2008 (median = 2006)

All US studies (except one in Netherlands)All US studies (except one in Netherlands)

10/12 heterosexually active samples; 10/12 heterosexually active samples; remainder MSMremainder MSM

Mean age = 22.52 (SD = 6.76)Mean age = 22.52 (SD = 6.76)

Study 2 (cont’d)Study 2 (cont’d) Types of interventionsTypes of interventions

Individually tailored (Individually tailored (kk = 6) = 6)

Group targeted (Group targeted (kk = 4) = 4)

Virtual decision-making (Virtual decision-making (kk = 2) = 2)

Delivery modesDelivery modes

On screen, on site (On screen, on site (kk = 8) = 8)

Internet (Internet (kk = 3) = 3)

Print materials (Print materials (kk = 1) = 1)

Study 1: OutcomesStudy 1: Outcomes

Knowledge: Knowledge: k = k = 1515

Attitudes: Attitudes: k = k = 1212

Self-efficacy: Self-efficacy: k = k = 1010

Condom communication: Condom communication: k =k = 6 6

Condom intentions: Condom intentions: k = k = 55

Perceived susceptibility: Perceived susceptibility: k = k = 44

Study 1: Efficacy of Study 1: Efficacy of InterventionsInterventions

Knowledge: Knowledge: dd = .276 [.226, .326], = .276 [.226, .326], pp<.001<.001

Self-efficacy: Self-efficacy: dd = .186 [.116, .255], = .186 [.116, .255], pp<.001<.001

Attitudes: Attitudes: dd = .161 [.106, .215], = .161 [.106, .215], pp<.001<.001

Perceived susceptibility: Perceived susceptibility: dd = .131 [.030, .232], = .131 [.030, .232], pp<.01<.01

Condom communication: Condom communication: dd = .119 [.037, .201], = .119 [.037, .201], pp<.01<.01

Condom intentions: Condom intentions: dd = .110 [.007, .2227], p<.05 = .110 [.007, .2227], p<.05

Study name Std diff in means and 95% CI

Bull et al. (2008) S1Bull et al. (2008) S2Davidovich et al. (2006)Downs et al. (2004)Kiene & Barta (2006)Lightfoot et al. (2007)Peipert et al. (2008)Read et al. (2006)Redding et al. (2004)Redding et al. (2002)Roberto et al. (2007)Scholes et al. (2003)

-1.00 -0.50 0.00 0.50 1.00Favors Control Favors Intervention

Condom Use (Study 2)

Study 2: Efficacy of Study 2: Efficacy of InterventionsInterventions

Condom Use: Condom Use: dd = .259 [.201, .317], = .259 [.201, .317], pp<.001<.001

Sexual behavior: Sexual behavior: dd = .427 [.251, .602], = .427 [.251, .602], pp<.001<.001

# of Partners: # of Partners: dd = .422 [.116, .728], = .422 [.116, .728], pp<.01<.01

Incident STDs: Incident STDs: dd = .140 [.035, .245], p<.01 = .140 [.035, .245], p<.01

Publication BiasPublication Bias Orwin’s method: How many null studies needed to Orwin’s method: How many null studies needed to

reduce effect size to one of a trivial magnitude? (e.g., reduce effect size to one of a trivial magnitude? (e.g., dd = .05)= .05)

Study 1Study 1

Knowledge (193 studies)Knowledge (193 studies)

Attitudes (54 studies)Attitudes (54 studies)

Self-Efficacy (83 studies)Self-Efficacy (83 studies)

Study 2Study 2

Condom use (51 studies)Condom use (51 studies)

Comparison to HDIs Comparison to HDIs (Mediators)(Mediators)

Comparison to HDIs – Comparison to HDIs – Condom UseCondom Use

Current Current dd = .259 (Study 2) = .259 (Study 2)

Logan et al. (2002): Logan et al. (2002): d d = .12= .12

Albarracin et al. (2003): Albarracin et al. (2003): dd = .02 = .02

Johnson et al. (2003): d = .07Johnson et al. (2003): d = .07

Albarracin et al. (2005): Albarracin et al. (2005): dd = .18 = .18

Herbst et al. (2005): Herbst et al. (2005): dd = .16 = .16

Crepaz et al. (2006): Crepaz et al. (2006): dd = .16 = .16

Heterogeneity and Heterogeneity and ModeratorsModerators

Study 1Study 1

Statistical testing indicated significant (Statistical testing indicated significant (pp<.001) <.001) heterogeneity among the effect sizes for heterogeneity among the effect sizes for knowledge and attitudes.knowledge and attitudes.

Study 2Study 2

Statistical testing indicated significant (Statistical testing indicated significant (pp<.001) <.001) heterogeneity among the effect sizes for heterogeneity among the effect sizes for condom use.condom use.

Study 1 – MediatorsStudy 1 – Mediators Interventions were significantly more efficacious Interventions were significantly more efficacious

when they:when they:

Targeted MSM (attitudes) (Targeted MSM (attitudes) (pp<.05)<.05)

Used individualized tailoring (attitudes) Used individualized tailoring (attitudes) ((pp<.05)<.05)

Were delivered over the Internet (attitudes) Were delivered over the Internet (attitudes) ((pp<.05)<.05)

Used Social Cognitive Theory (knowledge) Used Social Cognitive Theory (knowledge) ((pp<.05) <.05)

Were Were not not knowledge focused (knowledge) knowledge focused (knowledge) ((pp<.05) <.05)

Study 2 – Condom UseStudy 2 – Condom Use

Interventions were significantly more Interventions were significantly more efficacious when they:efficacious when they:

Were targeted to a single gender (Were targeted to a single gender (pp<.01)<.01)

Used individualized tailoring (Used individualized tailoring (pp<.001)<.001)

Used a stages of change model (Used a stages of change model (pp<.001) <.001)

Had 3+ intervention contacts (Had 3+ intervention contacts (pp<.05) <.05)

Study 2 – Condom UseStudy 2 – Condom Use

Discussion/ImplicationsDiscussion/Implications

DiscussionDiscussion CBIs are efficacious in changing mediators of safer sex CBIs are efficacious in changing mediators of safer sex

and sexual risk behaviorsand sexual risk behaviors

Effect sizes are similar to those achieved by more Effect sizes are similar to those achieved by more traditional human-delivered interventionstraditional human-delivered interventions

Greater use of targeting, and in particular use of Greater use of targeting, and in particular use of individualized tailoring, may increase the efficacy of such individualized tailoring, may increase the efficacy of such interventionsinterventions

Given the many advantages of CBIs, greater investment in Given the many advantages of CBIs, greater investment in them is warrantedthem is warranted

As technology continues to develop, opportunities to use As technology continues to develop, opportunities to use computer-based technologies for health promotion will computer-based technologies for health promotion will only grow (e.g., SMS, smart phones, social media)only grow (e.g., SMS, smart phones, social media)

Future Research (RE-Future Research (RE-AIM)AIM)

RReacheach Who can be reached with CBIs? Who cannot? Who can be reached with CBIs? Who cannot? For whom are CBIs most/least appropriate? For whom are CBIs most/least appropriate?

EEfficacyfficacy What types of CBIs are most efficacious with what What types of CBIs are most efficacious with what

populations? populations? What features of CBIs make them more or less What features of CBIs make them more or less

efficacious?efficacious? Second generation CBIs - mobile devices, social media, Second generation CBIs - mobile devices, social media,

etc.etc. Under what circumstances should CBIs be standalone Under what circumstances should CBIs be standalone

programs versus supplements to human or media-based programs versus supplements to human or media-based interventions?interventions?

Future Research (RE-Future Research (RE-AIM)AIM)

AAdoptiondoption What are practitioners, clinicians, and community What are practitioners, clinicians, and community

partners’ views about CBIs?partners’ views about CBIs? What are the barriers and facilitators to adoption of CBIs What are the barriers and facilitators to adoption of CBIs

in various settings? in various settings?

IImplementationmplementation How could CBIs be implemented and overseen in How could CBIs be implemented and overseen in

practice? By whom?practice? By whom? How might technical issues with CBIs best be handled?How might technical issues with CBIs best be handled?

MMaintenanceaintenance What kind of technical and other support (i.e., updates) What kind of technical and other support (i.e., updates)

are necessary to support the maintenance of a CBI?are necessary to support the maintenance of a CBI? What factors predict institutionalization of a CBI? What factors predict institutionalization of a CBI?

Relevant ReferencesRelevant References

Noar, S. M. (2008). Behavioral interventions to reduce HIV-related sexual Noar, S. M. (2008). Behavioral interventions to reduce HIV-related sexual risk behavior: Review and synthesis of meta-analytic evidence. risk behavior: Review and synthesis of meta-analytic evidence. AIDS & AIDS & Behavior, 12Behavior, 12(3), 335-353(3), 335-353..

Noar, S. M., Black, H. G., & Pierce, L. B. (2009). Efficacy of computer Noar, S. M., Black, H. G., & Pierce, L. B. (2009). Efficacy of computer technology-based HIV prevention interventions: A meta-analysis. technology-based HIV prevention interventions: A meta-analysis. AIDS, AIDS, 2323(1), 107-115(1), 107-115..

Noar, S. M., Crosby, R., Benac, C., Snow, G., & Troutman, A. (in press). Noar, S. M., Crosby, R., Benac, C., Snow, G., & Troutman, A. (in press). Applying the attitude-social influence-efficacy model to condom use among Applying the attitude-social influence-efficacy model to condom use among African-American STD clinic patients: Implications for tailored health African-American STD clinic patients: Implications for tailored health communication. communication. AIDS & BehaviorAIDS & Behavior. .

Noar, S. M., Pierce, L. B., & Black, H. G. (in press). Can computer-mediated Noar, S. M., Pierce, L. B., & Black, H. G. (in press). Can computer-mediated interventions change theoretical mediators of safer sex? A meta-analysis.interventions change theoretical mediators of safer sex? A meta-analysis. Human Communication ResearchHuman Communication Research..

Noar, S. M., Webb, E. M., Van Stee, S. K., Redding, C. A., Feist-Price, S., Noar, S. M., Webb, E. M., Van Stee, S. K., Redding, C. A., Feist-Price, S., Crosby, R., & Troutman, A. (2010). Using computer technology for HIV Crosby, R., & Troutman, A. (2010). Using computer technology for HIV prevention among African Americans: Development of a tailored information prevention among African Americans: Development of a tailored information program for safer sex (TIPSS). program for safer sex (TIPSS). Manuscript submitted for publication.Manuscript submitted for publication.

Contact InformationContact Information

Seth M. Noar, Ph.D.Seth M. Noar, Ph.D.Associate ProfessorAssociate ProfessorDepartment of Department of

CommunicationCommunication248 Grehan Building248 Grehan BuildingUniversity of KentuckyUniversity of KentuckyLexington, KY, 40506-Lexington, KY, 40506-

00420042Phone: 859-257-7809Phone: 859-257-7809Fax: 859-257-4103Fax: 859-257-4103Email: noar@uky.eduEmail: noar@uky.eduWeb: www.uky.edu/Web: www.uky.edu/

~snoar2~snoar2

Thank you!Thank you!

Email: noar@uky.edu