Ten evaluation questions guided the design and conduct of the evaluation:1. Which of the YSI’s core elements were implemented?2. Which of the YSI’s core elements were implemented as intended? 3. Were activities related to the partnership element implemented as intended? 4. How many (and what type of) SNI partners were engaged and involved in efforts to recruit YBCS to

participate in the YSI by accessing the YSI site? 5. Were activities related to the Web presence element implemented as intended? 6. What were the barriers and facilitators to YSI implementation? 7. How many people visited the YSI to obtain psychosocial and reproductive health support information? 8. What proportion of people who accessed the YSI match the demographic profile of the intended

target population? 9. How many African American YBCS visited the YSI to obtain psychosocial and reproductive health

support information? 10. Did African American BCS and YBCS value and use the information available on the YSI site?

The use of a mixed-method evaluation design helped evaluators identify the strengths and best practices of the YSI program. Findings reveal that there is great value in developing and implementing culturally appropriate interventions providing African American YBCS with psychosocial and reproductive health support. Partnerships with organizations that can reach vulnerable audiences and use of proven Web-based intervention strategies to implement culturally appropriate interventions may effectively contribute to the elimination of cancer health disparities.

Centers for Disease Control and Prevention. (1999). Frame work for program evaluation in public health. Morbidity and Mortality Weekly Report, 48(RR11), 1–40.

Background and PurposeAfrican American women under age 45 experience the highest breast cancer incidence of any ethnic/racial group and, across all ages, have a disproportionately higher breast cancer morbidity rate.

In March 2010, as part of the Patient Protection and Affordable Care Act, Congress passed the Education and Awareness Requires Learning Young (EARLY) Act. This directed the Centers for Disease Control and Prevention (CDC) to develop and implement national campaigns to educate young women (particularly African American and Ashkenazi Jewish women, who are at increased risk) and health care providers about breast cancer risk and early diagnosis. In September 2010, CDC launched the Breast Cancer in Young Women (BCYW) project, a 3-year study to identify, strengthen, promote, and evaluate real-world, evidence-based interventions that support YBCS.

Through the BCYW project, CDC supported Sisters Network® Inc. (SNI) for the development and evaluation of the Young Sisters Initiative: A Guide to a Better You! program (the YSI). The YSI is a Web-based program that provides psychosocial and reproductive health support to African American breast cancer survivors (BCS).

Collaborative Development of the YSIThe YSI was developed and evaluated through a collaborative partnership between CDC, SNI, and subject matter experts in the areas of breast cancer and health communications. The YSI was modeled after the Sisters Peer Counseling in Reproductive Issues After Treatment (SPIRIT) program, an evidence-based behavioral intervention designed for African American BCS of all ages. CDC, SNI, and subject matter experts conducted formative evaluation to inform the adaptation of SPIRIT for African American YBCS. The resulting YSI consists of culturally appropriate materials and messages intended to address the specific needs of African American YBCS. The YSI is broadly accessible and easy to use, and its images, design, and tone are tailored to appeal to this specific audience.

AuthorsTemeika Fairley, PhD • Dara Schlueter, MPH • Ashani Johnson-Turbes, PhD • Angela R. Moore, MPH • Natasha Buchanan, PhD • Mary Ann K. Hall, MPH

• Marnie House, PhD • Linda Baffo, MPH • Kelly Hodges, MS • Leslie Schover, PhD

AcknowledgementsThe project was funded by CDC contract number 200-2008-27957. Findings and conclusions are those of the authors and do not necessarily represent

the official position of CDC.

During the evaluation, YSI visitors commonly reported SNI as their primary source for learning about the YSI. Among BCS, the next most commonly reported referral source is the Internet (likely via the use of search engines). Over the course of the evaluation period, there were 1,442 visitors to the YSI; 524 of the visitors returned one or more times to the site, resulting in a total of 1,966 unique page views. Of the 1,442 visitors, 476 (33%) completed the demographic screener to access the YSI (Exhibit 2).

Visitors who accessed the site did so with a combination of devices. Exhibit 4 depicts the proportions of visitors (1,319) who used a desktop, along with the 647 others who used mobile devices (404) and tablets (243).

Visitors commonly accessed the YSI by typing the URL directly into their browsers (Exhibit 3). The second most common way to access the YSI was via Facebook, followed by referral from SNI’s program Web site, and then by e-mail links.

Key Results From Qualitative Data

Key Results From Quantitative Data

Key Results From Quantitative Data (cont.)

Evaluation Questions

Conclusion

References

Evaluation DesignFollowing the steps outlined by the CDC Framework for Evaluation in Public Health (CDC, 1999) and principles of culturally responsive research design, a mixed-method process evaluation design was developed to address the 10 evaluation questions. The evaluation design included mixed-method data collection before, during, and after intervention implementation to assess reach, Web site usage, and user satisfaction.

The qualitative and quantitative data sources used for this evaluation are in Exhibit 1

• As reported across all interviews, both of the YSI core elements—partnerships and Web presence—were implemented during YSI implementation from March to August 2013. The YSI core elements were implemented as intended.

Collaborative partnerships were essential to implementing and sustaining the YSI, as partners were involved to support YSI development (e.g., logic model review, review and finalization of the YSI site structure and Guide) and implementation (e.g., promotion, Web-based and “on the ground” recruitment).

• Key strategies for marketing the YSI included conducting personal outreach to the target audience to promote the YSI’s features and encourage use of the intervention; leveraging existing resources to market the YSI, including local and national partnerships and digital networks; disseminating consistent, culturally appropriate messaging in YSI Web-based materials; and using social media and other Web-based strategies (e.g., e-mail) for YSI promotion.

• Facilitators to implementation included CDC support for the YSI; existing partnerships; consistent YSI messaging, look, and feel; use of culturally appropriate materials, messages, and marketing strategies; and the YSI’s involvement in existing SNI activities (e.g., the SNI’s annual breast cancer walk).

• Barriers to implementation included the delay in launching the YSI; limited time for implementation; limited resources for implementation; inability to offer incentives to YSI visitors; challenges reaching African American YBCS; and the large amount of information in the YSI Guide.

* Interviews were conducted before the YSI launch and postimplementation.

The proportion of individuals who registered for and accessed the YSI Web site closely matched that of the intended target audience, particularly among BCS, African American YBCS, and African American YBCS younger than age 45 (Exhibit 5). When non-BCS participants were removed, the proportion of visitors who identified as Black or African American rose from 58.6% to 93.0%.

Most respondents—81% of overall visitors, BCS, and YBCS, and 88% of YBCS under age 45—were either somewhat or very likely to recommend the YSI to others.

YSI visitors were very pleased with the YSI’s provision of psychosocial and reproductive health information. Overall, respondents were satisfied with the YSI. Most in the four groups represented in Exhibit 6 reported that they were very satisfied. Seventy percent of all respondents and BCS, 74% of YBCS, and 81% of YBCS under age 45 were either somewhat or very satisfied with the YSI.

Tablets, 243Other, 647 Mobile devices, 404Desktop, 1,319

Collaborative Development, Promotion, and Evaluation of a Web-Based Intervention for African American Young Breast Cancer Survivors

Count of Visits Visitors Page Views1 1,442 6,1662 242 1,0683 72 2554 34 1225 23 766 16 487 11 248 11 20

9–14 33 12115–25 24 18326–50 45 21451–100 13 30

Exhibit 2. YSI Visitors and Multiple Site Visits

Count of Visits Page Views Direct entry 1,034 Facebook 338 SNI Web site 226 E-mail links 200 Search engine 72 SNI social network 35 Twitter 24 News Web sites 21 Nonprofit organizations 11

Exhibit 3. Web Traffic Sources

Exhibit 4. Devices Used to Access the YSI Site

Level of Satisfaction With the YSI Program

Overall Visitors(n = 180)

BCS(n = 161)

African American YBCS(n = 77)

African American YBCS < 45(n = 37)*

Very unsatisfied 9 (6.67%) 8 (6.67%) 5 (7.25%) 2 (6.25%)

Somewhat unsatisfied 4 (2.96%) 3 (2.5%) 2 (2.9%) 0 (0%)

Neutral 27 (20%) 25 (20.83%) 10 (14.49%) 4 (12.5%)

Somewhat satisfied 34 (25.19%) 31 (25.83%) 19 (27.54%) 8 (25%)

Very satisfied 61 (45.19%) 53 (44.17%) 33 (47.83%) 18 (56.25%)

Missing 45 41 8 5

Data Source Data CollectionPrimary or

Secondary Data Source

Evaluation Question Addressed

Qualitative Data Sources

SNI program documentsICF reviewed SNI program documents to gather information about the organization and provide context for program imple-mentation.

Secondary 1, 2, 3, 4, 5, 6

In-depth interviews of SNI staff members

ICF conducted interviews with SNI staff to understand their roles and experience in YSI development and implementation; engagement of partners; barriers and facilitators to implemen-tation; and best practices for implementation.

Primary 1, 2, 3, 4, 5, 6, 10

In-depth interviews of SNI partners

ICF conducted interviews with SNI partners to learn about their roles and experiences with the YSI, especially their roles in YSI recruitment and perceptions of the program’s value.

Primary 1, 2, 3, 4, 5, 6, 10

Quantitative Data Sources

Google Analytics

Google Analytics was used to investigate how visitors used the YSI site. Google Analytics data (i.e., detailed statistics about the YSI Web site’s traffic and traffic sources) were collected from launch and throughout the evaluation period.

Primary 7, 9

Demographic screener

The screener was administered through the YSI Web site to collect demographic data about YSI users. The 12-question screener gathered dataon how users heard about the YSI, along with information regarding age, race, survivorship status, family structure, treatment history, and willingness to participate in the post use survey.

Primary7, 8, 9

Postuse survey

The post use survey gathered data from YSI users, self-identi-fied as BCS, to understand their perceptions of the YSI, particu-larly their attitudes toward the YSI Guide, and whether the pro-gram met their needs for reproductive and psychosocial health information

Primary 8, 9, 10

Exhibit 6. Satisfaction With YSI Across Visitor Types

Exhibit 1. Mixed-Method Evaluation Data Sources*

Black or African

American

White AmericanIndian or

Alaska Native

Asian Native Hawaiian or Other

Pacific Islander

Overall Users BCS

250

150

100

50

0

300279 279

167

20 164 3 2 1 0

200

Exhibit 5. Racial/Ethnic Background of YSI Visitors