Civic User Testing Group (CUTGroup): Presentation at Code for America 2015 Summit

Sonja MarzianoProject Coordinator, Smart Chicago Collaborative

@ssmarziano #CUTGroup

Civic User Testing GroupUX Testing, Digital Skills & Community Engagement

Sonja Marziano • @ssmarziano • #CUTGroup

A civic organization devoted to improving lives in Chicago through technology.


What is the CUTGroup?


A community of over 1,000 residents in Chicago & all of Cook County


…who get paid to test websites & apps to help create better technology.


If it doesn’t work for you…it doesn’t work!

“”


CUTGroup is a new model for UX testingDigital SkillsCommunity Engagement


UX TestingDevelopers who work with us get specific, actionable feedback

Both quantitative and qualitative information

Stray from UX test design principles in a key way: by requiring the developer to participate in the test


Digital SkillsEach test consists of a mix of 10 – 20 people with varying digital skills

Introduce to new websites and technology

Developers learn how to conduct UX tests to improve their sites


Community EngagementDiverse groups of testers from different neighborhoods

Conversations about technology between residents and developers


CUTGroup BookWe wrote about how-to run a CUTGroup in your own city here:

www.cutgroupbook.org


What defines a CUTGroup?


The people:developers next toresidents


It’s not a CUTGroup if you only test civic apps


It is a CUTGroup when you engage with people around technology


People want to help make tech better.


“Loved being able to help contribute to the development of this product.”"It's so nice to blather on and on..."“I loved it. I am ready for the next one.”“I was not expecting this - this was so fun”


How do you engage through UX testing?


InviteAskListenRespond




Invite (everyone)• Recruit from inside and outside

your networks• Build individual 1:1 relationships

with your testers• Money is nice physical gift

card


Everyone is right for CUTGroup regardless of the tech you want to test (tech comes later, not first)


Everyone means everyoneRecruitment is never over!Find new ways to engage with residents and make it easier for them to join.


Physical gift cards cost more than digital gift cards


These gift cards do not require digital skills to use them


Text message allows people who do not have regular access to the Internet to participate


Choosing these new testers puts them in front of computers and new technologybit.ly/cutgrouptext




Ask• Regular & open communication

about testing opportunities• Gather information with

screening questions• Ask after every test about

CUTGroup • Learn why testers do or do not

participate in tests


Screening Questions• Signing up for CUTGroup is a

simple form• We do not capture tons of

demographic info• Work with developer to

determine what questions to ask to get the right testers


Experiences Demographics


Screening questions to gather relevant information about testers for test


Testers can be part of the CUTGroup and choose which questions to answer




Listen• Provide an opportunity for open

and honest feedback and then listen

• A community space is important• Listen to tech and non-tech

experiences• Capture the responses in

different formats


Meet people in the community


Proctoring• Developers next to residents• Training proctors (developers,

project managers, organization staff)

• Using Wufoo form as a script/outline and a place to capture answers/take notes


Talk about the person’s experiences not just about how they use technology


“How do you typically learn about your child’s school?” “Do you search online for…?” “How do you find out about changes in your neighborhood?”“How would you describe the availability of arts programming in your neighborhood?“What is your current relationships with health info on the Internet?”


Keep asking questions• “Why did you just click that?”• “What did you expect to

happen?”• “Why or why not?”• “Care to elaborate?”• “Can you tell me more…?”


How would you improve this [website]?Tell us anything & everything you can.




Respond• Share results, data, and more• Take the feedback and make

changes• Record and document• Respond to testers’

comments about the CUTGroup experience


Report & Data• In-depth analysis of each

question• Quantitative and qualitative

data• Videos, pictures and more!• Github issues when possible• Share the data


The next step…Find new ways to help developers make changes


If it doesn’t work for you…it doesn’t work!

“”


Let’s talk!…and while we do let me thank these foundations for their support:

Design

Civic User Testing Group (CUTGroup): Presentation at Code for America 2015 Summit