Engaging open source community in Quality Assurance

EngagingOpenSourceCommunityinQualityAssurance

PreethiThomas,RedHat@preethit

EngagetheCommunity-AchieveQuality

Flickr:opensourceway

IsQualityFree?

Flickr:victorianevland

“Qualityisfree.It'snotagiG,butit'sfree.The'unquality'thingsarewhatcostmoney.”

PhilB.Crosby

Flickr:59654451@N02Flickr:jp3g

DevelopmentinClosedSourceandOpenSource

Closed Source Open Source

Scope Defined Evolving scope

Well-defined development methodology Development methodology not defined or documented

Extensive project documentation Little project documentation

Business analysts define requirements

Programmers define requirements

Measureable goals used throughout project

Few measurable goals

Formal project planning and scheduling Little project planning and scheduling

Assigned resources Assigned and volunteer contributors

Formal and structured quality assurance and testing methodology

Quality assurance and testing methodology depends on sponsorship, expertise, resources

Black-box testing as early as possible User base performs black-box testing late in the process

WhataretheChallengesofOpenSourceQE?

release early release often

SustainableCommunityforOpenSourceQE

• Developcoderapidly• DebugcodeeffecWvely• Buildnewfeatures

TheOnionModel

doi.ieeecomputersociety.org:s10581

OpenSourceQualityEngineering

•  CodetoConsumermindset•  Codemodularityandpeerreview•  ConWnuousintegraWonandautomaWon•  CommunityengagementinsystemtesWng

ExamplesofOpenSourceProjectsEngagingCommunityinQualityAssurance

Pulp

h"p://www.pulpproject.org

Flickr.:caitlinator

WhatisPulp?

•  Managerepositoriesofcontent•  Supportsmanycontenttypes

—  RPM,Docker,Puppet

•  Pull-throughcache•  WebapplicaWonwri]eninPython•  ReleasedunderGPLv2•  h]ps://github.com/pulp

WhoUsesPulp

•  RedHatReleaseEngineering•  Katello/RedHatSatellite6•  RedHatEnterpriseLinuxonAmazonEC2•  Systemadministratorsdoingcontentmanagement

WhatisTheRoleofPulpQE

•  BugreporWng

— UsesRedminebugtrackerhostedbyPlanio—  h]ps://pulp.plan.io/

•  Bugtriage— DiscussionsinIRC#pulp-devchannelonFreenode

•  BugverificaWon•  WriWngtestscases•  AutomaWngtestcases

PulpSmash

•  TestSuiteforPulp•  GPL-licensedPythonlibrary•  ConWnuousintegraWonwithlatestPulpcommits•  Anyonecancontributetotestsuiteandexecutetests•  h]ps://github.com/PulpQE/pulp-smash•  h]p://pulp-smash.readthedocs.io/en/latest/index.html

Pixabay:hammer-holding-labor-worker

HowtoContributetoPulp&PulpSmash

•  WritePulpSmashtestcases•  ExecutePulpSmashtests•  Reportbugs•  Automatetestcases•  UpdatedocumentaWon•  h]ps://github.com/pulp/pulpproject.org

FedoraQA

CommunityTestDays

•  WhatareTestDays?

—  TestspecificcomponentinanupcomingFedorarelease—  Testcompletedorindevelopmentchange

•  WhenareTestDays?—  Chosenbyorganizers—  BetweenAlphaandGeneralAvailability

•  WhereareTestDaysheld?—  DiscussionsinIRC#fedora-test-daychannelonFreenode

HosMngYourOwnTestDay

•  Decidethefeatureyouwanttotest—  Choosethetargetorchangefeature—  Talkwithleaddevelopersand/orpackagers

•  Setthedate—  Involvedpeopleareavailable—  Featureclosetocomplete—  NoconflicWngevents

•  CreateaWcketorganizingyourFedoraTestDay—  TicketscanbecreatedontheFedoraQATrac—  Followthenamingformat

HosMngYourOwnTestDay(ConMnued)

•  CreateWikipage—  Makesureit’scomplete—  Usethetemplate—  Createtestcases—  SetuptheTestdayApp

–  Simpleweb-applicaWon–  Easiersubmissionsoftestdays'results

•  Getpeopleonboard—  MakesureallinvolvedparWesareaware—  Inviteexperiencedusers—  Promoteonblogsandsocialmedia

•  Buildliveimage— Livebootrecommended

AnExampleofaTestDayWikiPage

fedoraproject.org/wiki/Test_Day:2016-05-16_WorkstaWon_Graphical_Upgrade?rd=Test_Day:Current

AnExampleofaTestDayWikiPage

fedoraproject.org/wiki/Test_Day:2016-05-16_WorkstaWon_Graphical_Upgrade?rd=Test_Day:Current

AnExampleofaTestDayApplicaMonPage

testdays.fedorainfracloud.org/events/8

Foreman

Flickr:dollymixtures

CommunityBugDay

•  Triageopenbugs— VerifyifreportedissuessWllexist— Closestaleorduplicatebugs— Submitpullrequestswithfixesforeasybugs

— PrioriWzefixingremainingbugs

— IdenWfynewfeaturesneeded

theforeman.org

QualityMaSers,CommunityMaSers

Flickr:opensourceway

Flickr:vampiress144Flickr:marquisderad

Anyone Can Contribute to Open Source!

•  Givingbacktothesociety•  ParWcipaWnginanywayandanyWmeyoucan•  CollaboraWngandinnovaWngwithpeoplearoundtheworld•  Freedomandfun

DocumentaMon

•  h]p://pulpproject.org/docs/•  h]p://pulp-smash.readthedocs.io/en/latest/about.html#contribuWng•  h]ps://fedoraproject.org/wiki/QA/Test_Days•  h]ps://fedoraproject.org/wiki/QA/SOP_Test_Day_management•  h]ps://groups.google.com/forum/#!searchin/foreman-users/bug$20day%7Csort:relevance/

foreman-users/WVLNY3Cq7VA/BxGqwTM_BAAJ•  h]p://www.idi.ntnu.no/grupper/su/bibliography/pdf/OpenSource/Aberdour2007sw.pdf

QuesMons?

PreethiThomas

pthomas@redhat.com@preethit