Utah State UniversityDigitalCommons@USUInstructional Technology and Learning SciencesFaculty Publications Instructional Technology & Learning Sciences

4-1995

Using the Web as a survey tool: Results from thesecond WWW user surveyJames PitkowGeorgia Institute of Technology - Main Campus

Mimi ReckerUtah State University

Follow this and additional works at: https://digitalcommons.usu.edu/itls_facpub

Part of the Educational Assessment, Evaluation, and Research Commons, and the InstructionalMedia Design Commons

This Article is brought to you for free and open access by the InstructionalTechnology & Learning Sciences at DigitalCommons@USU. It has beenaccepted for inclusion in Instructional Technology and Learning SciencesFaculty Publications by an authorized administrator ofDigitalCommons@USU. For more information, please contactrebecca.nelson@usu.edu.

Recommended CitationPitkow, J. & Recker, M. (1995). Using the Web as a survey tool: Results from the second WWW user survey. Journal of ComputerNetworks and ISDN Systems, 27(6), 809-822.

UsingtheWebasaSurveyTool:ResultsfromtheSecondWWWUserSurveyJamesE.PitkowGraphics,Visualization,&UsabilityCenterGeorgiaInstituteofTechnologyAtlanta,GA30332‐0280MargaretM.ReckerCollegeofComputingGeorgiaInstituteofTechnologyAtlanta,GA30332‐0280Abstract:ThispaperpresentstheinitialresultsfromthesecondWorld‐WideWebUserSurvey,whichwasadvertisedandmadeavailabletotheWebuserpopulationfor38daysduringOctoberandNovember1994.ThesurveyisbuiltonourarchitectureandWebtechnologies,whichtogetherofferanumberoftechnicalandsurveyingadvantages.Inparticular,ourarchitecturesupportstheuseofadaptivequestions,andsupportsmethodsfortrackingusers'responsesacrossdifferentsurveys,allowingmorein‐depthanalysesofsurveyresponses.Thepresentsurveywascomposedofthreequestioncategories:generaldemographicquestions,browsingusage,andquestionsforWebinformationauthors.Inaddition,weaddedanadditional,experimentalcategoryaddressingusers'attitudestowardcommercialuseoftheWebandtheInternet.Injustoveronemonth,wereceivedover18,000totalresponsestothecombinedsurveys.Tothebestofourknowledge,thenumberofrespondentsandrangeofquestionsmakethissurveythemostreliableandcomprehensivecharacterizationofWWWuserstodate.ItwillbeinterestingtoseeifandhowtheusertrendsshowninourresultschangeastheWebgainsinglobalaccessandpopularity.Keywords:newapplications,surveys,demographics,tools.INTRODUCTIONInthefewyearssinceitsinception,theWorld‐WideWeb,orWWWorWeb,(Berners‐Leeetal.,1994)hasgrowndramaticallyinthenumberofusers,servers,anditsgeographicaldistribution(MeritNIC,1994).Thesetechnologiesforthefirsttimeholdthepotentialofusheringinthe"AgeofInformation"topeopleofallages,backgrounds,andeconomicstatus.Wide‐spreadnetworkingcoupledwiththeeaseofpublishingmultimediamaterialswithintheWebwillsupportradicalchangesinareassuchasmedicine,education,business,andentertainment.Theuniversalaccessibilityofinformationtechnologiesmeansthattheuserpopulationwillbeextremelydiverseintermsofskills,experiences,abilities,andbackgrounds.Assuch,acrucialingredienttothesuccessofsuchendeavorsisanunderstandingofitsuserpopulation.Onepowerfulmethodofcharacterizingthe

background,usagepatterns,andpreferencesofusersisviasurveys.Coupledwithothermethods,suchaslogfileanalysis(e.g.,PitkowandRecker,1994a),theseresultsenableappropriatetargetingofservices,andthedevelopmentofintelligentuser‐centeredapplicationsandinterfaces.InJanuaryof1994,weconductedthefirstsurveyofWorld‐WideWebusers(PitkowandRecker,1994b).ThissurveywasadvertisedandmadeavailabletoWebusersforonemonth,andreceivedover4,800responsestoallquestionnaireswithinthesurvey.Althoughquitesuccessful,thissurveysufferedfromanumberoftechnicalanddesignshortcomingsthatwewishedtoaddress.Tothisend,wemodifiedthebasicarchitectureinordertoenhancethecapabilitiesofthesurveys.Inaddition,weexpandedtherangeandfocusofquestions.Thesechangesimprovedtherobustnessofthesystem,thereliabilityofthedata,andthequalityofthehuman‐computerinteraction.Inparticular,wedesignedandimplementedadaptivequestions.Withtheuseofadaptivequestions,answersprovidedtocertainquestionsareusedtodeterminethenextseriesofquestions.Inthisway,respondentsneednotwadethroughaseriesofunrelatedquestions,andinsteadareonlypresentedwithrelevantones.Thus,adaptationservestoreducethenumberandcomplexityofquestionspresentedtoeachuser.Secondly,weimplementedmethodsfortrackingrespondentsinawaythatrespectsrespondentprivacyandguardstheiranonymity.Thisenablescross‐tabulationofresponsesacrosssurveysections,thusfacilitationmorein‐depthanalysesofsurveyresponses.Inaddition,thismethodenablesfuturelongitudinaltrackingoftheWebuserpopulation.Aswiththefirstsurvey,questionswerepresentedinseparatesurveycategories,whichprovidesseveraladvantages.First,byusingcategories,respondentswereabletoquicklyfinisheachsectionoftheoverallsurvey.Wenotethatonelongsurveycontainingallofthequestionsmaydiscouragepotentialrespondents,andaddsconsiderablytothesurvey'scomplexity.Second,manyWebbrowsershavedifficultymanagingdocumentswithalargenumberofembeddedforms.Third,categorizingquestionsallowsuserstodecideaprioriiftheparticularquestioncategoryappliestothem.Thefirstcategoryaskedgeneraldemographicquestionsabouttherespondent.Questionsabouttherespondentsbrowsingpatterns,motivations,andusagecomprisedthesecondcategory.Thethirdcategoryaskedquestionsofrespondentswhowereinformationproviders,aboutthenatureoftheirinformation,andtheiropinionsaboutexistingtools.Inaddition,weaddedanadditional,experimentalcategoryaddressingusers'attitudestowardcommercialuseoftheWebandtheInternet.Thiscategorywasdividedintoashortandlongversionofthequestionnaires,andrespondentscouldchoosewhichsectiontoanswer.WefeltthatthisstratificationwassufficienttohelpuscharacterizeWWWusers,theirreasonsforusingtheWWW,andtheiropinionofWWWtoolsandtechnologies.

ThesecondsurveywasadvertisedandmadeavailabletotheWebuserpopulationfor38daysduringOctoberandNovember1994.Duringthisperiod,wereceivedover18,000totalresponsestothecombinedsurveysforover4,000users.Tothebestofourknowledge,thenumberofrespondentsandrangeofquestionsmadethissurveythemostreliableandcomprehensivecharacterizationofWWWuserstodate.Thispaperdescribesthetechnicaldetailsoftheimplementationandfollowedbyabriefpresentationthesurvey'sresults.OVERVIEWThereareavarietyofmethodsforsurveyinguserpopulationsviatheInternet,thoughtheeffectivenessofWWWtechnologiespresentsmanyadvantages.Wedefinetheterm"effectiveness"fromanoverallmeasureoftimeandrespondentcomplexitywithrespecttoothersurveymethods.Thoughathoroughcomparisonofsurveyingtechniquesisbeyondthescopeofthispaper,wewillbrieflyoverviewseveralmethodsandthetrade‐offsinvolved.Traditionale‐mailbasedsurveysrequiretheusertoperformtextentry,usuallybyplacingX'sinboxesortypingnumbers,thensendingthemessageofftothesurveyors.Thisscenariofunctionsproperlyifthesurveyendsupinthemailboxesofrespondentswhoarewillingtorespond,thatis,iftheyself‐selectthemselves,andexpendthenecessarytimeandeffort.Inothere‐mailbasedsurveys,thequestionsarepostedtonewsgroups,whichthenrequiretheuserstoextractthemessageandproceedasabove.Eitherway,oncetheresponseshavebeensubmitted,thecollationofthedatacanbecomeproblematic,sinceconsistentstructurewithinresponsescanonlybesuggested,notenforced.Forexample,ifthequestionisposed"Howoldareyou?"theanswermayappearonthesamelineasthequestion,twolinesbelow,maycontainfractions,aninteger,orevenafloatingpointnumber.Phone‐basedsurveysimposelessofataskloadontheuser,butincreasecognitiveloadbyrequiringtheusertokeepalltheoptionsinmemory.Also,responsedatausuallyareenteredbyhumans,anerror‐proneprocess.Furthermore,respondentscannotreviewtheirresponses,andaretypicallysubjecttotimeconstraints.UseofWebtechnologieshelpstominimizetheabovecostsby:1)enablingpoint‐and‐clickresponses,2)providingstructuredresponses,3)usinganelectronicmediumfordatatransferandcollation,4)presentingthequestionsvisuallyforre‐inspectionandreview,5)imposingveryloosetimeconstraintsandfinally,6)utilizingadaptivequestionstoreducethenumberandcomplexityofquestionspresentedtousers.Forthepurposesofthispaper,complexityisdefinedasametricofthevisualandcognitivedemandsplacedonauserwhenansweringquestions.TheSecondWWWUserSurveyitselfwascomposedofthreemainquestionnairesandtwoexperimentalquestionnaires.Extendingandrefiningupontheinitialsetofquestionsaskedinthefirstsurvey,thethreemainquestionnaireswere:GeneralDemographicInformation,WWWBrowserUsage,andHTMLAuthoring/Publishing.Additionally,twoexperimentalconsumersurveys,developedbySunilGuptaattheUniversityofMichigan,wereincluded.Theseweredeployedastwoseparate

surveys,PartOneandPartTwo,withthelattercontainingmorein‐depthquestions.Wenotethattheinclusionofsurveysandquestionsdevelopedexternallyisconsistentwithourphilosophyofworkingwithotherinterestedresearchersinthecommunityduringquestiondevelopmentandrefinement.Inordertoconveythesenseofinteractionpresentwhilecompletingthesurveys,aquickwalk‐thoughfollows.Afterenteringauniqueonewordid(seeLongitudinalTrackingsectionbelowfordetails),theuserispresentedwiththesurveyhomepage.Accesstoeachofthesurveysisprovidedviaradiobuttonsanda"PressHeretoProceedtoSurvey"buttonatthebottomofthepage.Oncetheusersselectsasurveyinwhichtoparticipate,theQuestionEngine(seeArchitecturesectionbelow)generatestheinitialsetofquestionsspecifictothedesiredsurvey.Theinitialsetofquestionspresentedisthesameforallusers,i.e.noadaptationoccurredatthisstageofquestionpresentation.Theuserthenanswersthequestionsandsubmitstheresponsesbyclickingonthe"SubmitSurvey"buttonattheendofthepage.TheQuestionEnginethenprocessesthesubmittedresponses,withthreepossibleresultsforeachsubmittedresponse:1.Theresponsetriggersanadaptivequestiontobeaddedtothelistofquestionsaskedtotheuserduringthenextiterationofquestioning2.TheQuestionEnginerealizesthatthequestionhasnotbeenansweredandre‐asksthesequestionduringthenextiteration.3.Theresponseisfine,andnofurtheractionoccurs.1.TheresponsetriggersanadaptivequestiontobeaddedtothelistofquestionsaskedtotheuserduringthenextiterationofquestioningThelistofadaptedandun‐answeredquestionsisreturnedtotheuser.Thiscycleof"question‐answer‐adapt/re‐ask"repeatsuntilallquestionshavebeenanswered.Atthispoint,theuserisreturnedtothesurveyhomepagethatliststhesurveysthathavenotyetbeencompleted.Thisiterativecycleaccomplishesseveralgoals.Foremost,theadaptationofquestionsreducesthenumberandcomplexityofquestionspresentedtoeachuser.Forexample,aninterestingquestiontodevelopersaswellasWebdatabasemanagersis"Whouseswhatbrowsers?"Giventheexistenceofsevenorsomajorplatforms(e.g.,X/Unix,Macintosh,PC,etc.),withnumerousbrowsersreadilyavailableoneach,thespacerequiredtolistallplatformsandbrowserswouldeasilyfilltwoscreens.Clearlythisisundesirableandinefficient.However,bystagingthequestionintwoparts,onethatasksfortheprimaryplatformoftheuser'sbrowserandtheotherthatprovidesalistofknownbrowsersforthatspecificplatform,theamountofspacerequiredtoposethequestionisreducedaswellasthecognitiveoverheadnecessaryfortheusertoanswerthequestioncorrectly.Additionally,thismethodenablestheacquisitionofdetailedresponses,whichfacilitatesamorein‐depthunderstandingoftheuserpopulation.Forexample,withonlytwoquestions,theregionandstateoftheusercanbeobtained.

CLASSIFICATIONOFADAPTIVEQUESTIONSDuringthecourseofquestiondevelopment,weobservedacertainstructurethatexistedwithinquestionadaptation(seeTableOne).Aswithmosttraditionalsurveysandforallon‐linesurveyswehaveseenthatusetheWeb,mostquestionsdonotresultinanyadaptation,orinference.WerefertosuchquestionsasStandardquestions.Thesearethebuildingblocksuponwhichallothertypesofquestionsarebuilt.Inferentialquestions,ontheotherhand,defineaclassofquestionsthatarebaseduponanswerstopreviouslyaskedquestions.FortheInferentialclasswefoundithelpfultobaseourtaxonomyonaSingleQuestionandaMultipleQuestionbasis,withthelatterbeingcomposedofmorethanoneSingleQuestion.Inotherwords,amultiplequestiondefinesaquestionbasedupontheresponsestomorethanonequestion.InferentialQuestionClass:

MultipleClassProperties:

NumberofQuestionsUsed:

SingleClassProperties:

NumberofResponsesUsed:

SingleResponse,MultipleResponse,CompleteResponse

NumberofQuestionsTriggered:

SingleAdaptation,MultipleAdaptationStandardQuestionClass:

Properties:

Question,ValidResponses,InteractionTypeTable1.ClassificationofthetypesofAdaptiveQuestionsSingleQuestionadaptationisbasedonthefollowingproperties:theNumberofResponsesUsedandtheNumberofQuestionsTriggered.TheNumberofResponsescanbedividedintothreecategories.SingleResponseadaptationoccurswhenonlyoneresponsetoaquestionresultsinfurtherquestioning.Anexamplefromthesurveyasked"Areyouthesole/primaryuserofyoumachine?"withfollow‐upquestionsonlyfor`No'responses.MultipleResponseadaptationoccurswhenseveralresponsesresultsinadaptation.Oursurveydidnotincludeanyfromthiscategory.ItnaturallyfollowsthatCompleteResponseadaptationoccurswhenallresponsestoaquestionresultinadditionalquestions.Aquestionthatfallsintothis

categoryfromthesurveywas"Whichbrowserdoyouprimarilyuse?"Allanswerstothisquestionwerefollowedwithlistsofspecificbrowersforeachmajorcomputingplatform.Onceadaptationistriggered,eitherSingleAdaptationorMultipleAdaptationcanoccur.WithSingleAdaptation,theresponsetriggersonlyonefollow‐upquestion.WithMultipleAdaptation,morethanonefollow‐upquestionisasked.Forexample,thequestion:"DoyouoperateaWWWserver?"canbeclassifiedasCompleteResponse,sinceboth`Yes'and`No'answerstriggeredadaptation.A`No'responseresultsinSingleAdaptation,"CanyouadddocumentstoaWWWdatabase?"A`Yes"responseresultsinseveralquestionsrangingfromchoiceofservers,tothespeedofthenetworkconnectionstotheserver.MultipleQuestionadaptationdefinesthesetofquestionsthataretriggeredbytheresponsestomultiplequestions.Notethateachquestionthattriggersadaptationhasthepropertiesdescribedabove:theNumberofResponsesUsed,andtheNumberofQuestionsTriggered.Thoughthissurveydidnotincludequestionsfromthisclass,wearecurrentlyinvestigatingquestionsofthistypeforfuturesurveys.ARCHITECTUREThemainarchitecturalissuefacingthesurveywastheinfusionofstateinformationintothestatelessHTTPprotocol.Stateinformationwasnecessaryforsupportingseveralaspectsofthesurveys.First,theuser'sidneededtobetrackedbetweenquestionnairesinordertoperformbetween‐questionnaireanalysisandlongitudinaltrackingofusers.Second,accesstotheresponsestopreviouslyaskedquestionswererequiredinordertoenforcequestioncompletionwithinindividualquestionnaires.Thiswasalsorequiredtoimplementtheuseofadaptivequestions,sincethesearebasedupontheresponsesfrommultipleanswers.Third,informationregardingwhichsurveystheusershadfinishedwasrequiredinordertokeeptrackoftheremainingsurveys.Notethatallbutthelattercasecontaininformationthatcanbewrittentodiskandreadintomemorybetweeneachcycleofquestioning.However,wechosenottotakethisapproach,exceptforsurveycompletioninformation,inattemptstominimizethenumberofrequeststodisknecessaryontheserverandtoreduceserversideCPUload.Instead,ourapproachwasdesignedtoleverageoffofthehiddenattributesoftheTYPEfieldusedininputformsinHTML.Initially,weoptedtopassthedatafromtheclienttotheserverviatheGETmethod(1).SincetheURLcontainstheinformationpassedtotheserverviaGET,wedesignedthesurveyhomepagetouniquelyidentifyeachuserbymakingitonlyaccessibleviaaCGIfront‐end.Thus,userscouldaddthesurveyhomepagetotheirhotlistsandusethistore‐accessthenextroundofsurveyswithouthavingtowriteormanuallystoretheirid.Asitturnsout,thisdecisionhadseveralinterestingresults.First,wediscoveredthatseveralbrowsershadhard‐codedlimitstothelengthofURLs.Thus,oncethelimitwasreached,these

browsersfailedtoloadtherequestedURLs.Second,itforcedustore‐evaluateouruseofGETandPOST(2).Intheend,wedecidedtokeeptheuseofGETforaccesstothesurveyhomepage,butchangethemethodforthequestionnairestoPOST.Oneoftheoveralldesigngoalswastoimplementthesurveyswithasgenericanarchitectureaspossible.Wewantedtheunderlyingcodethatgeneratesandprocessesthesurveystoonlyrequireminoradjustmentsbetweenquestionnaires.Towardsthisend,wedecidedtomakeeachquestionnaireastand‐aloneexecutablethatutilizedacommonsetoflibraryroutinesandstructure.FigureOneshowsadiagramofthecomponentsofthearchitectureforonequestionnaire.

Figure1.Theabovediagramoverviewsthearchitectureusedfortheimplementationofonesurvey.Integraltothedesignisthequestionnairedatabase.Thedatabaseisessentiallyanassociativearrayofquestions,whichfacilitatesadirectmappingbetweentheadaptablequestionsandtheresponsesvaluesofthequestionstheyarecontingentupon.Additionalkeys/valuepairswereinsertedintothedatabasetoparameterizeiterationcontrolandenforcequestioncompletion.ThequestionsinthedatabasearemarkedupinHTML.TheId/SessionTrackermanagesidnamespacesaswellasaccesstoquestionnaires.Themotivationbehindtrackinguseridswithinthesurveywasto:1)allowforanalysisbetweeneachquestionnaire,(whichthefirstsurveydidnotdo),2)beflexibleenoughtomanageusersmakingsubmissionsfrommultipleIPaddresseswiththesamedomain,3)enablelongitudinalanalysisoftheuserpopulationand4)bequickandefficientfrombothaclientandseversideperspective.GiventhatthehostnameandIPaddressesarepassedintotheshellforkedbytheserver,wechosetomapnamespacestotheclassoftheIPaddress.Thatis,classAIPnumberscorrespondtonamespacesderivedfromthefirstoctet,classBIPnumberstothefirsttwooctets,andclassCIPnumberstothefirstthreeoctets.ThisschemepermitsuserstofilloutsurveysfromdifferentmachineswithintheirorganizationsallocatedIPnumbers(3)andallowsforquickconversionfromIPaddresstothe

directorywheretheuserinformationisstored.Forexample,auserwhoseIPaddressbeginswith130.207(ClassB)mustchooseauniqueidacrossallotherusersfromthesamedomain.Allsubsequentinformationfortheuseristhenstoredinthedirectory/130/207.Afileexistswithineachnamespacethatkeepstrackofidsandthesurveysthathavebeencompletedbytheuser.Everytimearequestismadeforapageinthesurvey,theidpassedtotheIdTrackerischeckedagainsttheidsregisteredinthisfile.Iftheidisnotfound,thesoftwarereissuestheidentrypage.Similarly,uponreentrytothehomepage,thefileisconsultedtodeterminetheremainingsurveystooffertheuser.TheQuestionEngineperformsseveraltasksbyexploitingthetransparentuseofassociativearraysanddatabaseroutinesinPerl.First,itgeneratestheinitialsetofquestionswhicharereturnedtotheuser.Thisisaccomplishedbyconsultingthedatabaseforthetotalnumberofbasequestionsinthesurveyandthenloopingthroughtheassociativepairs,andappendingthequestions(alreadyinHTML)totheoutputstream.Second,theenginedetermineswhetherquestionsposedtotheuserhavebeenanswered.Thistaskrequiresstateinformation,whichwehandledbymanglingtheresponsestoquestionsintospecialhiddenforms.Specifically,thevalueboundtoNAMEinthehiddeninputtagwasprependedwith`WWW_'andwasappendedtotheoutputstream.Thus,thestateofaquestioncouldbeeasilydeterminedbyinspectingthekeyofthekey/valuepairspassedbackfromtheuser.Finally,sincetheinitialsetofquestionsandtheirresponsesdetermineallsubsequentadaptation,thestateofalladaptedquestioncanbedeterminedbyevaluatingsimplebooleanexpressions,whichcleanlymapintotheclassificationofquestionsoutlinedabove.TheserverusedforthesurveyoperatesNCSA'shttpversion1.3andrunsonaSparcStation1000runningSolaris5.3withtwoco‐processors,over7gigabytesofdisk,and175megabytesRAM.TheserverresidesonGeorgiaTech'sexternalFDDIringwithtwoT3connections‐onetoNSFNET,andtheothertoSuraNET.TheserveralsoperformsotherfunctionslikeNNTP,Gopher,FTP,etc.TheSurveyModulesandlibraryroutinesarewritteninPerl4.36.METHODObviouslyasurveywithoutrespondentshasmarginalutility.Yet,thecurrentstateofWWWprovidesverylittlesupportforbroadcastingandraisingawarenessofallWebuserstotimelyorimportantevents.Asaresult,cooperationandendorsementfrombothCERNandNCSAwereobtainedinpublicizingthesurveys.Bothorganizationsplacedlinksinhighlyvisibleplaces‐CERN'sHomepageandNCSA's"What'sNewPage."Announcementsandre‐postingswerealsomadetoseveralWebrelatednewsgroupsandmailinglistsincluding:comp.infosystems.announce,comp.infosystems.www.*,comp.internet.net‐happenings,andwww‐talk.Additionally,severalsitesplacedlinkstothesurvey(Dr.Fun,CUISearchEngines,EiNet'sSearchPage,etc.).Additionally,severaltrademagazinescontainedarticles

aboutthesurvey.Werealizethatthismethodofsamplingisnotrandomandareactivelyseekingothermethodsforwidespreadawarenessofthesurveysinhopesofminimizingjudgementsamplingbias.Furthermore,theverynatureofsurveymethodologyintroducesself‐selectionconfoundsaswell.SURVEYQUESTIONSThesecondsurveywascomposedofthreemainquestionnairesandtwoexperimentalquestionnaires:GeneralDemographicInformation,WWWBrowserUsage,andHTMLAuthoring/Publishing.TheexperimentalquestionnairesaddressedcommercialusageoftheWebandtheInternet.TheGeneralDemographiccategorycontainedgeneralbackgroundquestionsaboutrespondentsandtheiruseoftheWeb(10questions,3adaptive).Forexample,thisquestionnaireposedquestionsabouttheuser'sage,gender,geographicallocation,occupation,andlevelofeducation.Inaddition,weaskedtheusertoidentifythekindofWebbrowseremployed.Userswerealsoaskedtoestimatetheamountoftimespentworkingwithcomputersperweek.Finally,weaskedtheusertoindicatetheirwillingnesstopayforaccessingWebdatabases(seeAppendixAforthefulllistofquestions).Aswithalloftheotherquestionnaires,atext‐entrycommentboxwaslocatedattheendofthesurveyforuserstocontributewhateveradditionalinformationdeemedrelevant.Thesecondcategorycontainedquestionsabouttherespondents'browseruse(20questions,0adaptive).Weaskedusershowoftentheylaunchtheirbrowser,theamountoftimespentbrowsing,andtheirprimarymotivationbehindbrowsing.SinceWWWbrowsersallowaccesstoalmostallInternetresources,wewereinterestedinthedegreetowhichthesebrowsersarereplacingtheclientsoftwaredesignedforeachindividualresource.Hence,weaskedquestionsonbrowserusetoaccessofGopher,FTP,etc.,aswellasquestionsonWebuseforexplorationandaccessingotherresources(e.g.,weather).SinceabenefitoftheWebisasamultimediapublishingenvironment,thethirdcategoryaddressedquestionstouserswhoareWebinformationproviders(11questions,8adaptive).Wewereinterestedindetermininghowdocumentpublishingismanagedandthereforeaskedtheusertoestimatethenumberofdocumentsauthored,thekindsofinformationprovided,andthenatureoftheorganizationserved.Weaskedproviderstoratetheircomputerexpertise,andhowdifficulttheyfoundittobecomeaWebinformationprovider.ProviderswerealsoaskedwhethertheyalsooperateaHTTPserver,andifso,thenetworkconnectivity,andplatform,hardware,andsoftwareused.Increasingly,theInternetandtheWebarebeingconsideredbythecommercialsector.Forthisreason,weaddedacategorythataddressedusers'attitudestowardcommercialuseoftheInternet.Sincetheseissuesarecomplex,wepresentedthesequestionswithintwosurveysections,ashortandlongversionofthequestionnaire.Userschosewhichversiontheywishedtoanswer.Theshortversioncontained

questionsaboutrespondent'suseandplanneduseoftheInternetforproductinformationandpurchasing.Inaddition,wewereinterestedindeterminingusers'attitudestowardthepurchaseofinformationviatheInternet.Thelongversionofthequestionnaireaddressesthesameissues,butinconsiderablemoredepth.(4)RESULTSFigureTwoshowsthedailynumberofvisitstothesurveyhomepageandthenumberofrespondentsofeachsurveycategoryduringthedaysthesurveywasavailable.Ascanbeseen,responsestartedslowly,butbuiltupasawarenessofthesurveyspread.Severalspikesareevidentthatcorrespondtowhenthesurveywasannouncedonhighlyvisiblepages(forexample,"What'sNew"atNCSA).Lowsareevidentduringweekends.

Figure2.Thenumberofsuccessfullycompletedaccessespersurveyonaperdaybasis.NotethedropinactivityduringtheweekofOctober18th(2ndInternationalWWWConference).Overall,therewereatotalof18,503responsestoallsurveyquestionnairescombined.Fromthis,709,or3.8%,duplicatesubmissionswereremoved.Duplicateswereidentifiedusingsoftwaretodetectmultiplesubmissionsofasurveybyauserinthesamenamespace.Twothirdsoftheduplicatesdetectedoccurredontheexperimentalconsumersurveys,whichcausedsometechnicalproblemsoncertainbrowsersduetothenumberofquestionsasked.Inallcases,thelastentrywasusedwithallothersbeingdiscardedfromthedataset.Invalidsubmissions,0.05%,resultedfrombrowsersthatmangledtheresponsedataduringsubmission.Exactly17,804records(462,001datapoints)werecollatedintothefinaldatasets.(5)Oneareaofdifficultythatoccurredinthepreprocessingstagewasrelatedtotheuseoftextentryfieldsonthreequestions.Asmentionedpreviously,unstructuredresponsesareaproblemwiththedatapreprocessingoftraditionalsurveyingmethods.Weexperiencedsimilarproblemsintransformingrespondententriesinto

uniformstructureddata.Thetwoquestionsthatenabledtheusertotypeanumberwillbereplacedinfuturesurveyswithrangesfortheinitialquestionandadaptationonthisresponseinordertodeterminetheexactnumber.Wecan,however,justifythecostsincurredinoneinstance,whereacquiringthenameoftheuser'sprimarybrowser(asenteredbytheuser)willassistindeterminingtherangeofoptionslistedforeachplatformduringsubsequentsurveys.Inthenextsection,wediscussthefindingsfromeachsurvey,followedbyadiscussionoftheseresults.PleaserefertoFigures3‐8foragraphicalrepresentationofsomeoftheresultsandAppendixAforcompleteresults.GENERALDEMOGRAPHICSTherewereover3522validresponses(indicatinganequalnumberofrespondents)inthissurveycategory,accountingfor20%ofalltheresponses.Theresultsindicatethatthemeanageoftherespondentsis31,andthat44%arebetweentheagesof26and30.Theyoungestrespondentwas12,whiletheoldestwas73.Figure3showsahistogramofusers'age.Interestingly,theagedoesnotfollowanormaldistribution,insteadshowingarapidrisearoundtheageof20.WesuspectthisresultsfromtherelativeinaccessibilityofWebtoolsforpeopleyoungerthanuniversityage.Over90%aremale,and87%describetheirraceaswhite.94%donotsufferanydisabilities.Morethan71%oftherespondentscamefromNorthAmerica,23%fromEurope,and3%fromAustralia.Amoredetailedbreakdownshowsthat12%arefromCalifornia,8%fromtheU.K.,and6%fromCanada(Figure7).Intermsofoccupation,27%describethemselvesasworkinginatechnicalfieldand26%asuniversitystudents(thetwolargestcategories)(Figure4).Intermsofhighestlevelofeducationcompleted,over33%haveuniversity‐leveldegrees,while23%havecompletedpost‐graduatework,and18%describethemselvesashaving"some"university‐leveleducation.Over51%saythattheirInternetaccesscomesfromtheeducationalsector,while30%accesstheInternetfromthecommercialsector.Only30%reportsharingtheirprimarymachinewithotherusers.Forthosesharingamachine,thenumbersharedwithvariedwidely,withameanof539,amedianof20,andamaximumof60,000.Twenty‐ninepercentsaytheyuseacomputerover50hoursperweek,andover19%useitbetween41and50hours.ThemostcommonplatfromisX(43%)followedbyPC(29%andMacintosh(19%).Similarly,themostusedbrowserisXmosaic(40%),followedbyWinMosaic(18%),and‐thereleasedmiddleofthesurvey‐Netscape(18%countingallX/PC/Macversions).OfinteresttoenterprisescontemplatingcommercialuseoftheInternet,71%oftherespondentsansweredaswillingtopayfeesforaccesstoWWWrepositories,dependingbothonqualityandcost.Only21%saytheywouldnot.BROWSERUSAGETherewere2921validresponsestoquestionsregardingbrowseruseandactivities.Manyusersaccesstheirbrowsers1to4timesdaily(40%);38%saytheyspend0to

5hoursperweekusingtheirbrowser,while35%claimtospend6to10hoursperweek(seeAppendixA).WesurveyedusersastohowoftentheyusetheirWWWbrowser,insteadofaccessingspecificclientservices,where1="never"and9="always."Theresultsindicatethat,overall,usersshowastrongpreferenceforusingtheirWWWbrowserinsteadofthestandardGopherandWaisclients(mean=6.5),andtofindreferenceandresearchmaterials.Usersdonotfrequentlyusetheirbrowsertoaccessconferenceinformation,governmentdocuments,Newsgroups,andweatherinformation.UsersreportthefollowingreasonsforusingtheWeb:browsing(79%),entertainment(65%),education(59%),workandbusiness(47%),academicresearch(42%),businessresearch(27%),other(10%),andshopping(9%).INFORMATIONPROVIDINGThesurveyforinformationproviderswasansweredby1669people.Asexpected,giventhequestioncategory,over97%oftherespondentshaveauthoredHTMLdocuments,withpeople,onaverage,authoring31.5documents.Sixtypercentoftheauthorshaveauthoreddocumentsforotherpeople.Ofthese,36%reportauthoringdocumentfor1to10people,24%for11to50people,and21%forover100people.Amongauthors,61%operateaHTTPserver,58%runNCSA'sserver,19%runCERNorGN(abuginourloggingsoftwareresultedinanswersforeachtogettalliedtogether),10%runMacHTTP.Asmirroredbyothermeasurementsofportactivity,themajorityoftheservers(88%)listenonport80.Ofthosewhodonotoperateaserver,81%canstilladddocumentsdirectlytotheserverarea.Amajorityofserveradministrators(52%)reportnetworkconnectivityof1megabyte/sec.Intermsofpagetopic,81%reportauthoringdocumentsonwork,77%onbiographicalinformation,44%onresearch,35%onentertainment,26%onother,25%onmeta‐indexes,20%onnews,18%onproductinformation,14%onads,13%onart,11%onconferences,and6%onsports.Mostproviders(91%)knowhowtoprogram,and60%haveoversevenyearsofprogrammingexperience.Over60%reportlearningHTMLin1to3hours,with89%sayingitwas"easytolearn"andmostsayingtheHTMLdocumentationwaseasytounderstand.Fifty‐sevenpercenthavelearnedFORMS,andmost(84%)founditeasytolearn.Forty‐fivepercenthavelearnedISMAP,andmost(82%)founditeasytolearn.Intermsofuseofmedia,91%reporttheuseofimages,34%sounds,and24%movies.Intermsoflinkstootherdocuments,99%reportusinglinks,91%uselinkstootherWebservers,65%useFTPlinks,and50%useGopherlinks.Finally,47%reporttheuseofFORMS,42%reportauthoringinteractivedocuments,and25%reporttheuseofISMAP.CONSUMERSURVEYS

Whileamorein‐depthanalysisoftheresultsfromtheconsumercommercializationsectionsisforthcoming,wepresentsomeinterestingpreliminaryresults.Mostpeople"DisagreeSomewhat"withtheassertionthatitissafetousecreditcardswhenmakingpurchasesfromWebvendors.Similarly,nearly82%oftheusersviewthesecurityofsensitiveinformationas"VeryImportant,"withanadditional15%regardingthisissueas"SomewhatImportant."Interestingly,thequalityofinformationaboutpurchasingchoicesandthereliabilityofInternetvendorsrankevenhigherthantheabovesecurityissues‐89%and84%respectively.Intermsofmaritalstatus,53%oftheusersrespondedassingle,and42%respondedasmarried.Mostuserscurrentlyusethefollowingproducts(listedindecreasingoforderofuse):compactdisc(CD)players,VRC/videoplayers,modems,andCD‐ROMS.SlightlymorethanoneoutoftenusersgainaccesstotheInternetviaworkorschool,with28%payingforInternetaccesspersonally(notethesetwoarenotmutuallyexclusive).Over42%oftheusersreporttheirincomeasbetween$35,000and$75,000,though15%choosenottoreporttheirincome.Thirteenpercentreporttheirincomeasbelow$15,000.

Figures3through8.Contrarytopopularbelief,thedistributionofagesofWWWWusersdoesnotfitanormaldistribution.Technicalprofessionalsanduniversitystudentstogethercomprisethemajorityoftheuserpopulation,withmostusersutilizingtheirWWWbrowseronetofourtimesaday.ThemostwidelyusedplatformisX/UNIX,followedbyPCsandMacintoshes.Thegraphoflocationrepresentsthetopfourlocationsofuses,withCaliforniaaccountingfornearlyoneseventheofallrespondents.Finally,mostusersareeitheraffiliatedwitheducationalinstitutionsorcommercialorganizations.

CONCLUSIONInthispaper,wereportedresultsfromasurveyofWorld‐WideWebusers.ThesurveyisbaseduponasetofWebtoolsthatallowstheuseofadaptivequestions,andenablesthetrackingofusersforlongitudinalanalysis.Asdemonstratedbythehighnumberofsurveyrespondents,theWebprovidesaneasy‐to‐use,reliable,andlowoverheadsurveymedium.Theresultsfromoursurveyprovide,tothebestofknowledge,themostcompletecharacterizationofWebuserstodate.Theysuggestthatthetypicaluserisa30‐yearoldeducatedmalefromNorthAmericawhoworkswithcomputers.ItwillbeinterestingtoseeifandhowthesetrendschangeastheWebgainsinpopularity.Inthefuture,weplantodeployoursurveyeverysixmonths.WebelievethatthiswillbeausefulmeansfortrackingthegrowthandchangesinWebusesandpopulation.GiventhedynamicnatureofWWWuseandtechnologies,webelievethatsurveysruntwiceayearoughttoprovideanoptimaltrade‐offbetweenmaintainingrespondentsfromsurveytosurveyandchartingtheWeb'sgrowthandchanges.Inaddition,wehopethattheWWWcommunitywillallowustoremainthesoleWebsurveyorsinthisdomain.WefearthatifotherresearcherscluttertheWebwithsimilarsurveys,theoverallutilityofsuchsurveyswillbegreatlydiminished.Inlightofsucharequesttothecommunity,wegladlyopenourselftosuggestionsandspecificresearchagendasofotherresearchers.REFERENCES(Berners‐Leeetal.,1994)Berners‐Lee,T.,Cailliau,R.,Luotonen,A.,Nielsen,H.,andSecret,A.(1994).TheWorld‐WideWeb.CommunicationsoftheACM,37(8):76‐‐82.(MeritNIC,1994)MeritNIC.(1994).NSFNETStatistics.Availablevia:URL.(PitkowandRecker,1994a)Pitkow,J.andRecker,M.(1994a).Bottom‐upandtop‐downanalysisforintelligenthypertext.InProceedingsoftheThirdInternationalConferenceonInformationandKnowledgeManagement.Maryland:NIST.(PitkowandRecker,1994b)Pitkow,J.andRecker,M.(1994b).ResultsfromthefirstWorld‐WideWebsurvey.SpecialissueofJournalofComputerNetworksandISDNsystems,Vol.27,no.2.ACKNOWLEDGEMENTSThankstoallmembersoftheGraphics,Visualization,&UsabilityCenteranditsdirector,Dr.JimFoley,fortheirsupportandhelp.SpecialthanksextendtoMichaelMeallingandtheentireOfficeofInformationandTechnologyfortheirtechnicalsupportduringthesurveysandtheirgenerousdonationofequipment.AdditionalthanksgotoLaurieHodgesandColleenKehoeforherhelpwithdatacollationandanalysis.Finally,JameswouldliketothankDr.JorgeVanegasforimplicitfundingthroughoutthesurveys.

AUTHORINFORMATIONJAMESPITKOWreceivedhisB.A.inComputerScienceApplicationsinPsychologyfromtheUniversityofColoradoBoulderin1993.HeisaGraphics,Visualization,&UsabilitygraduatestudentintheCollegeofComputingatGeorgiaInstituteofTechnology.Hisresearchinterestsincludeusermodelling,adaptiveinterfaces,andusability.MIMIRECKERreceivedherPh.D.fromtheUniversityofCalifornia,Berkeley,in1992.SheiscurrentlyaResearchScientistintheCollegeofComputingattheGeorgiaInstituteofTechnology.Herresearchinterestsincludecognitivescienceapproachestolearning,metacognition,instruction,interactivelearningenvironments,human‐computerinteraction,cognitivemodellingandmultimedia.

Footnotes(1)Essentially,theGETmethodappendsthedatabeingpassedfromtheclienttotheserverontotheURL,whereasthePOSTmethodpassesthedatatotheserverwithoutalteringtheURLoftherequesteddocument/program.(2)Intandemwithdialogsonwww‐talk,weconcluded1)useGETforlogicalindependenttasks/retrievaland2)currentlyPOSTsshouldnotbehotlisted/cached,

thoughthisisdoableviarestructuringtherepresentationofhotlistobjectsbytheclient.(3)Thisschemedoesnothandlecertainorganizationswhoownmultipleipswithinthesameclass,likeGeorgiaTech.(4)Aversionoftheallquestionnaireswithadaptationareavailablevia:http://www.cc.gatech.edu/gvu/user_surveys/survey‐09‐1994/.(5)Alldatasetsarepubliclyavailableviaftp://ftp.cc.gatech.edu/gvu/www/survey/survey‐09‐1994/datasetsandtheaboveURL(footnote4).