Green Ribbon Schools Usability Report

8/1/2019 Green Ribbon Schools Usability Report

1/28

GreenRibbonSchoolsUsabilityReport Date:MAY7,2012

1

USABILITYEVALUATIONPLANANDREPORT

GREENRIBBONSCHOOLS(GRS)WEBSITE

Preparedfor: SteveAmos,GreenRibbonSchools

WillLindow,GreenRibbonSchools

RandolphBias,UniversityofTexasatAustin

Author: RobScordino,UniversityofTexasatAustin

Date: May7,2012


2/28


2

CONTENTS

USABILITYEVALUATIONPLANANDDESIGN

Introduction 3

PurposeofUsabilityEvaluation 4

Participants 5

DesignofUsabilityEvaluation 6

DataCollectionMethods 9

References 10

USABILITYREPORT

Findings 11

Suggestions 18

Conclusion 21

APPENDICES

A.InformationFormforParticipant 23

B.Pre-evaluationSurvey 24

C.ExploratoryTaskProtocol 25

D.SpecificTaskProtocol 26

E.Open-endedTaskProtocol 27

F.Post-evaluationSurvey 28


3/28


3

INTRODUCTION

ThisdocumentdescribesausabilityevaluationfortheGreenRibbonSchools(GRS)

website.Thepurposeofausabilityevaluationistopredicttheexpectedperformanceofthe

actualcustomerusingthecurrentproductandmaterials,aswellasdetectanydesign

issuestobeaddressedinafutureredesignoftheproduct.Thisevaluationdocument

containsboththeevaluationplanandreport,containingthefollowingsections:

Evaluationplan

Purposeoftheusabilityevaluation ParticipantData Designoftheusabilityevaluation Datacollectionmethodology

Evaluationreport

Researchfindings Suggestions Conclusion

ETHICALAPPROVAL

Thisstudyuseshumanparticipantsandthereforewassubjecttoethicalevaluation.A

proposalofthisresearchwassubmittedtotheOfficeofResearchSupportattheUniversity

ofTexasatAustintobeevaluatedbyInstitutionalReviewBoardbeforedatawascollected.

Thisresearchwasexemptfromfullboardreview,asparticipantsinthisstudyfacedriskno

greaterthanoutsideofaresearchenvironment.Nevertheless,carewastakentoprotect

participantprivacyandtheconfidentialityoftheirremarks.

THEGRSWEBSITE

GreenRibbonSchools(GRS)isanawardprogramthatrecognizesschools

participatinginactivitiesthatpromoteandencourageahealthyandenvironmentally

friendlylearningenvironment.TheseactivitiesmustfallundertheFourCornerstonesofa

GreenRibbonSchool(Eco-Campus,Health&Fitness,NatureAdventure,Natural

Classrooms).Byencouragingtheseactivities,GRShopestopromotehealthier,happierand

smartercampusesforstudents,teachersandthecommunities.


4/28


4

TheGreenRibbonSchoolsWebsiteistheembodimentoftheGRSprogram.Thesite

servesthreemajorfunctions:projectplanningandmanagement,socialnetworkingand

collaboration,andpublication.Itisthetoolwithwhichmembersplanprojectsandtrack

progress.Thewebsiteisalsohowteachersandstudentscommunicateandcollaborate

withotherteachersandclasses.Thesuccessofthesiteandprogramrelyonthis

connectivity.Inorderfortheprogramtogrow,theGRSwebsitemustbeusableforits

members.

DESIGNUSABILITYGOALS

TheGRSWebsitewasdesignedwithgeneralusabilityandaccessibilityissuesin

mind.Thistoolismeanttobeusedbyteachersandstudentsinprimaryandsecondary

schools.Therefore,thesiteisintendedtobeusedintheshortamountsoftimeavailable

duringclassorbeforeandaftertheschoolday.

Theevaluationwillfocusondeterminingiftheneedsoftheuseraremetinaneasy

tounderstand,useful,andproductivemanner.Specificmeasurablegoalsfortheusability

evaluationareoutlinedintheUsabilityEvaluationGoalssectionofthisdocument.

PURPOSEOFTHEUSABILITYEVALUATION

Formalend-userevaluationoftheGRSWebsitehasnotyetbeencompleted.The

aimofthisusabilityevaluationistoinvestigateusersexperienceswiththeinterfaceand

contentoftheGRSWebsite.

CONCERNS

Stakeholdersmainconcernswiththecurrentwebsiteliewithinthecurrentuser

interfacedesign.Anefficientuserinterfaceisessentialforthesuccessfuluseofthissite.A

numberofemailconversationsandaface-to-facemeetingtookplaceinordertodetermine

specificconcernsofthestakeholders.Asaresult,fourguidingquestionsweredetermined

tobethefocusoftheusabilityevaluation.

Specificquestionstobeaddressedinthisusabilityevaluationinclude:

1. WhatareusersinitialreactionstotheGRSWebsite?2. CanusersnavigatetheGRSWebsite?

Aretheiconsandtheirmeaningsobvious?


5/28


5

Canusersfindspecificitemsthroughthecurrentuserinterface? Areusersabletofindhelpresourcesusingthecurrentinterface?

3. Areusersabletousespecifictoolsonthewebsite?4. DoestheGRSWebsitemeettheneedsoftheuser?

USABILITYEVALUATIONGOALS

Specificusabilitygoalsweredeterminedfromtheaboveconcerns.Itisuponthefollowing

goalthattheevaluationwasbased:

Participantswillbeabletocompleteactivitiesorlocatespecificinformation. Givenatask,userswillbeallowedtofreelynavigatethesitetocompletethistask. Participantswillbeabletomovefromitemtoitemwithlittledifficulty Participantswillbeabletofindrelatedinformationwithlittledifficulty

Asurveywasalsocreatedtodetermine:

Usersopinionsofnavigabilityofthesite. Usersopinionsoforganizationofinformation. Usersopinionsofease/difficultyofuse. Usersopinionsofavailabilityandtypeofcontent. Usersuggestionsforimprovement.

PARTICIPANTS

Theselectionofparticipantswhosebackgroundandabilitiesarerepresentativeofa

productsintendedenduserisacrucialelementoftheevaluationprocess.Validresults

willbeobtainedonlyiftheparticipantsselectedaretypicalendusersoftheproduct,orare

matchedascloselytothecriteriaaspossible.Forthisreason,participantsforthisstudy

werechosenthroughpurposefulsampling(Patton,2002).AccordingtoMerriam(2009),

purposefulsamplingisbasedontheassumptionthattheresearcherwantstodiscover,

understandandgaininsight,andthereforemustselectasamplefromwhichmostcanbelearned(p.77).

BecausetheGRSProgramisintendedforuseinschools,K-12educatorsarethe

idealparticipantsforthisstudy.ParticipantswererecruitedAtotalofsevensubjects


6/28


6

participatedintheusabilitystudy.Allworkedinsecondaryschools-sixweresecondary

teachersandonewasaninstructionalspecialist.Dataoncharacteristicsofindividual

participantswascollectedthroughasurveyusingGoogleDocuments.Participantswere

askedtoreporttheiroccupationandexperiencewithGreenRibbonSchools.Participants

werealsoaskedtoratetheirabilitieswithbasicInternettools(suchasbrowsingand

email),theirexperiencewithusingsocialnetworkingtools(forpersonalandprofessional

use),aswellasratetheirexperiencewithusingcontentmanagementsystems(CMS).Self

reportedratingsweregiveonascaleof1(low)to4(high).Informationfromthissurveyis

summarizedinthefollowingchart.

Participant Occupation Internet SocialNetworkUse

CMS

Skills

Familiar

with

Skills (Personal) (Professional) GRS

1 MiddleSchool 4 3 1 1 1

2 MiddleSchool 4 4 2 1 1

3 HighSchool 4 3 3 3 4

4 HighSchool 4 4 3 3 4

5Elementary

School3 1 1 1 3

6Elementary

School

4 3 1 2 1

7Instructional

Specialist4 4 4 4 4

DESIGNOFTHEUSABILITYEVALUATION

Ineachusabilityevaluationsession,theparticipantwasrequiredtoanswerapre-

evaluationquestionnaire,completeseveraltasksusingtheGRSwebsite,andanswerapost-

evaluationquestionnaire.Thedesignandrationaleofeachcomponentofthisusability

evaluationdrewfromRubinandChisnells(2008)HandbookofUsabilityTesting,aswellas

usabilitytestingmethodologybyNielsen,etal.(2000).


7/28


7

PARTICIPANTGREETINGANDBACKGROUNDQUESTIONNAIRE

Atthebeginningoftheevaluation,eachparticipantwasgreetedbythemoderator.

ThemoderatorexplainedtheGRSWebsite(ifnecessary)andthepurposeoftheevaluation.

Theparticipantwasremindedthattheevaluationisofthewebsiteandnotoftheuser

him/herself,andthatallfeedbackwouldbeconfidential.Theparticipantwasgivenaform

detailingtheirrightsasaparticipant(seeAppendixA)andcompleteashortbackground

questionnaire(seeAppendixB).Inmostcases,theparticipantswereabletocompletethe

backgroundquestionnairebeforemeetingfortheevaluation(throughuseofGoogle

Documentssurveyform).

PERFORMANCEEVALUATION

ParticipantswerethenpromptedtodoseveraltasksusingtheGRSWebsiteasthe

moderatorobserved.ThesetasksweredevelopedusingNielsens(2008)methodologyand

involvefirstimpressions(exploratory),specificitem,andopen-endedtasks.

Thefirsttask(exploratory)requiredtheparticipantstofreelyexplorethesitefor

severalminutes.Themoderatorwillobservetheparticipantandpromptthemwith

questionsastheybrowse(seeAppendixC).Thepurposeexploratorytasksistwo-fold.

First,itallowsparticipantstogainfamiliaritywiththesitetoallowforamorerealisticuser

experience.Secondly,itallowsthemoderatortonoteparticipantsinitialthoughtsofthe

website.

Thenextsetoftasks(specific)requiredtheparticipanttofindspecificinformationas

instructedbythemoderator(seeAppendixD).Themoderatorobservedparticipantsas

theysearchedfortheitems,makingnotesofapproachestheparticipantusedaswellasany

errorscommittedwhenfindingeachitem.Thepurposeofaspecificitemtaskisto

evaluatethenavigationofasitewithaclearendgoal.Evaluationusingthesetypesoftasks

tendstobeveryobjective,becausetasksareeitherpassedwithmeasurablestatistics,or

failedoutright. Thefinaltask(openended)requiredtheusertousethewebsitetocompleteagoal(see

AppendixE).Theuserwillfreelynavigatethesite(andmakeuseofhowevermanyorfew

toolsneeded)tocompletethisgoal.Thepurposeofthistaskistomimicrealuseofthe


8/28


8

website.Thisallowsthemoderatortoevaluatehowanendusermightusethesiteina

realisticandunguidedsituation.

PARTICIPANTDEBRIEFINGANDPOSTEVALUATIONSURVEY

Aftertheobservedevaluation,participantswillfilloutasurveyabouttheir

experienceswiththeGRSWebsite(SeeAppendixF).Participantswillratevariousaspects

ofthewebsite,including:navigability,informationorganization,contentavailabilityand

clarity,andoverallqualityofthewebsite.Throughthesurvey,participantswillalsobe

giventheopportunitytoprovidesuggestionsrelatingtotheseelements.

LOGISTICS

Eachparticipantwastestedseparatelyathisorherindividualworkplaces.Each

evaluationwasdoneusingaMacBookProLaptopwitha2.6GHzprocessorand4MBof

RAM.ParticipantsweregiventheoptionofusingMozillaFirefox,AppleSafari,orGoogle

Chrometocompletetheusabilityevaluation;allparticipantsweremostcomfortablewith

Firefox.Ineachevaluation,thetestingenvironmentmostcloselyresembledthesimple

single-roomsetup(seebelow)fromRubinandChisnelsHandbook.

Simplesingle-roomsetup.

Noneoftheparticipantsrequiredadditionalsupportstomeetaccessibilityneeds.

(However,atwo-buttonexternalmousewasprovided,whichmostparticipantspreferredtouseoverthelaptoptrackpad.)Inalarger-scaleusabilitytest,itwouldbeimportantto

involveparticipantswithvaryingabilitiestoaddressaccessibilitydesignissuesaswell.In

suchcases,modificationswouldhavetobedonetothecurrentconfiguration(suchas


9/28


9

implementationofscreenreaders,screenmagnification,modifiedkeyboards,etc.)to

properlyevaluatetheusabilityofthesewebsites.

DATACOLLECTION

Collectionsofdataoccurredduringtheindividualusabilitysessions.Datasourcesincluded:

Backgroundquestionnaire Moderatornotesfromobservation Screencastsofindividualevaluations Audiorecordingsofparticipantsduringevaluations Postevaluationquestionnaire

Themajorityofdatacollectedforthisstudycamefromqualitativedata.The

questionnairesweredesigntocollectsomequantifiabledata,however,becauseofthesmallnumberofparticipants,itisunlikelythatformalstatisticanalysiswouldyield

significantresults.Therefore,analysisofquantitativedatawaslimitedtodescriptive

statistics.

Qualitativedataanalyzedincluded:

Subjectiveopinionsoftheusabilityandaestheticsoftheproductexpressedbytheparticipants

Non-verbalcuesexpressedbytheuserduringevaluation Participantsanswerstoopen-endedquestions Participantssuggestionsforimprovement

Quantifiabledatacollectedfromtheevaluationincluded:

Participantsratingsofcertainaspectsofthewebsite


10/28


10

REFERENCES

Merriam,S.(2009).Qualitativeresearch:Aguidetodesignandimplementation.San

Franscisco,CA:JosseyBass.

Nielsen,J.,Snyder,C.,Molich,R.,andFarrell,S.(2000).E-CommerceUserExperience:

Methodology.NielsenNormanGroup.[Online]Available

http://www.nngroup.com/reports/ecommerce

Patton,M.(2002).Qualitativeresearchandevaluationmethods(3rded.).ThousandOaks,

CA:Sage.

Rubin,J.andChisnell,D.(2008).HandbookofUsabilityTesting:HowtoPlan,Design,and

ConductEffectiveTests(2nded.).Indianapolis,IN:JohnandWiley&Sons.


11/28


11

RESULTSOFUSABILITYEVALUATION

FINDINGS

Findingsofthestudyareorganizedbyaddressingthefourguidingquestionsthatwerecreatedto

addresstheconcernsofthestakeholders.

QUESTIONONE:WHATAREUSERSINITIALREACTIONSTOTHEGRSWEBSITE?

Beforebeingguidedthroughthetasksoftheevaluation,participantswereaskedto

exploretheGreenRibbonSchoolswebsiteandgivetheirinitialthoughtsofthewebsite.

Initialreactionsweregenerallypositive.Thevisualdesignisappealingandinformative,

however,twoparticipantsmentionedthatthemainpagewasalittlecrowded.Several

participantsmentionedthegraphicsandiconsaspleasing.Withminimalinvestigation,

evenfirst-timeuserswereabletodeterminethattheysitewasintendedforteacherand

studentuse,andpromotedecologicallyfocusedlearning(SeeFigure1).

Figure1.TheGRSMainPage.


12/28


12

Aninterestingthemethataroseduringtheexploratoryphaseoftheevaluationwas

thepersonalconnectionstothesitethatparticipantsmade.Althoughthesiteisapolitical,

usersattachedtheirownbeliefstotheirinterestinthesite.Participantshighlightthe

importanceoftheecologicalandactivistnatureoftheprojects,whileothershighlighted

moreconservativeidealssuchasindividualresponsibility.Oneparticipant(ateacherata

Christianschool)evensuggestedthatlearningthroughnaturewasreligiouslysignificant.

ThisshowsthatthereisabroadpotentialappealforusingtheGRSwebsite.

Itisveryimportanttonotethatalthoughusersfoundthesitetobeappealing,

noviceusersofthesitewerelargelyunawareofthecapabilitiesofgreenribbonschools.org.

Whenaskedtheiropinionaboutthepurposeofthesite,eachofthethreenoviceusers

assumedthatGRSwasarepositoryoflessonideas.Itwasnotuntiltheyweredirected

throughtheevaluationtasksthatusersbegantobecomeawareofsomeofthecapabilities

ofthesite.Additionally,althoughparticipantsmadesomepedagogicalconnectionstothe

potentialuseofthesite,theyalsoproposedsomepedagogicalbarriers.Forexample,one

participantnoted:

asan8thGradeteacher,IdontthinkIcouldusethis.Lastyear,whenI

taught7thGrade-whichislifesciences-IthinkIcouldhaveusedthis.

Thissuggeststhatthereisaninitialperceptionthatthepotentialofthisprogramislimited

toscience(morespecifically,biologicalscience)duetotheobviousecologicalfocusofGRS.Atthebeginningoftheevaluation,eachparticipantwasaskedtologintotheGreen

RibbonSchoolsWebsite.Threeoftheparticipantshadmadeaccountsinthepast,andwere

easilyabletologintostarttheevaluation.Fouroftheparticipants,includingthethree

first-timeusers,hadneverloggedintoGRS,andhadtofirstregister.Eachofthese

participantswasabletofindtheRegisterbuttoninthetoprightcornerwithnodifficultly.

Whenattemptingtoregister,theuserisfirstpromptedwitharequestforaregistration

code.Threeofthefourparticipantswhofacedthisexpressedsomeformofconfusion,asif

thiswasnotexpected.However,eachoftheparticipantsclickedtheDonthaveacode?

linkwithoutanyguidancefromthemoderator.Atthispoint,themoderatorpausedthe

evaluationtoredirecttheparticipantsbysupplyingthemwitharegistrationcode.When

askediftheywouldhavefilledouttheregistrationtogetacode,twoparticipantssaidthey


13/28


13

would,andtwosaidthattheywouldhavetoknowmoreaboutthesitetomakethat

decision.

QUESTIONTWO:CANUSERSNAVIGATETHEGRSWEBSITE?

AfterprovidinginitialperceptionsoftheGRSWebsite,userswereguidedthrougha

seriesoftaskstotestthenavigabilityofthesite.Mostuserswereabletofindinformation

whenprompted,butissuesforbothnewusersandprevioususerscameupwhensearching

forcertaininformationonthesite.Findingideasforactivities,forexample,issomewhat

difficult.Whenauserlogsin,heorshelandsatthedashboard,whichhasinformation

abouttheuserscurrentprojectsaswellastheprojectsintheschool(seeFigure2).These

aretheonlyprojectsthatarevisibletotheuser,soiftherearenootherusersintheschool,

thisareaisempty.

Figure2.UserDashboard.Thisteacherslacksinformationbecauseneitherhernoranyoneinher

schoolstartedaproject.

ThereareFeaturedPublishedActivitieslistedonthesitesmainpage,buttheuser

mustknowtogobacktothemainpagefromthedashboard.Tofindideasforprojects,the

usermayalsosearchthroughthetopnavigation,orsidebuttonlabeledResources:Ideas

forActivities.However,whentheuserisbroughttotheactivitiespage,theyarepresented


14/28


14

withlistsofpotentialactivitieswithoutanydetail.Itisnotobviousthattheideatitlesare

clickableandwillexpandwithshortdescriptions.Eveniftheuserisabletofindthe

description,thedescriptionisminimal,andtheusermustleavethesitetofindfurther

information(seeFigure3).

Figure3.ListofActivities.Somedescriptionisgivenwhentheseareclicked,butitisnotobviousthat

theseareclickable.

Someparticipantsexpectedtofindreadyresources(e.g.lessonplansormaterials)

whensearchingforactivityideas,butwereunabletofindthem.Threeparticipants

mentionedthisasdiscouragingtocontinuingevaluationofGRS,citingtheamountoftimeit

requiredtogetstartedusingthesite.

FindingotherGRSmemberswasalsoproblematic.Asmentionedbefore,theusers

dashboardonlyallowshimorhertoseeuserswithintheirschool.Allusersnoticedthe

InvolvedSchoolsbuttonontheleft,andattemptedtouseittofindnearbyparticipating

schoolsandteachers.However,whenclickingonthis,usersaremetwithalargelistofall

participatingschools.Asearchtoolisprovided,butsearchresultsarereturnedintheform

ofalistingofactivities,whichmaynotbewhattheuserisexpecting,havingjustcomefrom

alistingofregisteredschools(seeFigure4).


15/28


15

Figure4.ASearchforAustinafterclickingonInvolvedSchoolsbringsuserstoalistingofprojects,

notschools.

QUESTIONTHREE:AREUSERSABLETOUSESPECIFICTOOLSONTHEWEBSITE?

Thefinaltaskoftheusabilityevaluationrequiredtheparticipantstostartand

activity.Participantsweregiventheopportunitytosimplyshowhowtheywouldstartan

activity,iftheywerenotabletothinkofone.However,eachoftheparticipantswasableto

thinkofanactivityheorshewoulduseintheirclassroom,andeachsuccessfullybeganan

activity.Findinghowtodothiswasdonewithlittleeffortfromallparticipants.Inall

cases,theparticipantshadanactivityinmind,butwerenotalwayssurehowtocategorize

itwhenenteringtheactivityinformation.Inmostcases,theactivitytheyhadinmindfit

underacoupleofpossiblecategories.Additionally,twoparticipantsmentionedthatthey

forgotorwereunawareofwhattheycategoriesmeant(seeFigure5).Despitethis,each

participantsuccessfullyenteredtheneededinformationwhenstartinganactivity.


16/28


16

Figure5.Categorizationofuser-createdactivitieswasmetwithsomeconfusion.

Partofthetaskalsorequiredparticipantstoshowhowtheywouldinviteastudent

toanactivityonceitwasstarted.Notallusersunderstoodthattheyactivitycodethey

weregivenneededtobesenttostudentsinordertoregister.Thosewhodidsaidthatthey

woulddisplaythecodeonanoverheadprojector,writethecodeonthewhiteboardinfront

oftheclass,orotherwiseguidestudentsinclasstojointheactivity.Threeofthe

participantsincorrectlyassumedthatclickingresendinviteemailwouldenablethemto

directlysendthecodestotheirstudents.Instead,thisopenedthemailclientonthe

evaluationcomputer(seeFigure6).


17/28


17

Figure6.Multipleparticipantsclickedonresendinviteemaillinkexpectingtobeabletosendalink

directlytoanotheruser.Instead,thelinkopenedtheemailclientonthecomputer.

NotalloftheparticipantsinteractedwiththepublishingtoolsontheGRSWebsite,

asthiswasnotrequiredbytheevaluation.However,fouruserscontinuedafterthefinal

tasktoexplorethepublishingtoolsavailable.Onenoviceusernotedthatthepublishing

toolsweresimilartoablogandsuggestedthatthetoolsforpublishingwerefamiliar

enoughtouse.Therewerenospecificcomplaintsaboutthepublishinginterface,however

onemoretechnicallyadvancedparticipantsuggestedthatknowledgeofHTMLisnecessary

togobeyondbasicuseofthepublishingtool.

QUESTIONFOUR:DOESTHEGRSWEBSITEMEETTHENEEDSOFTHEUSER?

Uponcompletionoftheusabilitytasks,aquestionnairewasgiventogatherany

additionalcomments.Theparticipantwasalsoallowedtoaskthemoderatorquestions

duringtheevaluationandgiveverbalfeedbackinadditiontoresponsestothe

questionnaire.Itwasduringthistimethatuserswereabletoreflectonthelikelihoodof

theirabilitytousethisintheirclassrooms.Despiteinitialexcitement,thethreenovice

usersgavepoorratingsforthesurveyquestionsHowlikelyisitthatyouwouldusethis

inyourclass?andHowlikelyareyoutorecommendthistoaco-worker?Expanding

uponthis,userscitedthedifficultytheywouldhaveincorporatingthislearningintotheir

class.Perhapsiftheusabilitystudywereconductedatadifferenttimeoftheyear,user

responseswouldbedifferent,however,therealityofstandardizedtestingwasonthe


18/28


18

mindsofmostoftheteachers.Oneteacherremarkedthatshewouldliketouseit,but

probablyhavetowaituntilaftertheSTAAR(Texasstandardizedtest).Otherteachers

whohadusedGRSinthepastnotedthattheywouldhavelikedtouseitthisyear,but

couldntfindtimeforit.

Beyondpressurefromstandards,confusionaboutthesite(previouslydiscussedin

thisreport)alsoseemedtobeabarriertoperceivedusefulnessofthesite.Whenaskedto

explaintheirdissatisfactionwiththesite,usersnoteduncertaintyofthesitesobjective,

inabilitytofindlessonplans,anddifficultyconnectingwithothersasreasonstheywould

notlikelyusetheGRSsiteintheirclass.

SUGGESTIONS

MAKEPURPOSEOBVIOUSThecurrentinterfacehasalotofinformationfeaturedonthefrontpage.Withinthis

informationaremanylinkstoinformationontheGreenRibbonSchoolsprogramaswellas

thefeaturesofthesite(seeFigure7).Despitetheinformationavailable,datacollectedfrom

noviceusersshowedthattheywereunabletodistinguishthefullpotentialofusingtheGRS

website.Ineachofthethreecases,noviceusersassumedtheGRSsiteasarepositoryof

lessonideas,andhadtobeexplicitlytoldthatthepurposeofthesitewasforcontent

creationandprojectplanning.Designersshouldmakethecapabilitiesasblatantas

possible,highlightingthestudent-focused,interdisciplinary,andprojectbasedlearning

(PBL)natureofthetypesoflessonsGRSpromotes.Thisinformationisreadilyavailable

onthesite,butisnotgivenapriorityintheusersvisualfield,sotheusermustactively

searchtofindinformationexplainingwhytheyshouldusethesite.Asdiscussedearlier,

participantsofthisstudyratedtheirperceivedlikelihoodofusingandsuggestingtheGRS

sitelowduetotheirinabilitytoseetherelevanceoftheprogramasitpertainedtorequired

curriculum.Therefore,itisalsoimportanttoconnectthepossibleprojectstostateand

nationalstandards.


19/28


19

Figure7.Thefrontpagecontainsmanylinkstoinformationexplainingtheprogram,butfailsto

explicitlyoutlinebenefitsupfront.

PROVIDEAVARIETYOFEXAMPLES

Awaytohighlightthecapabilitiesofthewebsiteistoprovideexampleprojects

completedbyactualusers.Itisobviousthatthedesignersofthissitewereawareofthat,

asthemainpageofthesiteallowsuserstoseeFeaturedPublishedActivities.Thisisan

importantelement,andshouldremainonthemainpage.Itissuggested,however,thatthis

elementshouldbeabitmorerobust,featuringpublishedactivitiesbygradeleveland

subject.

Additionally,itshouldbeeasierformemberswhoareloggedintofindexamplesof

projects.Asmentionedinthefindings,userswhoareloggedinareonlyabletoeasilysee

exampleprojectsofmembersintheirschool.Thisisproblematicforthosearethefirstintheirschooltousetheproject,astheyarenotabletoeasilyfindactivities.Apossible

solutiontothisistoeasilylinkteachersfromtheirdashboardstootheruserswhoare

outsideoftheirschool-eitherinthesamedistrict,gradelevel,orgeographicarea.

Additionally,teachersshouldbeabletobrowsemorethanjusttheendproductofaproject,


20/28


20

butbeabletoseematerialsofthevariousstepsittooktogettothatproduct(suchas

lessonplans,studentmaterials,etc.).Anotherpossiblesolutiontoprovidingteacherswith

exampleprojectsistohaveavailablestarterprojectscontainingnotonlytheideasfor

projects,butthematerialsthattheyneedtoplanthem.Thisoffersscaffoldingtothe

teacherswhoareusingthesystemforthefirsttime,sotheyareabletogainexperienceand

comfortwithusingtheprojectmanagementandpublishingtools.

MAKEMINORADJUSTMENTSTOTOOLS

Itisessentialforallpotentialuserstobeabletousetheactivitycreationand

publicationtools.Thismeansthatthesetoolsmustbesimpletousebyeventhosewho

havelowexperiencewithtechnology.Tohelpwiththeusabilityofthesite,designers

shouldexaminethecurrentprocessesbywhichteachersinvitetheirstudentstoprojects

andpublishprojects.

Withthecurrentdesign,teachersmustmakenoteoftheiractivitycodeandshareit

withtheirstudentsthroughexternalways,suchassharingitontheprojectororcopying

andpastingitinanemail,bothofwhichareabitawkward.Itmightbehelpfultoinvite

studentsdirectlyfromtheuserdashboard,ormessagethatpopsupwhencreatingan

activity.Oneparticipantmentionedasitethatheusesinhisclass,andnotedthatheliked

thesitesapproachtojoiningactivities.Onthesitemediaxperiments.com,activities

createdbyateacherarecreatedinasimilarfashiontogroupslikethoseonFacebook(seeFigure8).Theparticipantsuggestedthatthismightsimplifyjoiningactivities,because

studentswouldbeabletojointhesiteandsearchforthegroups,insteadofhavingtohave

thecode(whichtheyarelikelytoforget,lose,etc.).Thisway,anactivestudentmember

canfindanactivitywithoutneedingtorememberthecode,butcanfinditthrough

searchingdescriptions.


21/28


21

Figure8.Mediaxperiments.com.Classprojectsareorganizedasgroups,andsearchablebystudents.

Thefinalsuggestionshouldnotbeconsideredahigh-priority,becauseno

participantstruggledwiththecurrentinterface.However,itwassuggestedbyoneuser

thatanadjustmentbemadetotheinterfacethroughwhichuserspublishmaterials.

Currently,thereareavarietyoftoolsavailabletothemtoformatthefinalpublication.

UsingHTMLisnotnecessary,howeverabilitywithHTMLwouldbebeneficialfor

customization.Itwassuggestedthatallowingusersdrag-and-dropabilityandtheabilityto

editcontentdirectlyintheinterface(similartoKeynoteorPowerPoint)mightbehelpful

foruserswithlimitedtechnicalskills.Althoughthisadjustmentshouldnotbeconsidered

necessary,itwouldaddtotheoverallusabilityofthepublicationprocess.

CONCLUSION

GreenRibbonSchoolspromoteshands-on,problem-basedlearningthrougha

important,relevant,andhighly-discussedtheme.Althoughtechnologybased,theGRSprogrampromotesphysicallyactivelearninginnaturalenvironments.Ithaspotentialto

providesbenefitstostudentlearninginbothcognitiveandaffectivedomains.Inaddition,

itprovidesteacherstheabilitytocollaboratewithothereducatorsandpromote

themselvesasprofessionalsofcontentandpedagogy,whichisbecomingincreasingly


22/28


22

difficulttodointhiseraofboxedcurriculumandstandardizedtesting.Itisforthese

reasonsthatGRSandsitelikeitareimportanttoeducation.Teachersareanextremely

busygroupwithdiverseopinionsonandexperienceswithtechnologyandcontent

creation.Whendesigningforteacheruse,itisimportanttokeepthatinmind.Teachers

arebombardedwithresourcestouseintheirclassrooms,andtodifferentiatethehelpful

fromthenot-sohelpful,designersoftheseresourcesneedtomakevisiblethepotential

impactandpedagogicalsignificanceoftheirresource.Additionally,carehastobetakento

makethesetoolsaccessibletothoseteacherswithlowtechnologyskillsandtraditional

approachestoteaching.


23/28


23

APPENDIXA-INFORMATIONFORPARTICIPANTS

UnderstandingYourParticipation

Youhavebeenaskedtoparticipateinausabilitystudyabouthowwebsitesmeetuser

needs.Ourpurposeinconductingthisstudyistounderstandwhatfeaturesmakesa

websiteeasiertouse.Inthesession,wellaskyoutovisitawebsiteandlookforvarious

things.Whilebrowsingthesite,themoderatorwillaskyouquestionsaboutthesite.

Pleasebehonestinyourresponses,andbeasdescriptiveasyouwish.Pleasekeepin

mindthatthisisatestofthewebsite;wearenottestingyou.Ifyouareunableto

finishatask,itisduetoaflawinthedesignofthewebsite.Allinformationwecollect

concerningyourparticipationinthesessionisconfidential.Participationisvoluntary.Therearenophysicalorpsychologicalrisksassociatedwith

participatinginthisstudy.Youmaybeaudiorecordedduringtheevaluation.Ifyoudonot

wanttoberecorded,pleasetellthetestmoderator.

Thisevaluationwilltakeabout45minutestocomplete.Duringthesession,thestudy

administratorwillassistyouandansweranyquestions.Youmaytakeshortbreaksas

neededandmaywithdrawfromthisevaluationatanytime.

Ifyouhaveanyquestions,youmayasknoworatanytimeduringthetest.


24/28


24

APPENDIXB-PRE-EVALUATIONQUESTIONNAIRE

1.AreyoucurrentlyregisteredwiththeGreenRibbonSchoolswebsite?

[Yes][No]

2.Pleasecheckallthatapplytoyou:

[]Secondary(6-12)Teacher

[]ElementaryTeacher

[]Administrator

[]Pre-serviceEducator

[]Other_________________________

3.HowwouldyourateyourexperienceinbasicInternetability(suchassearching,

browsing,usingemail)?

[Noexp.] 1 2 3 4 [Highlyexp.]

4.Howfamiliarareyouwithusing/contributingtosocialnetworks(e.g.Facebook)?

[Notfamiliar] 1 2 3 4 [Veryfamiliar]

5.Howfamiliarareyouwithusingprofessionalsocialnetworks(e.g.teacherdiscussion

boards)?


6.Howfamiliarareyouwithusingcontentmanagementsystems(Drupal,Moodle,

WordPress)asauser?


7.Howfamiliarareyouwithusingcontentmanagementsystems(Drupal,Moodle,

WordPress)asacontributor?



25/28


26/28


26

APPENDIXD-SPECIFICTASKS

Iamgoingtoaskyoutousethesitetodoseveraltasks.Iwillbetakingnotesasyoucomplete

thetasks,butremember,Iamnotevaluatingyou,Iamevaluatingthesitesabilitytoneed

yourneedsforperformingthetask.Ifyouarecomfortablewithdoingso,pleasethinkout

loudandverbalizeyourthoughtprocessforcompletingthesetasks.Sometaskswillbe

difficultifyouhaveneverusedthesite.Ifyouareunabletocompleteatask,trytofindhelp

usingthesite.

1. FindinformationabouttheGreenRibbonSchoolsAwardProgram.2. Findanideaforaprojectyoucanusewithyourclass.3. Findawaytoshareinformation-eitheraboutaspecificactivity,oraboutyour

account.

4. FindschoolsinAustinthatareparticipatinginGRS.5. Suggestanideaforanactivity.(Ifyoucantthinkofanidea-justshowmewhere

youwouldsuggestone.)


27/28


27

APPENDIXE-OPENENDEDTASK

Scenario

Usethesitetocreateanactivityofyourchoiceforyourschool.Makesureitissomething

youcouldactuallyimplementinyourclass.Ialsowantyoutoshowmehowyouwould

recruityourstudents(butdontactuallydoit).


28/28


28

APPENDIXF-POST-EVALUATIONQUESTIONNAIRE

1.Howwouldyouratethevisualelementsofthissite?(Icons,Colors,etc.)

[Low] 1 2 3 4 [High]

2.Howeasywasittofindtheinformationyouneeded?

[Difficult] 1 2 3 4 [Easy]

3.HoweasywasittonavigatetheGRSWebsite?


4.Howclearwerethenavigationaltools,suchasmenusandicons?

[Confusing] 1 2 3 4 [Clear]

5.Whatchangestothenavigationwouldyouliketosee?

6.Wereyouabletofindusefulorinterestingmaterialonthewebsite?[Yes][No]

7.Ratetheorganizationoftheinformationavailableonthewebsite.

[Poor] 1 2 3 4 [Good]

8.Howeasywasittounderstandtheinformationyoufoundonthewebsite?


9.Howlikelywouldyoubeabletoapplytheinformationavailableonthissite?

[Unlikely] 1 2 3 4 [Likely]

10.Whatchangesincontentorinformationorganizationwouldyouliketosee?

11.Howlikelyisitthatyouwouldrecommendthissitetoanothercolleague?

[Unlikely] 1 2 3 4 [Likely]

12.HowsatisfiedareyouwithyourexperiencewithGRSWebsite?

[Dissatisfied] 1 2 3 4 [Satisfied]

13.Ifyouarenottotallysatisfied,pleasedescribethereasonforyourdissatisfactionbelow.

14.WhatotherchangeswouldyoulikedonetomaketheGRSWebsiteeasiertouse?