Website Evaluation Hawthorne Elementary School Library ... · Page 1 Website Evaluation Hawthorne Elementary School Library Jessica Snapke IST 611 July 3, 2010. Assignment 2, Task

Assignment 2, Task 2 Jessica SnapkeIST 611 July 3, 2010

Page 1

Website EvaluationHawthorne Elementary School Library

Jessica SnapkeIST 611

July 3, 2010


Page 2

Website EvaluationHawthorne Elementary School Library

Name and URL of Website:

The banner at the top of the website reads "Hawthorne Elementary." This titleshould be considered the name of the website. This is a standing banner for the entireHawthorne Elementary School website. The URL is: http://www.hawthornelibrary.tk/.

Evaluation Consultant Contact Information:

Jessica [email protected]

Goals and Learning Objectives:

Goal:

The goal of this project is to thoroughly and objectively evaluate an elementaryschool library website. This evaluation will be completed by using the Content ValidityScale (CVS) and the Website Motivational Analysis Checklist (WebMAC). The websitecreator will use the analysis presented by the evaluation consultant to make anychanges that will enhance the look of the website, the information contained within thewebsite, and/or the ease of use of the website.

Learning Objectives:

• Evaluation Consultant will learn how to professionally evaluate a website usingspecific evaluation tools

• Evaluation Consultant will learn how to create a professional report to expressevaluation results

• Evaluation Consultant will use evaluation results to make recommendations towebsite creator to improve the website

Website Description:

The Hawthorne Elementary School (HSE) Library website is very organized andrather simple. It includes the elementary school running banner at the top of the page, a


Page 3

light gray background, and light blue, sectioned blocks containing the content of thewebsite. The primary purpose of the website is to provide HSE students with links tovaluable educational resources as well as links to fun (yet educational) leisureresources. A second purpose of the website is to provide students (as well as someteachers and parents) links to "happenings" at the school. For example, there is a link tothe Student Council website, multiple links to teacher websites, and a link (for parents)to information about internet safety for kids.

The primary intended audience of the website is the HSE students. A majority ofthe content on the website is geared towards children. The content of the website ismainly links to children's educational tools and resources. The different sections of thewebsite are:

• Calling All Super Heroes

◦ summer reading contest information and rules

• Websites to explore for learning and fun

◦ links to online educational activities, divided by grade

• Summer Fun

◦ links to businesses or organizations with summer reading programs

◦ link to summer outdoor activity program

• SEARCH Library Resources

◦ links to children's research resources

• Student Council

◦ link to the school's Student Council website

◦ includes activities and events Student Council is participating in

• Helpful Links

◦ links to teacher websites

◦ link to school district website

◦ link to student projects

◦ same link to online educational activities, divided by grade

• Reference Sites for Kids

◦ links to reference websites for children

• Become a Facebook Fan

◦ link to library's Facebook page

• Visitors

◦ webpage traffic counter

• Kids on the computer this summer?

◦ link (for parents) to child internet safetyThe two most unique features of the website are the Facebook link and the "SummerFun" section. The Facebook link provides students, parents, and teachers with a very


Page 4

popular and (usually) familiar way to keep up with library activities. The "SummerFun" section is a great way to keep kids engaged with learning in the summer monthswhen they are not receiving lessons every day.

Evaluation Description:

The website evaluation was conducted using two instruments: Small and Arnone’sWebsite Motivational Analysis Checklist (WebMAC) – Professional Version andArnone and Small’s Content Validity Scale. These instruments are, essentially, surveytools.

With the WebMAC, the surveyor is asked 32 questions about the website. Thesequestions pertain to 4 aspects of the website: stimulating (S), meaningful (M), organized(O), and easy-to-use (E). Each category (S, M, O, E) is given a score, ranging from 0 to24. Any category that receives a score ranging from 0 to 8 is considered to need muchimprovement, 9-16 is considered to need some improvement, and 17-24 is consideredhighly motivating. Using these same scores, the WebMAC also measures the website'svalue and expectation for success. To determine value, the scores from the S and Mcategories are added together. To determine the expectation for success, the scoresfrom the O and E categories are added together. These are, then, plotted on a scoringgrid to enable the evaluation consultant and eventually the website create to view howsuccessful the website is. A high mark in both value and expectation for success isdesired.

The Content Validity Scale is, simply, a survey. The person being surveyed is asked ifthey agree or disagree on a 1-5 scale (1, being strongly disagree and 5, being stronglyagree). The statements/questions asked are those mainly pertaining to the website'scontent validity. For each of the three surveys taken, the evaluation consultant hasgiven an overall score to each survey, ranging from 10 to 50 (a score of 10 would be theresult of a surveyor answering every question with a 1 and a 50 would be the result of asurveyor answering every question with a 5).

For the purposes of this evaluation, the evaluation consultant has averaged the scoresfor the three surveys for each of the two evaluation. These average scores will be usedfor data analysis and synthesis.


Page 5

Website Evaluators:

1) Evaluator 1: Evaluation Consultant, 27 year old graduate student2) Evaluator 2: 29 year old biochemist3) Evaluator 3: 30 year old software engineer

Synthesis of Overall Findings:

Table of Data Scores and Meaning:

Subject of Evaluation AverageScore

HighestPossible Score

Meaning

Stimulating 7 24 Needs Much Improvement

Meaningful 17 24 Highly Motivating

Organized 13 24 Needs Some Improvement

Easy-to-use 14 24 Needs Some Improvement

Value 24 48 Average

Expectancy for Success 27 48 Slightly Above Average

Content Validity 44 50 Above Average, Content is Valid

Table 1: Survey results averaged from survey evaluations (3 CVS and 3 WebMAC). Results were rounded up tothe nearest decimal.


Page 6

Scoring Grid for Website's Value and Expectancy for Success:

Low Expectation for SuccessGraph 1: Results averaged from WebMAC (3 surveys)Red = Expectation for Success (27)Blue = Value (24)

Synthesis:

Content Validity Scale:

The results from the CVS were above average. Once tabulated, the evaluationconsultant determined the website scored a 44 out of 50. A high score such as this oneindicates that the website is accurate and free of bias, the sources are credible, andinformation provided is easy to understand.


Page 7

Website Motivation Analysis Checklist:

The WebMAC survey allowed for a much more detailed analysis of the library'swebsite. As stated previously, the survey divides the results into "stimulating,""meaningful," "organized," and "easy-to-use" categories. Again the S and M categorieswere added together to determine value (24) and the O and E categories were added todetermine expectancy of success (27).

The website received a score of 7 for the stimulating category, which indicates it needsmuch improvement.

• Survey answers that impacted this score are:

◦ Unattractive or unappealing screen layout

▪ Example: There is little indication the user is actually on the library page

◦ Lack of opportunity for user interactivity

◦ Lack of variety in formats such as colors, text, images, or sound

▪ Example: The colors used for the website are very basic (red banner, lightblue background)

◦ Lack of novel or unique features that make the website appealing to usersThe website received a score of 17 for the motivating category, which indicates that it ishighly motivating.

• Survey answers that impacted this score are:

◦ The website provided many links to other interesting and/or useful websites

▪ Example: "Website to explore for learning and fun" link is an excellentresource for educational information and games

◦ Website information was accurate and unbiased

◦ Little or no unimportant or redundant information provided on the website

◦ The librarian included his/her email on website to enable communicationfrom users

The website received a score of 13 for the organized category, which indicates that itneeds some improvement. Since this category received an average score, positive andnegative impacts to the overall score will be examined.

• Survey answers that positively impacted this score are:

◦ The website provided plenty of useful information for users

◦ Information provided was presented clearly and consistently

◦ The text on the website was well-written without grammatical, spelling, orother errors

• Survey answers that negatively impacted this score are:

◦ Visual or audio information included on website do not help to clarify or


Page 8

describe the topic

◦ The purpose of the website is not clear

▪ Example: The website contains information for students, teachers, andparents; this information is not clearly sectioned on the website

The website received a score of 14 for the easy-to-use category, which indicates that itneeds some improvement. Since this category received an average score, positive andnegative impacts to the overall score will be examined.

• Survey answers that positively impacted this score are:

◦ Navigating the website does not require any special skills or experience ofusers

◦ All buttons and navigation mechanisms work the way they should

• Survey answers that negatively impacted this score are:

◦ There is no "Help" function provided to users on the website

◦ Some links provided on website are broken, or not functional

▪ Example: When clicking on the links for canvastic.net and picknik.net, thelinks are broken. The URL that appears in the address bar for both sitesstarts with "http://http//."

Recommendations/Suggestions for Improvement:

As determined from the evaluation data, the website has an average value and aslightly above average expectancy for success. Making improvements on each of thefour categories would improve scores for these categories. Here, recommendations toimprove scores in each category will be given, based on the data received. Theevaluators' answers to the open-ended questions asked at the end of the WebMACsurvey are also provided (as recommendations).

Recommendations to improve S score:

• Adjust the style of the website so it is geared more towards kids

◦ add brighter colors

◦ change heading fonts

◦ include pictures of the library and patrons using the library

• Make the website more interactive for the students

◦ Create a space where students can post questions, interesting finds, orprojects they are working on

• Make the primary focus of the site functions of the library (such as how andwhere to find resources) and event taking place at the library

◦ Make the catalog more prominent


Page 9

◦ Create a separate section for the links Recommendations to improve M score:

• Make resource links more prominent

• Make interaction between librarian and users more available

◦ Instead of just providing an email address, provide quicker access to thelibrarian, such as chat reference

Recommendations to improve O score:

• Clarify the primary audience of the site

◦ create a clearly designated section for teachers and parents such as a"Teachers' and/or Parents' Corner"

Recommendations to improve E score:

• Fix broken links

• Create a help menu with tips for navigating the website as well as tips forfinding resources

◦ quicker access to the librarian will also help with this

Open-ended questions:

In addition to surveys, the website evaluators were also given the opportunity toanswer open-ended questions about the website's strengths and what needed to beimproved on. These are some of their responses:

• "The strengths of this website lie in all of the educational links provided."

• "The appearance of this website needs to be changed in order to appeal tochildren."

• "The website seems to be more of a collection of links rather than a website forthe library."

• "When you view the website, you don't really get a 'feel' for what goes on at thelibrary. Perhaps some lists of library projects or pictures of students using thelibrary would help with this."

• "The purpose of the site is not clear to me, is it for students or parents? Someheading at the top of the page clarifying the purpose and hinting at the intendedaudience would be helpful."


Page 10

Reflection of Evaluation Experience:

I think evaluating school library websites is an important task. With the overflow ofinformation on the web, it is easy for children to simply go there to obtain theirresources for school, regardless of its validity. What school libraries should aim to do ismake the library a fun and useful place to go, physically and virtually by providingaccurate and valid resources, lessons on how to obtain these resources, and a way tointeract with other students, librarians, and teachers about what they are finding as aresult of access to these resources.

Evaluating websites created by libraries can bring in opinions and suggestions fromvarious sources on how to get kids in the virtual door of the library. I think theWebMAC survey does a good job of addressing the various elements that should be thestrengths of a library website. I do think, however, that the website's appearance is keyin drawing kids; kids do judge a book by its cover. The WebMAC survey touches onthis element, however, I do not think enough questions regarding this issue were asked,thus, lower scores (as was the case with this website) were not given much weight inthe final score in the "stimulating" category. I think that all the elements that theWebMAC covered are important, but I think that a more open-ended survey would bemore effective. Evaluators would be able to "back up" their scoring with moreinformation about why they chose what they did. I also feel that the lack of any open-ended questioning with the Content Validity Scale made the survey virtually useless.In many cases, evaluators came up with completely different answers and withoutopen-ended questions, the person analyzing the data was unable to determine whywithout more information.

I enjoyed the opportunity to make an impact on the library's website. When I amfinished here, I will be bookmarking the library's page and checking back to see thechanges!


Page 11

Bibliography

Arnone, M. & Small, R. (1999). Content validity scale. Retrieved on June 20, 2010 from https://ilms.syr.edu/webct/urw/lc9140001.tp0/ cobaltMainFrame. dowebct

Arnone, M. & Small, R. (n.d.). Website motivational analysis checklist (WebMAC) professional. Retrieved on June 20, 2010 from https://ilms.syr.edu/webct/urw/lc9140001.tp0/ cobaltMainFrame. dowebct.


Page 12

Content Validity Scale and Website Motivational Analysis Checklist

Content Validity Scale© (Arnone & Small, 1999)

Instructions: Rate your agreement with the following statements.

1. The source of information for this Web site is credible.

Strongly Disagree 1 2 3 4 5 Strongly Agree

2. There is a way to contact the author of this Web site, if necessary.


3. The factual information or content of the Web site seems accurate.


4. If the Web site presents concepts or principles in its domain (e.g. science,

art), they are appropriately presented without confusing or missing information.


5. There are no typographical or spelling errors that could potentially cause

the information at this Web site to be misunderstood.


6. The content is appropriate for the intended audience.



Page 13

7. The links from this site appear to be credible.


8. This Web site appears to be free of bias.


9. The information at this Web site is current enough for the type of

information it includes.


10.The links from this Web site appear current and unbiased.



Page 14

Website Motivational Analysis Checklist (WebMAC)

Professional (v. 2.0)

DIRECTIONS:

WebMAC Professional was designed for professional educators to assess Web sites for use in

classroom instruction or for student use in homework assignments or projects. Before using

WebMAC Professional, it's a good idea to spend some time exploring the Web site to be evaluated

in order to have some familiarity with its content and structure. You may need to go through the

Web site at least once more to complete this checklist.

Rate your level of agreement with each of the 32 items on the next two pages by placing the

appropriate number value on the line in front of each item. If you are not sure about any item,

select the best response you can give.

3 = strongly agree

2 = somewhat agree

1 = somewhat disagree

0 = strongly disagree

NA = Not Applicable

Example of completed item:

3 0. This Web site makes me happy.

After you have rated all 32 items, go back and look at each NA item. You should place one of the

following scores next to the NA for that item.

0 = the Web site would have benefited if it had included the item (e.g. audio).

1 = the Web site didn't require that item.

2 = the Web site was better off for not including that item.

Copyright 1999 (revised 2004) by Ruth V. Small & Marilyn P. Arnone, Syracuse, New York. All

rights reserved. No part of the material protected by this copyright notice may be reproduced or

utilized in any form or by any means, electronic or mechanical, including photocopying, recording ,

or by any information storage and retrieval system (except for non-profit educational use),

without the written permission of the copyright owner.


Page 15

WebMAC Professional (v. 2.0)

Your Name

______________________________________________________________

_______________

Grade level/subject

______________________________________________________________

________

Web Site URL

______________________________________________________________

____________

Directions: Place the appropriate number about this Web site on the line preceding

each item.

3 = strongly agree

2 = somewhat agree



NA = Not Applicable

___ 1. The screen layout of this Web site is attractive.

___ 2. There is a menu or site map at the beginning that describes what content

is contained within the Web site.

___ 3. Visual (e.g. videos, photographs) or audio information included in this Web

site helps to clarify or describe the topic.

___ 4. Navigating this Web site does not require any special skills or experience

by me or my students.


Page 16

___ 5. There is an eye-catching title and/or visual on the home page of this Web

site that will attract students' attention.

___ 6. This Web site provides valuable links to other useful Web sites.

___ 7. The purpose of this Web site is always clear to me.

___ 8. The Web site has a help function that I or my students can use at any

time.

___ 9. This Web site provides opportunities for interactivity.

___ 10. The Web site's information is provided by credible sources.

___ 11. The directions for using this Web site are simple and clear.

___ 12. This Web site makes it easy to search or query for information.

___ 13. The information on my topic included in this Web site is interesting.

___ 14. The information contained in this Web site is current and up-to-date.

___ 15. There is useful information on my topic at this Web site.

___ 16. This Web site's graphics are crisp and clearly visible.

___ 17. The variety of formats (e.g. text, images, sound) helps to maintain

attention.

___ 18. The information at this Web site is accurate and unbiased.

___ 19. All of the information at this Web site is presented using clear and

consistent language and style.

___ 20. All of this Web site's hyperlinks are active and fully functioning.


Page 17

WebMAC Professional (v. 2.0)

page 2

Name_______________________________ URL

_____________________________

3 = strongly agree

2 = somewhat agree



NA = Not Applicable

___ 21. This Web site has novel or unique features that make it more interesting

for students.

___ 22. There is little or no unimportant or redundant information at this Web

site.

___ 23. The text at this Web site is well-written without grammatical, spelling,

or other errors.

___ 24. At all times, I can control what information at this Web site I wish to

see or show my students.

___ 25. This Web site stimulates curiosity and exploration.

___ 26. This Web site provides opportunities for users to actively participate or

contribute ideas.

___ 27. This Web site provides the appropriate amount of information on the

topic for a lesson or assignment.

___ 28. All buttons and other navigation mechanisms for moving around at this

Web site work the way they should.


Page 18

___ 29. The colors and/or background patterns used in this Web site are

pleasing.

___ 30. This Web site provides opportunities to communicate with its authors.

___ 31. No matter where I am in this Web site, I can return to the home page.

___ 32. The amount of time it takes for pictures, games, videos, etc. to appear

on the screen at this Web site is reasonable for teaching/learning

situations.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ __ _ _ _ _ _ _ _ _ _ _ _ _

There is enough at this Web site to warrant a return visit. YES

NO

I would recommend this Web site to my colleagues. YES

NO

What is the greatest strength of this Web site?

_______________________________

______________________________________________________________

______

______________________________________________________________

______

What needs improvement at this Web site?

___________________________________

______________________________________________________________

______


Page 19

Scoring WebMAC Professional (v. 2.0)

Before transferring scores, go back to each item designated as "NA" and choose a score for

that item based on the following criteria:

• 0 points if you feel that the site would have benefited if it had included the item (e.g. audio).

• 1 point if you feel that the Web site didn't require that item.

• 2 points if you believe the site was better off for not including that item.

When all 32 items have a number score, transfer each score into the appropriately numbered

space in one of the four columns below. Then add the scores for each column and write the total

on the appropriate line.

S M O E

1. ___ 2. ___ 3. ___ 4. ___

5. ___ 6. ___ 7. ___ 8. ___

9. ___ 10. ___ 11. ___ 12. ___

13. ___ 14. ___ 15. ___ 16. ___

17. ___ 18. ___ 19. ___ 20.

___

21. ___ 22. ___ 23. ___ 24.

___

25. ___ 26. ___ 27. ___ 28.

___

29. ___ 30. ___ 31. ___ 32.

___


Page 20

TOTAL S ___ TOTAL M ___ TOTAL O ___ TOTAL E ___

The S column reflects how STIMULATING this Web site is for you.

The M column reflects how MEANINGFUL this Web site is to you.

The O column reflects how ORGANIZED you think this Web site is.

The E column reflects how EASY-TO-USE this Web site is for you.

Transforming Scores

Now you may transform your scores into visual representations so that you can

clearly see the results of your evaluation.

11. Take each of the four total scores above and plot them (either with a dot or

a bar) on the graph below.

12. If you use dots, connect the dots to make a line.

Now take the total scores from the four categories (S, M, O, E) and add them together in

the following way to get your two overall motivational quality scores.

S + M = ______ (V) O + E = ______ (XS)

30 24

2 16

8

0


Page 21

The V score reflects a summary motivation score on the Value dimension; i.e.

how stimulating and meaningful the Web site is. The XS score reflects a summary

motivation score on the Expectation for Success dimension; i.e. how organized and easy-

to-use the Web site is.

Plotting the Scores

To plot scores on the scoring grid,

2. Place a dot for the V score along the Value continuum and a dot for the XS

score along the Expectation for Success continuum on the grid.

3. Draw straight lines to their intersection point, representing the overall

motivational quality score of the Web site

In the example below the evaluated Web site received a Value score of 40 and an

Expectation for Success score of 18. Their intersection point falls within the lower right

quadrant of the grid. Now go to the next page to record your scores.


Page 22

Example Completed Scoring Grid

To record your actual scores onto the scoring grid.

1. Plot the score for V along the Value continuum and the score for XS along

the Expectation for Success continuum.

2. Draw straight lines to the intersection point.

| | | | | | | | | | | | | | | | | | | | | | |

0

48

0 483216

16

32

2440

40

8

8

Average to high for value/Below average to

low for expectation for successBelow average to low for value/Below

average to low for expectation for success

High Expectation for Success

Low Expectation for Success

Low Value High Value

High Expectation for Success

Low Expectation for Success

Low Value High Value

X 40V;18XS


Page 23

Average to high for value/Average to Average to high for expectation for

Documents

Website Evaluation Hawthorne Elementary School Library ... · Page 1 Website Evaluation Hawthorne Elementary School Library Jessica Snapke IST 611 July 3, 2010. Assignment 2, Task