Evaluation Of Commercial Website

International Journal of Information Management 24 (2004) 313–328

Quantitative evaluation of commercial web sites:an empirical study of Spanish firms

F.J. Miranda Gonz!alez*, T.M. Banegil Palacios

Facultad de Ciencias Econ !omicas Empresariales, Avda de Elvas, s/n, Badajoz 06071, Spain

Abstract

Despite the rapid growth of Electronic Commerce in recent years few studies have been published whichassess the efficacy of current commercial web sites. This paper aims to provide a detailed and original WebAssessment Index, which focuses on four categories: accessibility, speed, navigability and content. Thesuggested methodology is equally valuable for analyzing commercial, educational and non-profitmakingorganizations. The scoring mechanism is explained and the index is applied to assess the web sites of thelargest firms in Spain. Finally, a series of implications of this study are presented.r 2004 Elsevier Ltd. All rights reserved.

Keywords: World Wide Web; Electronic commerce; Content analysis; Web design

1. Introduction

The Internet has been evolved from a basic tool of communications into a vast and interactivemarket of products, services and ideas involving over 240 million users worldwide. The radio tookover 30 years to reach an audience of 60 million while television took 15 years. The Internet hasobliterated records for the rapid diffusion of a major technology (Simeon, 1999).The diffusion of the Internet, and with it electronic commerce, is promoting a transformation in

the business landscape as new business models emerge as feasible alternatives to existing ones. TheInternet has the potential to market products and services to customers, to communicateinformation to a global community, to provide an electronic forum for communications and toprocess business transactions such as orders and payments.

ARTICLE IN PRESS

*Corresponding author.

E-mail address: [email protected] (F.J. Miranda Gonzalez).

0268-4012/$ - see front matter r 2004 Elsevier Ltd. All rights reserved.

doi:10.1016/j.ijinfomgt.2004.04.009

According to Forrester Research,1 the number of firms on the Internet was expected to increasefrom 287,000 in 1998 to 1 million by the year 2002. The dollar value of electronic commerce (EC)transactions also continues to increase at an outstanding rate. The amount of money spent byonline shoppers virtually doubled every year and it was expected to reach $4.4 billion by the year2003 (Gartner Group2). As the Internet expands and web sites proliferate, more systematicapproaches to web site assessment are necessary.Among the Internet applications, the World Wide Web is one of the most relevant driving

forces in the commercial usage of the Internet, although it is still at the initial stage of itsdevelopment with a relative minor quantitative importance compared with the traditionaltransactions. In 1994, there were only about 3000 web sites on Internet, and a few years later, thenumber was increased to 4.27 million, and 1.5 million web pages were born daily (Gromov, 2000).The purpose of this paper is to develop a web site assessment index that can be employed to

compare the current usage of the Internet by different organizations. We begin by identifying themain factors considered as determinants of web site quality as outlined in previous studies.Secondly, we discuss the design of the web assessment index. In order to test its validity weanalyzed the web sites of the top 200 Spanish firms. The paper concludes with a discussion of theresults and future research directions.

2. Web sites assessment tools

The number of web sites and their size are increasing. Researchers at OCLC determined thatthe World Wide Web has about 3.6 million sites (these sites contain nearly 300 million web pages),of which 2.2 million are publicly accessible (OCLC Research, 1999).Often web developments are undertaken with only limited resources in terms of time and money

and as a result of this many sites are poorly designed. The Gartner Group pointed out that 75% ofbusiness web sites do not meet customer requirements and thus need to be redesigned(Christopher, 1997). However, a comprehensive mechanism for systematically assessing thevarious components of a web site has not yet been introduced.That is why we propose a new assessment tool in order to measure a web site position and

identify its main strengths and weaknesses.A review of the recent literature on web site assessment reveals some attempts to measure web

site validity (Selz & Schubert, 1997; Liu, Arnett, Capella, & Beatty, 1997; Ho, 1997; Evans &King, 1999; Simeon, 1999; Huizingh, 2000; Young & Benamati, 2000; Bauer & Scharl, 2000).Most of the previous approaches have focused either on basic content management or on aspecific set of web site outcomes. Besides, majority of these studies have examined only a fewaspects of commercial sites and have an after-the-fact orientation in that they focus solely oncustomer attraction and performance of existing web sites.The web site quality evaluation method (QEM) proposed by Olsina, Godoy, Lafuente, and

Rossi (1999) can be considered as one of the main approaches. Among the main factors analyzedin this study we can find: functionality (global search, navigability and content relevancy),

ARTICLE IN PRESS

1E-commerce report. Forrester Research, Inc. http://www.forrester.com2http://www.economist.com/editorial/freeforall/current/index sb3684.html

F.J. Miranda Gonzalez, T.M. Banegil Palacios / International Journal of Information Management 24 (2004) 313–328314

http://www.forrester.com

http://www.economist.com/editorial/freeforall/current/index_sb3684.html

http://www.economist.com/editorial/freeforall/current/index_sb3684.html

usability (site map, addresses directory), efficiency and site reliability. However, the excessivenumber of attributes employed raises some subtle problems of a computational nature (Bauer &Scharl, 2000). The desired assessment can often be better achieved using only a few, but highlyrelevant features of the sample to be analyzed.Most of the previous assessment models employ subjective factors, such as easy access, clear

text, presentation quality, colours, sounds, etc. To minimize this subjectivity, site evaluatorsshould be given precise guidelines to rate each factor and a large group of evaluators is needed(Evans & King, 1999).Trying to avoid the main failures of previous models, we will proceed to develop a new web site

assessment index that can be employed to compare the current use of the Internet by differentcompanies.According to Evans and King (1999), a web assessment tool must have five main components:

categories, factors, weights, ratings and total score. The first step is to choose the categories andfactors that are critical to web site effectiveness. Based on a detailed review of literature available,the index developed in this paper (Web Assessment Index or WAI) uses four broad categories asthe basis for a quality web site: accessibility, speed, navigability and site content (see Fig. 1).Also based on this literature and in the researchers experience, in teaching electronic commerce,

the key factors within each category were enumerated. These factors must reflect what aregenerally considered to be important components and features of web sites by typical users.The first category in the web assessment index (WAI) is accessibility. It is clear that the quality

of a web site is increased if the site is easily identifiable and accessible to the users. Merelycounting ‘‘hits’’ on a page is not an accurate measurement of quality or success of a web site(Murray, 1997). To truly value a web site, measurable objectives need to be defined. So, we haveemployed two factors to measure this category: search engine presence and link popularity.Higher search engine rankings translate into greater traffic of a site and therefore, increase its

degree of accessibility. In our research we have chosen Yahoo Spain and Terra, because thesesearch engines are most frequently used by Spanish internet users according to the last GeneralMedia Study (June, 2000) (see Fig. 2).The second factor used to measure accessibility is the site-popularity. The most common

measure of web performance is the number of ‘‘hits’’ a site generates. However, there are accuracyproblems with this, because if a page contains graphics, each graphic is counted as a new hit andthere are some methods to artificially increase the number of hits. So, the total hits do notnecessarily reflect the actual number of visits to the site.

ARTICLE IN PRESS

CONTENT

SPEED

NAVIGABILITY

SITEQUALITY

Fig. 1. Web site quality assessment model.

F.J. Miranda Gonzalez, T.M. Banegil Palacios / International Journal of Information Management 24 (2004) 313–328 315

With this in mind, we decided to employ another means of measurement: we defined linkpopularity as the number of external links on the web that point to the web site being analyzed.The advantages of a large number of links to a site are evident: firstly, the more sites that link toyou, the more traffic you can expect to receive and secondly major search engines will rank yourpages more highly when you have more links to your web site. In our study we used theMarketposition Web Service (www.marketposition.com/linkpopularity.htm) to check the linkpopularity of each site.Access speed and response time are obviously very significant, because time is always a critical

factor. The access speed can be measured with a chronometer, but this recording is influenced by agreat number of factors such as hardware employed, connection hour, web traffic, etc. So, we aregoing to employ an indirect measure of speed: the size (in bytes) of the home page, including thesize of all the image files included. The speed at which the main page is loaded completely isrelated to its size, so a shorter home page will imply a faster access.3

The third category in our index is what we have called navigability. Forrester Researchestimates that poor web design will result in a loss of 50% of potential sales due to users beingunable to find what they want, and a loss of 40% of potential repeated visits due to an initialnegative experience.Given that users should never feel lost, each page should be self-sufficient and provide links to

the main contents. The factors used to assess this category are the following:

* Permanent site menu allowing a rapid access to the different sections from every page.* Keyword search function, for users to locate available interesting items within the companyhome page.

ARTICLE IN PRESS

0

5

10

15

20

25

Yahoo

Terra

Altavista

Ozu'

Lycos

Excite

Infoseek Ya

Telepolis

Hispavista

MSN

Other

Fig. 2. Search engines ranking in Spain.

3Although this is not an optimal measure of speed, its objective nature made it an acceptable indicator of speed.


The content quality of the web site will be measured assessing the presence of relevantinformation to the users. A site must have content that satisfies users’ needs and it should beupdated at least every month. Web sites can contain various features, including virtual catalogues,order forms, contact information to enable customer feedback, general company information, etc.Important information should be immediately accessible. If 80% of your users are seeking 20%

of your information, then that information should be the most visible and the easiest to access.Basic contact information of the organization should be on the main menu page and relatedinformation should be grouped together rather than scattered in different sections of the site.The factors selected were based on site contents identified in previous studies (Young &

Benamati, 2000; Huizingh, 2000), practitioner journals and the researchers’ experience in teachingelectronic commerce. We have considered three sets of factors to assess the content of a web site:

2.1. Informational factors

Web sites are largely informational providing commercial and non-commercial informationabout the firm. They can provide insight into the background of the company, partners,important customers and social policies. Another important factor is the product/servicedescription, including price, specifications, photographs, etc. Of late companies are using theirweb sites to provide information that is useful to its stakeholders, especially financial information.So, we have considered the following nine informational factors:

* company background,* product/service description,* daily news highlights,* financial information,* job opportunities,* link to firm divisions,* search for dealers, agents, stores, etc.,* check order/shipment status,* external links.

2.2. Transactional factors

A transactional web site refers to a site where there are pages for placing orders or conductingonline financial transaction. Some sites request log-on to carry out a transaction, others do not.

2.3. Communicational factors

Given that web sites are often entry points to a company and visitors typically want to accessinformation about the company, contact information for the organization should be on the mainmenu page and therefore easily accessible.However, web site visitors often are looking for entertainment elements like jokes, cartoons,

games, screensavers, etc. Like TV-commercials, web sites must increasingly provide informationabout product and services and simultaneously entertain their web visitors.

ARTICLE IN PRESS


In order to evaluate the communicational capacity of each web site, we identified whether thesite contains the following items:

* contact e-mail,* contact telephone or address,* receive news by e-mail,* free e-mail service,* personalization capacity,* entertainment elements.

After that, the categories and factors were weighted (based on a total of 100 points4). Giventhat site content is critical, as is the ability of the site to access and deliver information, we haveassigned to this category 50 points. Accessibility and navigability are assigned 20 points each,followed by speed with 10 points.These weightings have been assigned from the analysis of previous studies and take into

account the views of different electronic commerce experts. The use of these subjective weightingsmay be considered as the main limitation of our study; however, many recent studies (Olsina et al.,1999; Bauer & Scharl, 2000; Buenadicha et al., 2001) have employed similar subjective scales.The results of this final phase is the Web Assessment Index (WAI) shown in Table 1 above.

ARTICLE IN PRESS

Table 1

Web assessment index

Categories Weights Categories Weights

ACCESSIBILITY 20 NAVIGABILITY 20

Presence in search engines 5 Site map 15

Popularity 15 Keyword search function 5

SPEED 10

Site size (bytes) 10

CONTENTS QUALITY 50

Informational content 20 Transactional content 15

Company background 4 Online orders 15

Product/service description 4 Communicational content 15

Daily news highlights 2 Contact e-mail 5

Financial information 2 Contact telephone or address 4

Job opportunities 2 Receive news by e-mail 1

Check order/shipment status 2 Free e-mail service 1

Search for dealers, agents, stores, etc. 2 Personalization capacity 2

External links 1 Entertainment elements 2

Link to firm divisions 1

TOTAL 100

4The greater the number of points, the more important the factor.


3. Spanish top 200 firms web sites assessment

Using the WAI the top 200 Spanish firms web sites were visited and reviewed between Octoberand December, 2000. The ranking published in the Actualidad Economica Journal5 and thesearch engines Yahoo (es.yahoo.com) and Terra (www.terra.es) were used to attempt to locate theURL for each firm. After that a separate search was performed for home pages of companieslisted as not having a page in the search engines used. Telephone calls were made to these firms.This additional survey found three additional company home pages.Most of the top 200 Spanish firms (70.1%) have a public web site in Spanish. According to the

WAI, the best web sites are in the transport industry. However, the most valuable output fromour study is not the ability to identify the best sites, but to see how each firm site is compared torelated sites and to identify ideas and practices that can improve firm sites.Table 2 on the next page presents summary data on the results of the study by industry. The

main results are summarized and discussed in the following sections, which correspond, respectively,to the broad categories previously identified: accessibility, speed, navigability and site content.

3.1. Accessibility

Mass media and transport industries show the greater values (Fig. 3), especially in thepopularity index,6 followed by travel & tourism, electric energy, electric & electronic equipment,etc. However, the highest value obtained in the popularity factor by a Spanish firm (Telef !onicawith 8.898) is far from the value obtained by the main international firms (Microsoft with14.261.108).The results of our study show a high correlation value7 (0.714) between site popularity and firm

size so, the greater the number of employees the greater the popularity of the web sites. This resultwas expected, because larger firms can obtain more resources to invest in the promotion of theirweb sites, which implies they are the most popular.

3.2. Speed

Typically, those sites whose whole pages loaded quickly were also easily navigable. For userswith a typical modem connection waiting several seconds for a single page to appear is commonand although telecommunication firms are researching and enacting bandwidth-boosting optionslike ISDN and cable modems to improve web speed and performance, transmission speed is oneof the main obstacles for Internet operability. Everything possible must be done to maintain thespeed of downloading web pages: adding servers and links, upgrading Java, using the latestreleases of Real Player and Shockwave, include self-content features, etc (Fig. 4).As already mentioned, the speed with which, the main page loaded completely was estimated in

our study indirectly by the size of the main page. Iron and Steel, and paper & graphic arts were thesites that received the highest possible rating in this area.

ARTICLE IN PRESS

5The ranking can be found at the site: www.recoletos.es/actualidadeconomica.6Popularity has been obtained as the larger value in the different search engines.7Significant at 99% (two-tailed).


http://es.yahoo.com

http://www.terra.es

ARTICLE IN PRESS

Fig. 3. Accessibility by industry.

Table 2

WAI values for the Top 200 Spanish web sites

Industry (number of firms) Speed Accessibility Navigability Content Index

Transport (12) 67.83 41.33 83.33 72.83 68.13

Travel and tourism (5) 73.69 32.45 75 65.6 61.66

Mass media (3) 62.27 44.68 83.33 55.33 59.49

Electric and electronic material (4) 76.17 26.41 75 53.5 54.65

Petrol (5) 73.63 22.13 85 50 53.78

Construction (7) 78.73 24.06 85.71 44.28 51.97

Electric energy, water and gas (6) 75.99 28.22 83.33 42 50.91

Textile and shoes (3) 73.34 9.47 75 52 50.22

Trade (16) 76.88 21.21 75 45.62 49.74

Iron and steel industry (4) 87.90 22.18 75 42 49.22

Paper and graphic arts (2) 86.87 6.45 87.5 42 48.47

Others (5) 78.79 23.93 75 38.4 46.86

Business service (9) 71.07 16.95 72.22 42.88 46.38

Machinery and mechanical equipment (2) 81.54 23.15 87.5 30 45.28

Chemistry and pharmacy (6) 69.68 19.93 70.83 40 45.12

Transport equipment (18) 69.91 24.39 75 35.66 44.70

Food, drinks and tobacco (13) 78.75 17.98 65.38 35.07 42.08

Metallic and mineral products (2) 81.89 25.25 37.5 22 31.74

Total 74.47 24.40 75.81 45.50 50.24

Note: Each factor was measured on a 1–100 scale.


3.3. Navigability

We assessed how easy it was to navigate around the site, how easy it was to return to the mainpage of the site and how easy it was to find relevant information (Fig. 5). The site must be easy tonavigate and make it simple to return to the home page, links to components within the siteshould be available from throughout the site and security must also be appropriate for theinteractions conducted at the site (Evans & King, 1999).

ARTICLE IN PRESS

Fig. 4. Speed index by industry.

Fig. 5. Navigability index by industry.


As previously stated, two factors have been used to analyze the navigability of a web site:

* Permanent site menu allowing rapid access to the different sections of every page,* keyword search option.

The better sites in terms of navigability were paper & graphic arts, and machinery & mechanicalequipment.Almost 94% of sites provided a permanent menu or site map allowing easy navigation through

the site. The presence of this menu is essential to avoid users from getting lost, informing them ofwhere they are on the site and how to find the information they want.When a visitor accesses the firm site, he is usually looking for some specific information. In

order to find this information quickly, the presence of a keyword search function is extremelyuseful as it helps the user to locate the specific information easily. So, if users can access theinformation they seek quickly (in a little number of clicks8) their satisfaction will be increased andthe site will probably increase the number of its visitors. This function appears in only 23% of thesites analyzed, which is therefore a bad indicator of the navigability of Spanish firm web sites.

3.4. Site content

Site content has been assessed by considering the relevant information that must appear in afirm web site according to the needs of potential users. Sites that show a higher value in thecontent factor are the transport, and, travel & tourism industries, followed by mass media, electric& electronic equipment and textile & shoes (see Table 3).Given their high informational content, transport, electric energy, and gas & water industries

must be highlighted. The kind of information that is most often available on a web site is specificproduct details (see Fig. 6). Information about the background of the company is available on91.8% of the sites.Just one-fifth (18.9%) of the sites in the study were transactional,9 though this percentage varies

greatly between each industry. Transport (83.3%) and travel & tourism firms (80%) have a higherpercentage of transactional web sites than other industries. The results show that the era ofelectronic commerce had not come to Spain and most corporations were not motivated byInternet sales to set up a presence on the web.Mass media, paper & graphic arts, and electric & electronic materials show the highest values in

the communicational content factor. A contact e-mail address was found only in 71.3% of theweb sites assessed. Most sites provide an alternate means of contacting them with only about 82%of them providing a phone number and 76.2% a mailing address.About 25% of all sites contain some form of entertainment (pictures, videoclips, games, etc).

Indeed, some sites are exclusively dedicated to entertaining their visitors, with videoclips, gamesand competitions. This is particularly the case of most drinks firms: in fact it is unusual to find anyinformation about the firm, contact information or product prices on their web sites.

ARTICLE IN PRESS

8Research shows that you have just three clicks to answer web visitors’ questions before they leave in frustration

(Iconocast, 2000).9 In this study, a transactional site refers to a site where there are pages for placing orders or conducting online

transactions, with or without user log-in.


ARTICLE IN PRESS

Backg

roun

d In

form

ation

0

10

20

30

40

50

60

70

80

90

100

Produ

ct se

rvice

des

cript

ion

News h

igh lig

hts

Searc

h fo

r dea

lers,

store

s, et

c.

View fin

ancie

rs

Link t

o fir

m's

divisi

ons

Job

oppo

rtunit

ies

Extern

al lin

ks

Check

ord

er sh

ipmen

t sta

tus

Fig. 6. Informational content.

Table 3

Content category by industry

Industry Informational

content

Transactional

content

Communicational

content

Content

Transport 75.41 83.33 58.88 72.83

Travel and tourism 61 80 57.33 65.6

Mass media 61.66 33.33 68.88 55.33

Electric and electronic equipment 65 25 66.66 53.5

Textile and shoes 61.66 33.33 57.77 52

Petrol 68 20 56 50

Trade 60.93 25 45.83 45.62

Construction 68.57 0 56.19 44.28

Business services 63.33 11.11 47.40 42.88

Electric energy, gas and water 70 0 46.66 42

Paper and graphic arts 55 0 66.66 42

Iron & steel 65 0 53.33 42

Chemistry and pharmacy 61.66 0 51.11 40

Others 62 0 45.33 38.4

Transport equipment 55.27 0 45.18 35.66

Food, drinks and tobacco 53.46 0 45.64 35.07

Machinery & mechanical equipment 55 0 26.66 30

Metallic and mineral products 32.5 0 30 22

Total 61.80 18.85 50.43 45.50


Our index shows a clear orientation towards sites that sell products online. However, electronicordering is more appropriate for some industries than others. Some products cannot be sold offthe shelf, because they must be configured for each customer, so it is not fair to score sites sellingcustomized and non-customized commodities in the same way.So, we decided to use a different weighting scheme for industries that use their sites to support

sales and another scheme for industries that use their sites to promote the company’s image,attracting customers to their physical stores.10 The first group (firms that sell or try to sellproducts online) include the following industries: Transport, Travel and tourism, Mass media,Electric and electronic equipment, Textile and shoes, Petrol, Trade and business services. In thesecond group (firms that use their web sites only as a promotion mean) we have included:Construction, Electric energy, Gas and water, Paper and graphic arts, Iron & steel, Chemical andpharmaceutical, Transport equipment, Food, drinks and tobacco, Machinery & mechanicalequipment, and Metallic and mineral products.The weighting scheme for the first group appears in Table 1 and for the second group in the

following table (Table 4). As we can see, only the content category has changed (the transactionalcontent factor has disappeared from the index and other factor weights have changed), trying toreflect the relative importance of the different factors when online transactions are not the maingoals of the sites.

ARTICLE IN PRESS

Table 4

Web assessment index (second group)

Categories Weights Categories Weights

ACCESSIBILITY 20 NAVIGABILITY 20

Presence in search engines 5 Site map 15

Popularity 15 Keyword search function 5

SPEED 10

Site size (bytes) 10

CONTENTS QUALITY 50

Informational content 30 Communicational content 20

Company background 6 Contact e-mail 8

Product/service description 8 Contact telephone or address 6

Daily news highlights 4 Receive news by e-mail 2

Financial information 2 Free e-mail service 1

Job opportunities 2 Personalization capacity 1

Check order/shipment status 1 Entertainment elements 2

Search for dealers, agents, stores, etc. 4

External links 1

Link to firm divisions 2

TOTAL 100

10Different factors and weighting schemes would be necessary if we wanted a better understanding of the special

characteristics of each industry. Depending on the sector it is possible to rate each category (accessibility, speed,

navigability and content) in such a way as to capture their relative importance, but that is not the goal of our study.


Table 5 shows the results of the study by industry. Industries have been classified in the twogroups previously defined in order to reflect the strategic decision of the industry to sell or not theproduct using Internet. The table does not show significant differences between the two groups.In order to complete our study we have analyzed the correlation between the different factors

used in our index. We measure the degree of relationship present between the four factors used:speed, accessibility, navigability and content. Fig. 711 shows the significant correlations identified.Site content is significantly related with accessibility and with navigability. Both correlationssuggest that the best designed sites are richer in content, easily accessible and navigable. It isinteresting to note that navigability is also related with accessibility, so easily accessible sites arealso easily navigable.The negative relationship between speed and content was also expected. More complex sites,

with more informational, communicational and transactional elements, are usually more slow.

ARTICLE IN PRESS

Table 5

WAI values: first and second group

Industry (number of firms) Speed Accessibility Navigability Content Index

First group

Transport (12) 67.83 41.33 83.33 72.83 68.13

Travel and tourism (5) 73.69 32.45 75 65.6 61.66

Mass media (3) 62.27 44.68 83.33 55.33 59.49

Electric and electronic material (4) 76.17 26.41 75 53.5 54.65

Petrol (5) 73.63 22.13 85 50 53.78

Textile and shoes (3) 73.34 9.47 75 52 50.22

Trade (16) 76.88 21.21 75 45.62 49.74

Subtotal 72.35 26.85 77.67 54.92 55.61

Second group

Construction (7) 78.73 24.06 85.71 71.14 65.40

Iron & steel (4) 87.90 22.18 75 69.5 62.98

Electric energy, water and gas (6) 75.99 28.22 83.33 65.67 62.74

Paper and graphic arts (2) 86.87 6.45 87.5 68 61.48

Others (5) 78.79 23.93 75 62.4 58.87

Transport equipment (18) 69.91 24.39 75 60.66 57.20

Machinery and mechanical equipment (2) 81.54 23.15 87.5 52 56.29

Chemistry and pharmacy (6) 69.68 19.93 70.83 62.33 56.29

Food, drinks and tobacco (13) 78.75 17.98 65.38 56.15 52.63

Business service (9) 71.07 16.95 72.22 44.11 47.50

Metallic and mineral products (2) 81.89 25.25 37.5 33 37.24

Subtotal 76.27 22.33 74.24 61.09 57.49

Total 74.47 24.40 75.81 64.10 56.62

11� = Significant at 95%, �� = Significant at 99%.


However, this situation can be easily avoided distributing the content in different pages andkeeping the home page as simple as possible. One way to handle this trade-off (speed versuscontent) is to plan intra-site links so that certain features are self-contained. Thus, a userinterested in a video demonstration of the product or in a virtual visit to the firm could click on aspecific link, learn how long the download will take and avoid being slowed down if he/she is notinterested in this feature.On the other hand, there are not significant relationships between speed and accessibility and

between speed and navigability (a ¼ 0:05).

4. Discussion and conclusions

The primary goal of this study is to clarify the need for organizations to evaluate their web sites.The Web Assessment Index (WAI) provides an integrated approach for evaluating the potentialof organizational web sites, allowing researchers and managers to compare attributes andcomponents of Internet sites in a sector so as to determine the drawbacks and opportunities.The main challenge in the elaboration of the index was to avoid subjective factors, which have

been predominant in previous assessment tools. Our index is based on four broad categories: sitecontent, speed, accessibility and navigability.The validity of the index has been tested in our evaluation of the web sites, belonging to the top

200 Spanish firms. The results of this analysis have demonstrated the high flexibility of the WAIand have detected the main weaknesses of the sites assessed.Transport, and Travel & tourism industries show the highest scores in the WAI, thus they are

the Spanish industries which are implementing better web policies, also showing a greaterunderstanding of how Internet can enhance their company activities.It is possible to extend the application of WAI by adjusting the manner in which the different

factors are scored. Depending on the sector it is possible to rate each category (accessibility, speed,navigability and content) in such a way as to capture their relative importance. So, the suggestedmethodology is equally valuable to analyze sites belonging to commercial, educational and non-profitmaking organizations.Finally, we have analyzed the results statistically to identify the correlations among factors, in

order to help firms of each industry to decide the factors they should invest upon.

ARTICLE IN PRESS

Fig. 7. Bivariated correlations between factors.


There are several limitations for this analysis. First, all the data in the survey was collected froma single visit to each site at one point in time, despite the fact that the web is a highly dynamic andchangeable medium. Similar studies at different times are likely to show different results. Thisevaluation over time will also shed some light on whether there is greater divergence orconvergence of web activities.A second concern was the subjective nature of factor weightings, which, although are based on

the results of previous studies and personal interviews with Internet experts, introducessubjectivity into our analysis.Another limitation is that the results cannot be general to all businesses. Although large firms

generally provide leadership in their usage of information technology, some differences do existbetween small and large businesses, especially when using the Internet to compete in the marketplace. Therefore, careful interpretation of the results must be made, especially as to theirapplicability to small businesses.A subsequent strength of this study lies in its foundation for future research. Having identified

the critical categories and factors in the WAI, the next stage is to test this tool in different sectorsand measure the relationship between the index value and organization success. Further researchin this important area is currently under way.As further information on web design and usability methods becomes available, the assessment

index presented here can be refined into an empirically validated toolkit for the design of usablecorporate sites. The proposed index indicates a possible method for evaluating web sites andmaking a comprehensive analysis of the usage of the new medium. To further validate theproposed framework and assess electronic commerce applications, the model needs to be appliedto other company sites, but especially to a business-to-business case, as this is expected to be themajor growth area in electronic commerce in the years to come.

References

Bauer, C., & Scharl, A. (2000). Quantitative evaluation of web site content and structure. Internet Research: Electronic

Networking Applications and Policy, 10(1), 31–43.

Buenadicha, M., Chamorro, A., Miranda, F. J., & Gonz!alez, O. R. (2001). A new Web assessment index: Spanish

universities analysis. Internet Research: Electronic Networking and Applications Policy, 11(3), 226–234.

Christopher, D. (1997). Sites lacking good information design. www.netb2b.com, article 1, September. 14 (pp. 1–3).

Evans, J. R., & King, V. E. (1999). Business-to-business marketing and the World Wide Web: Planning, managing and

assessing web sites. Industrial Marketing Management, 28, 343–358.

Gromov, G. R. (2000). http://www.internetvalley.com/intvalstat.html.

Huizingh, E. (2000). The content and design of web sites: An empirical study. Information & Management, 37,

123–134.

Ho, J. (1997). Evaluating the World Wide Web: A global study of commercial sites. Journal of Computer Mediated

Communication, 1(3).

Liu, C., Arnett, K., Capella, L., & Beatty, R. (1997). Web sites of the fortune 500 companies: Facing customers through

home pages. Information and Management, 31, 335–345.

Murray, M. (1997). Evaluating web impact-the death of the highway metaphor. Direct Marketing, 59, 36–39.

OCLC Research Project: Measures Scope of the Web (1999). http://www.oclc.org.

Olsina, L., Godoy, D., Lafuente, G. J., & Rossi, G. (1999). Specifying quality characteristics and attributes for websites.

First ICSE workshop on web engineering, Los Angeles USA.

ARTICLE IN PRESS


http://www.netb2b.com

http://www.internetvalley.com/intvalstat.html

http://www.oclc.org

Selz, D., & Schubert, P. (1997). Web assessment: A model for the evaluation and the assessment of successful electronic

commerce applications. Electronic Markets, 7(3), 46–48.

Simeon, R. (1999). Evaluating domestic and international web-site strategies. Internet Research: Electronic Networking

Applications and Policy, 9(4), 297–308.

Young, D., & Benamati, J. (2000). Difference in public web sites: The current state of large US firms. Journal of

Electronic Commerce Research, 1(3), 94–105.

ARTICLE IN PRESS


Technology

Evaluation Of Commercial Website