164 Family Relations

A Process Evaluation of a Website for Family Life Educators*Robert Hughes, Jr.**

Effective professional development is critical to maintaining high-quality family life education. Most professional development continuesto be conducted through workshops, newsletters, and other traditional ways. The growth of information technology provides an im-portant new teaching capacity for professional development. This article explores the development of a website for family life educators,examines procedures for obtaining formative evaluation data about websites, and considers the implications of this work for developingmore effective professional development experiences for family life educators.

Professional development is critical to maintaining high-quality family life education programs. The NationalCouncil on Family Relations (NCFR) has long been com-

mitted to fostering competent family life education by publishinga journal devoted to applied work in human development andfamily science and by offering professional development oppor-tunities at their annual meetings. A centerpiece of these effortsis the recently developed certificate program for family life ed-ucators to promote continued professional education (Czaplewski& Jorgensen, 1993).

At present, most professional development opportunities forfamily life educators and many other professionals are conductedby face-to-face workshops, by newsletters, or by other traditionaleducational approaches. Nevertheless, recent surveys of profes-sional development indicate that this may be changing. For ex-ample, in a recent review of professional development for theAmerican Society for Training and Development, Bassi and VanBuren (1999) report that training in classrooms is declining andthat training using learning technologies has been increasing.Across a wide range of training organizations, learning technol-ogies including CD-ROM and the World Wide Web (WWW)were used in about 9% of the training in 1997 and were antici-pated to be used in about 23% by 2000. Among companies andorganizations that were defined as ‘‘leading-edge organizations,’’it was reported that by the year 2000, 93% expect to use CD-ROM, 70% will use the WWW, and 33% expect to use virtualreality. Bassi, Cheney, and Lewis (1998) concluded that ‘‘al-though classroom training will remain a necessary vehicle forcreating learning, it will increasingly be augmented with, and insome cases replaced by, electronic means of learning’’ (p. 72).

Information technology offers an important new capacity tofoster professional development for family life educators. For sev-eral years family life educators have been using e-mail. One ex-ample is FAMNET, a discussion list (listserv) that facilitates com-munication among a variety of professionals interested in familylife education. There are many other electronic discussion listsdesigned to allow professionals to exchange information about

*The website described in this article was created while the author was at The OhioState University (OSU). The work was supported by OSU Extension, and many faculty andstudents were involved in the development of the website, including David Andrews, AaronBeckstrom, Matt Dawson, Roy Fish, Thelma Hurston, Eunjee Joo, Jacqueline Kirby, LaurieMeschke, Ben Perry, Chris St. Clair, Dalauna Sutton, Steve Toepfer, Kerrie Ulishney, andKim Whaley.

**Address correspondence to: Robert Hughes, Jr., Ph.D., Department of Human De-velopment & Family Studies, College of Human Environmental Sciences, 110 Gwynn Hall,University of Missouri, Columbia, MO 65211-7700; (573) 882-7828; FAX (573) 884-5768;e-mail: hughesro@missouri.edu

Key Words: family life education, information technology, professional development,World Wide Web.

(Family Relations, 2001, 50, 164–170)

more specific interests such as child care, violence prevention,marriage, and so forth. Another example of the use of informationtechnology for professional development is the use of e-mail andthe WWW for course work and professional development. Forexample, Hughes (1996) conducted an e-mail in-service coursefor county-based extension educators in which participants re-ceived weekly e-mail lectures and then shared their insights andobservations with other electronic participants. E-mail and web-sites have also been used to support satellite video conferencesby providing a means for participants to interact with conferencepresenters and to obtain additional information (e.g., Better KidCare, http://www.psu.edu/dept/BETTERKIDCARE/).

The development of websites creates an important additionalcapacity for offering professional development. Many govern-ment agencies, professional associations (e.g., NCFR, AmericanPsychological Association, National Association for the Educa-tion of Young Children, and many others), and national resourcecenters (e.g., National Clearinghouse for Alcohol and Drug In-formation, National Clearinghouse on Families and Youth, etc.)have made a wide range of information available on the web. Itis increasingly possible for family life educators to obtain a widevariety of information through the Internet.

Despite these pioneering efforts to use information technol-ogy in professional development, there is still much to be learnedabout the design and evaluation of information tools. At present,most attempts to teach using the WWW have done little morethan convert existing classroom practices into web-based mate-rials. Indeed, some of the most popular courseware systems (e.g.,WebCT) are designed to facilitate traditional classroom educa-tional procedures involving lecture, discussion, and multiple-choice testing capacities (for an exception to this pattern, seeSmith, 1999). An alternative is to develop models of teachingthat are not bound by typical classroom or professional devel-opment procedures, but that take advantage of information tech-nology capabilities. As these models are created, it is importantto examine the ways in which websites for professionals are bestdesigned and to collect data about the effectiveness of these de-signs.

In terms of the evaluation of websites, we are at the earlystages of program design, and we need information about whatJacobs (1988) has called ‘‘program monitoring.’’ That is, weneed to ask questions about what participants in family life ed-ucation professional development websites are doing and whattheir reaction is to this experience. These formative evaluationdata will inform program design issues and help designers un-derstand the nature of these websites as educational experiencesso we can address questions about what people are learning asthey participate in these experiences.

The development and implementation of a website for fam-ily life educators is explored here. In this article the conceptualdesign and process evaluation data are described. This is fol-

2001, Vol. 50, No. 2 165

Table 1Indicators of Website Activity

Indicator Name Definition

Activity informationHit (access) The number of times a file is requested. This

includes all types of files, including graph-ics.

Page view The number of times a webpage (text file) isrequested.

Visitor informationUnique visitor The number of unique remote computers

(identified by domain names or Internetprovider [IP] addresses) accessing a web-site.

Repeat visitor The number of repeat remote computers(identified by domain names or IP address-es) accessing a website.

Origins Many domain names provide a limited coderegarding the type of location from whicha visitor originated. The most common are‘‘com’’ (commercial), ‘‘edu’’ (education),and ‘‘org’’ (organization). Visitors fromoutside of the United States have a codefor their country.

Visitor interaction informationVisit A visit is a period of time in which a visitor

is requesting webpages. A visit can be asshort as a request for one webpage or aslong as requests for every webpage.

Referrals This is the website (actual webpage) that pre-ceded a visitor’s entry into the currentwebsite.

Page views per visitor The number of page views for each uniquevisitor.

Key words When a search engine is used to locate awebpage, the key word used in the searchis identified.

Trails The path through a website as followed by avisitor.

lowed by a discussion of ways in which website monitoring datahave influenced instructional design and how more effective pro-fessional development experiences for family life education canbe developed.

Development of a Website for Family LifeEducators

In 1995 a website for family life educators was created(http://www.hec.ohio-state.edu/famlife/). Initially, the primarypurpose of this website was to provide information updates tocounty extension staff. Initially, there was little rationale forthe organization of the website. The primary content of thewebsite was lists of educational resources and hypertext linksto other major sources of information about children and fam-ilies. In its earliest form, the entire website had less than adozen pages of information. The next major development wasthe conversion of a printed newsletter for family life educatorsinto a web version of the newsletter. This quarterly newsletterwas the primary content of the website for the first year.

As time went on, a conceptual model for the website wasdeveloped. The development of the model for this website beganwith the question: What are the instructional functions that sup-port family life educators? Five major functions were identified:(a) communicating new research or practice information aboutsignificant child and family issues, (b) providing descriptions ofuseful education resources or the materials themselves, (c) guid-ing the evaluation of programs, (d) identifying or providing in-formation about useful professional development opportunities,and (e) updating family life educators about information tech-nology. In short, the website was conceptualized as a web-based‘‘person’’ who provided support for practicing family life edu-cators. This model resulted in organizing the website aroundeach of these five functions so that a family life educator couldfind helpful content in each of these areas.

In 1998, a topical index was added so that a family lifeeducator interested in the topic of ‘‘divorce’’ or ‘‘families,’’could find all the resources on the site including educationalmaterials, professional development, and evaluation related to aparticular topic area. Adding this new dimension resulted in or-ganizing the website by topics as well as by function. Becausethere was both a topical and a functional index to the website,professionals could examine the contents either by function (i.e.,What are all the evaluation resources in the site?) or by topic(i.e., What are all the contents [educational materials, profes-sional development opportunities, etc.] in the website related to‘‘divorce’’?).

A core section in the website was a newsletter for familylife educators. Although this newsletter was designed for print,a web-based version was also created. The newsletter had foursections: a research update about a major human developmentor family topic, information about evaluation, practical sugges-tions for program delivery, and an annotated list of educationalresources. The newsletters were organized by issue and by sec-tion so that a family life educator could find a particular issueor all the articles on evaluation, practical suggestions, and soforth. The website also included information about how to re-ceive the newsletter by e-mail, and visitors could download aportable document file (pdf) of the newsletter that prints on al-most all printers.

Monitoring Websites: Formative Evaluation

Like all education materials, it is important to get feedbackabout the success of the instructional design of a website. Thereare many methods that can be used to obtain information abouteducational materials, including focus groups of users, reviewsby experts, and various types of pilot testing (Smith & Ragan,1993). All of these methods are important to understanding thedesign of websites, but the WWW provides some additional op-portunities for analyzing utilization and instructional design. Be-cause the WWW is based on transactions between computers,the data resulting from these transactions can be used as a sourceto examine how the website is used. In the following sectionthere is a description of the major types of information that canbe obtained about a website, and this information is illustratedwith examples from this family life educator website.

Website IndicatorsA summary of the major website activity indicators can be

found in Table 1. These definitions are based on the standardsdeveloped by the Media Measurement Task Force (1997) of theInternet Advertising Bureau. In general, the website indicatorsprovide information about the basic activity on the website, thevisitors to the website, and the interactions of the visitors withthe website. In the popular press, the most common type of in-formation that is mentioned about all types of websites reflects

166 Family Relations

Figure 1. Total Website Page Views

Figure 2. Number of Unique Visitors to Website

the activity on the website—hits (accesses) or page views. Al-though this activity information provides a rough measure of theamount of general use of the website, it provides little helpfulinformation regarding the design. Another problem is that thesemeasures are often misunderstood or misinterpreted. For exam-ple, the number of ‘‘hits’’ on a website includes both graphicsand text files and therefore should not be viewed as a goodmeasure of the amount of time information is viewed. For familylife education websites, a measure of ‘‘page views’’ is a moreappropriate way to monitor the website, because the textual ma-terial (and not the images) is the content that visitors are seeking.However, both hits and page views can overestimate activity be-cause of the use of ‘‘bots’’ (automated computer programs thatsearch websites), which frequently access all or many pageswithin a website, thus generating many hits and page views. Ingeneral, these activity indicators are fairly crude indicators.

Some limited information can be obtained about the visitorsto a website, including the numbers of unique visitors and repeatvisitors and the origin of each visitor. Information about thenumber of unique visitors and repeat visitors is probably a betterindicator of whether a website is attracting new viewers or asteady group of viewers. For the website developer, this infor-mation could be collected about the website as a whole or aboutvarious parts of the website to determine the relative use of dif-ferent sections. Information about the origin of visitors has lim-

ited value for visitors from the United States, but this informa-tion can be of some value for knowing what international visitorsare using the site.

The most useful information that can be obtained regardingwebsites is about the patterns of interaction by visitors to thesite. By observing the length of time that visitors remain engagedin the site and the average number of pages viewed, it is possibleto monitor the degree to which the website is engaging visitors.Again, this same information can be obtained about specific sec-tions of the website. By looking at referrals and key words, thewebsite developer can obtain information about the type of in-formation visitors are looking for and how visitors are findingthe site. The pattern of trails through a website provides the mosthelpful information about the website, because it records the se-quence of pages that visitors follow in exploring a website. Fromthis information, a designer can either identify common paths ormonitor new paths when new organizational frameworks are in-troduced. The following section explores how these indicatorscan be used to understand the design and development of a fam-ily life education website.

Monitoring a Website for Family Life Educators

The use of this website for family life educators has beenmonitored since fall 1995. The first complete month of data wascollected in October 1995. Since the beginning, the followingwebsite indicators have been routinely monitored: timing of re-quests, page views, unique visitors, and origins of visitors. InJuly 1997 trails and visits were added. In January 1999 keywords and referrals began to be collected. In all cases, the dataanalyzed were based on logs generated by a Windows NT server.The web analysis program that has been used since January 1997is Wusage, developed by Boutell, Incorporated (1999). The web-site indicators that were added reflect advances in this softwareover time.

Page Views Over TimeIn the first month (October 1995) of monitoring of the web-

site, there were 464 page views of the 26 total webpages at thissite. Since that time, there has been substantial growth in theoverall size of the website in terms of both number of webpages(about 1,000 pages) and amount of activity (over 40,000 pageviews in May 1999) with this material. Figure 1 graphs the num-ber of page views since the beginning of the website. On theone hand, this graph indicates that there have been increasedrequests for webpages; however, this has also been a period ofrapid growth of Internet access. Thus, these data may say lessabout this website and more about increased access to the Inter-net. In order to understand the overall implications of websiteactivity, it would be helpful to have some other appropriate web-sites for comparison. As additional websites are developed, sim-ilar types of data should be collected to enhance possible com-parisons and to understand the implications of different strategiesfor teaching and presenting information.

Visitors Over TimeAnother type of information is the number of unique visitors

who have accessed the website over time. In this case, uniquevisitors were calculated by week. This means that if a personvisits the site more than once per week, she is only counted once;however, if she visits each week, she will be counted in eachweek that she makes a request. Figure 2 shows that the number

2001, Vol. 50, No. 2 167

Figure 3. Number of Page Views per Visit

Table 2Percentage of Visitors From Various Locations

Year

Locations

Education Commercial NetworkOrganiza-

tionGovern-

mentInter-

national

1996199719981999a

64321614

17306770

10241111

1211

1111

71164

a Data were lost because of a server malfunction, so the percentages for this yearare based on only 2 months.

of unique visitors has changed over time. For convenience ofgraphing, these data were aggregated by month. There were few-er than 100 visitors per month when the website was firstlaunched. In April 1999 there were about 8,000 monthly visitors.These numbers again reflect both the growing contents of thiswebsite and the growth in access to the Internet.

In a marketing analysis of the WWW, there has been a de-bate about the right website indicator to use in monitoring web-sites (cf. Winnet, 1998). The two most common indicators havebeen page views and unique visitors. However, it might also beuseful to monitor the number of repeat visitors. The softwareused to analyze this website does not make this indicator avail-able, but the information is available in the server data logs tocalculate this information.

Engagement of VisitorsAlthough counting page views and visitors is a good way

to track the overall changes in the website, there are other in-dicators that are more helpful for informing instructional design.The most crude indicator is calculating the number of webpagesrequested by each visitor. Roughly, this provides some measureof the extent to which visitors are engaged by what they findand continue to request additional information. Of course, thiscould also reflect poor design, such that visitors must continueto search to find what they are seeking. Nevertheless, this num-ber has served as a crude indicator of the degree to which thewebsite engages visitors.

In Figure 3, the number of webpages accessed by a visitoris tracked over time. The pattern in this graph is quite differentfrom that presented in Figures 1 and 2. There are a number offactors that influence this change. First, the general decline inthe mean number of accesses, to about two per visitor, is real.In part, this reflects a high percentage of visitors who acciden-tally access this site and have no interest in the content. This ispartly a function of the untidy way in which search engines workand the lack of order of the WWW itself. A more helpful indi-cator might be one that excluded visitors who only accessed onewebpage and compute this engagement index on the remainingvisitors.

Another factor that distorts this indicator is the presence ofbots that access websites. A bot is an automated web searchmechanism designed by the people who maintain web searchengines (e.g., Yahoo, Altavista, etc.) to gather new sources of

information. These bots routinely access every page on a web-site. Since September 1998, there has been a dramatic increasein the activity of bots on this website. The large number of ac-cesses by these bot visitors has increased the engagement indexin ways that do not correctly reflect ordinary visitors. Neverthe-less, this index still serves as a crude indicator of engagement.The results of these bot searches increase the visibility of thiswebsite through the use of search engines. Again, a more helpfulindicator for assisting in instructional design would be one thatexcludes the activity of bots.

Origins of VisitorsDomain names provide a limited amount of information

about the locations from which visitors originate. Domain namesprovide some general ability to determine whether the peopleusing a website are from educational, governmental, organiza-tional, or commercial providers. Also, it is possible to identifythe source of visitors from countries outside of the United States.In Table 2 the percentage of different types of visitors is pre-sented for 3 years of the website. In 1996 the largest percentageof visitors was from educational institutions (64%), but by 1998the largest percentage of visitors was from commercial sites. Ingeneral, this reflects the changing access to the Internet duringthis period. With the introduction of commercial providers suchas America Online, more people without ties to educational in-stitutions have gained access. Table 2 also shows that the ma-jority of the users were from within the United States, with onlya small percentage coming from international locations.

Website Indicators of a Newsletter for FamilyLife Educators

In the previous discussion, the focus of analysis has beenon the activity of the overall website; however, the same analysiswas conducted specifically on a selected part of the website. Inthis case, there has been special attention to the activity on theweb version of the newsletter. In the following section, infor-mation is provided about the most frequently accessed webpages,key words, and trails. Each of these indicators is discussed, asis how this information can be used to inform instructional de-sign.

Access of PagesIn addition to looking at the overall number of page views

on the total website, it is possible to track the number of timeseach webpage is requested or accessed. The newsletter includesboth index pages for each topical issue of the newsletter andwebpages for each article. By tracking the number of times eachindex page and each article is accessed, it is possible to identify

168 Family Relations

Table 3Frequently Accessed Documents in the Newsletter (April 1999)

Type of Document Number of Page Views

Index pagesMain newsletter indexChild abuse preventionAggression and peer rejectionAdolescent pregnancyPoverty and stressDivorce education programsAdolescent problem behaviorsAdolescent sexualitySingle parents and poverty

466241234209197107914745

Newsletter articlesAdolescent behavior problemsPoverty and single parentsAggression and peer rejection

20813933

Table 4Three Examples of Trails Through the Newsletter Website (April 1999)

TrailNumber Document Sequence Frequency

1 1. welfare.infor.inter.net/teen2. Adolescent Pregnancy Prevention Evaluation3. Adolescent Pregnancy Prevention (pdf file) 14

2 1. ink.yahoo.com/bin/query2. Aggression and peer rejection index page3. Aggression and peer rejection article 4

3 1. www.agnr.umd.edu. . . .2. Welfare-to-work issues3. Welfare-to-work transition article 3

Note: pdf 5 portable document file.

which topics are most frequently accessed. Table 3 provides asample of the most frequently accessed pages during a recentmonth. First, a ranking of the topical issue that was most fre-quently accessed is presented, followed by the most commonlyaccessed article. Adolescent issues were frequently accessed dur-ing this particular month. (Note that late in this month the shoot-ings occurred at Columbine High School in Colorado.) By ex-amining what material is being accessed, it is possible to addother material on the topic or to modify the design of webpagesso it is easy to access additional material on this issue from thesefrequently accessed pages.

Key Words

When a visitor uses key words to locate information on theWWW, this is recorded by the server. Counting key words pro-vides an indication about the kinds of topics that are of interestto the visitors on the website. This type of analysis has beendone only in recent months for the newsletter. For example, inthe month of April 1999 the key word that was most frequentlyused to find material at this website was adolescent pregnancy.Other frequent key words were aggression and peer rejection,which may also be linked to the specific events of that month.For the most part, the key words match content in the newsletter.

There a number of ways to use key words to increase visi-tors’ ability to find information on a website. One way to in-crease visitors’ ability to find webpages is to include a series ofkey words in the HTML coding (these are called metatags).These tags are seen only by the search engines and are used toidentify more quickly the content of a webpage. Initially, the keywords for these metatags would be selected by the webpage de-veloper, but key words used by visitors might be added if theywere different from those selected by the developer. Anotherapproach is to organize the website navigation so that all of thematerial related to commonly used key words is easily foundfrom a variety of locations within the website. At present, thenavigation of the newsletter is by type of content—evaluation,research summaries, and so forth. Soon, the design will expandto include a topical list (e.g., divorce, parenting, adolescence,etc.) that has been generated as a result of the list of key wordsthat were used by visitors. Making changes in the design of thewebsite based on key words has improved the educational designof the website.

Trails

As a visitor travels from one webpage to another through awebsite, this path can be tracked and is referred to as a trail.These trails can be analyzed to identify the most common trails.To the extent possible, this information can provide the designerwith an analysis of how people actually use the site regardlessof the intended organizational structure. In regards to the news-letter, it is possible to examine whether visitors look at all thearticles in a single issue, look at all the articles on a particulartopic, or use some other strategy. Again, like key words, ex-amining trails provides the ability to assist visitors to more easilynavigate or to track changes in visitor behavior as new organi-zational frameworks are introduced. Table 4 gives the three mostfrequently used trails for a recent month. To put this in context,it is important to recognize that there were 1,666 unique trailscreated during the month. Also of interest is the fact that duringthis month slightly over half of the top trails each week (21 of40) were begun with the results of a search engine (e.g.,InfoSeek, Yahoo, and America Online). This indicates that vis-itors are using search engines to find this type of material andthat the search engines have this website cataloged in a way thatvisitors can locate it. However, in this particular month, the mostcommon trail began with a welfare information site that provideddirect links to material about teen pregnancy on our site. Therewere also trails to an article about peer rejection and about thewelfare-to-work transition for families. At present, even thesefrequent trails are followed by a small number of people, indi-cating that the paths taken through the site are not similar acrossvisitors. Trails can also be examined as design changes are im-plemented. One futuristic learning scenario that has been sug-gested is that skilled learners would report the trails they tookthrough a range of digital information and would record theirobservations and insights. These trails and notes would be madeavailable to more novice learners so they could follow a similarpath and gain insights from more skilled people.

Implications for Professional Development ofFamily Life Educators

The primary purpose of this article was to provide infor-mation about the design of WWW-based professional develop-ment experiences for family life educators and to consider pro-cess evaluation data that would make these efforts more effec-tive. The following section considers ways in which web-basedprofessional development experiences can be improved.

2001, Vol. 50, No. 2 169

Instructional DesignThe basic conceptual design, using professional develop-

ment functions and topics as the two primary organizational di-mensions, provides one useful way of conceptualizing websitedesigns. Others providing support to family life educators willobviously identify more functions or more specific functions. Forexample, the function of providing evaluation assistance mightbe organized into more specific functions, such as providing ba-sic evaluation strategies, illustrating measurement tools, showingtypes of evaluation reports, and so forth. There are undoubtedlymany other ways to organize websites for family life educators.As we explore the use of web-based professional development,it is critical to design models that are not bound by other edu-cational delivery methods.

Since this website was launched, there have been many newbooks that offer ideas about how to apply fundamental instruc-tional design principles to websites (e.g., Khan, 1997; Palloff &Pratt, 1999). It is important for website developers to considerhow basic teaching and learning strategies apply or fail to applyto the WWW. This includes careful consideration of the needsand situation of the audience (or learners) of a website and ananalysis of the material to be taught so the website can be de-signed to facilitate learning. Additionally, there is a growingbody of knowledge about the usability of websites (e.g., Neilson,1999a) that considers the nature of providing information andusing information technology as well as offering suggestions formore specific needs of users with disabilities and users frominternational locations. Continued development of effective web-sites for family life educators needs to incorporate this knowl-edge.

The weakest aspect of this website for family life educatorsis that it does not include many opportunities for the visitor tointeract with the material or with other family life educators. Theprimary way that information is delivered in this site is similarto the way that it would be presented on a printed page. Aswebsites for family life educators evolve, it is important to takeadvantage of the interactive capabilities of the Internet. In otherwords, visitors to the site will need to be able to add their owncomments to the material presented or add their own recom-mendations for additional resources. In designing these new in-teractive opportunities, it is important to invite discussion be-tween the developers of this material and to encourage discus-sion between family life educators who visit the website. In thisway, visitors not only can gain new ideas, but also may be ableto share their ideas with others.

Another way to develop interactivity is to design teachingtools that provide guidance to visitors about a topic or issue. Anexcellent example of this type of tool has been developed byDeBord (1997). She created an evaluation design tool that guidesvisitors through the construction of an evaluation design for afamily life education program. In this case visitors not only learnabout evaluation design, but also have an opportunity to developa useful product for their own future use.

Process DataThe process data that can be generated to monitor a website

provide valuable information for understanding the overall op-eration of a website and insights into improving the design. Bytracking where visitors are using the website, new informationcan be added, or new navigational options can be created toincrease the visitors’ ability to find other information on a similar

topic. Unlike most other forms of training materials, the instruc-tional design of web-based materials can be modified to matchthe learner’s interests or preferences. The website process infor-mation offers a data-based system for informing design deci-sions. Also, as design changes are introduced, visitors’ behavioron the site can be monitored to observe whether the new changesalter activity on the website.

There are many obvious ways in which these process dataare limited. As noted previously, much of the data generatedprovide only general ideas about activities and visitors. In ad-dition to these problems, a significant issue is that despite theability to track a visitor in quite detailed ways, little is knownabout the visitor’s personal characteristics. Although this websitewas designed for family life educators and other professionalsinterested in human development and family issues, it is notknown who the visitors are and whether they are professionalsor lay people. Not knowing exactly who the audience is createsa substantial problem for instructional designers. In other words,the pattern of activity, the trails followed, and so forth may bequite different for the professional and the lay person or fordifferent types of professionals. Yet all data are treated alike. Ofcourse, it would be ideal to know even more about the visitorsto the website in terms of their professional work, their back-grounds, and other demographic information that would assist inunderstanding who did and did not find this work helpful. Onesolution to this problem is to ask or require users to complete aregistration form prior to being able to gain access to the website.However, an important consideration is the degree to which thiswould limit the people who would access this site (see Neilson,1999b, for a discussion of this issue). The extent to which theregistration process was brief and anonymous would influencethe effects of the procedure.

Although these website process data can be informative,they should not be viewed as the sole method of obtaining feed-back about the usefulness or helpfulness of a website. Websitedesigners should continue to seek feedback from experts aboutthe effectiveness of the design and use focus groups and othermethods to ascertain information about what is helpful and whatworks. Likewise, it would be useful to provide numerous op-portunities within the website itself for users to provide feedbackas to whether they found the sought-after information.

Learning Outcomes for WebsitesAnother problem that is mentioned often in regards to web-

site delivery of information is whether people are truly learninganything with this delivery method. This issue is not unique toweb-based teaching; it is also an issue for all type of methodsof delivering professional development, including newsletters,journals, and workshops. It is important to note that the proce-dures described here do not solve this problem. These proceduresare only process measures; they are not outcome measures. Thenext logical step is to find ways to obtain additional measuresthat address outcomes. In part, additional measures should in-clude client satisfaction surveys, much like our current teachingevaluation procedures, which would provide feedback aboutwhether visitors found the information useful or how satisfiedthey were with their experience with the website. Family lifeeducators could also be asked questions about the extent towhich they gained knowledge about a topic, or they could beasked questions that require the application of information tohypothetical problems.

In the long run, we need to obtain information about chang-

170 Family Relations

es in behavior or practice as a result of web-based professionaldevelopment. This is an issue not just for web-based training,but also for all types of work with professionals. In more tra-ditional training, the preferred methods would be tests of knowl-edge or follow-up assessments of practice change. Both of thesemethods could be adapted for use via the Internet and wouldrequire some type of registration by participants. As we developmore web-based professional development opportunities, weneed to carefully examine real changes in knowledge and prac-tice. This will require the use of experimental designs that com-pare alternative professional development experiences.

Conclusion

The Internet and the WWW represent dramatically newtools for use in the professional development of family life ed-ucators. It is important to develop effective web-based teachingdesigns that allow family life educators to use information tech-nology to gain knowledge and skills. This article provides someinsights into the strengths and weaknesses of one website forthis purpose, but further website professional development ex-periences are needed. The process data presented here are a use-ful first step in understanding web-based delivery. However,these data are limited, because outcome evaluation is needed.The future strength of family life education is dependent on con-tinued exploration of the Internet as a tool for helping profes-sionals learn.

References

Bassi, L. J., Cheney, S., & Lewis, E. (1998). Trends in workplace learning:Supply and demand in interesting times. Training and Development, 52, 51–75.

Bassi, L. J., & Van Buren, M. E. (1999). The 1999 ASTD State of the IndustryReport. Training and Development, 53, 2–27.

Boutell, Incorporated. (1999). Wusage. Retrieved September 23, 1999 from theWorld Wide Web: http://www.boutell.com/wusage/

Czaplewski, M. J., & Jorgensen, S. R. (1993). The professionalization of familylife education. In M. E. Arcus, J. D. Schwaneveldt, & J. J. Moss (Eds.),Handbook of family life education (Vol. 1, pp. 51–75). Newbury Park, CA:Sage.

DeBord, K. (1997). National network for family resiliency interactive programevaluation development site. Retrieved September 23, 1999 from the WorldWide Web: http://www.ces.ncsu.edu/depts/cs/nnfr/

Hughes, R., Jr. (1996). Children and divorce: An Internet in-service experiencefor professionals. Retrieved October 4, 1999 from the World Wide Web: http://www.hec.ohio-state.edu/famlife/divorce/childins/index.htm

Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context and imple-mentation. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs(pp. 37–68). New York: Aldine de Gruyter.

Khan, B. H. (1997). Web-based instruction. Englewood Cliffs, NJ: EducationalTechnology Publications.

Media Measurement Task Force. (1997). Metrics and methodology. RetrievedJune 28, 1999 from the World Wide Web: http://www.iab.net/advertise/metricscource.html

Neilson, J. (1999a). Designing web usability. Indianapolis, IN: New Riders.Neilson, J. (1999b). Web research: Believe the data. Retrieved September 23,

1999 from the World Wide Web: http://www.useit.com/alertbox/990711.htmlPalloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace:

Effective strategies for the online classroom. San Francisco: Jossey-Bass.Smith, C. A. (1999). The WonderWise Parent: Courses. Retrieved February 28,

2000 from the World Wide Web: http://www.ksu.edu/wwparent/courses/Smith, P. L., & Ragan, T. J. (1993). Instructional design. New York: Macmillan.Winnet, B. (1998). Troubles with tracking. Retrieved June 28, 1999 from the

World Wide Web: http://www.hotwired.com/webmonkey/98/25/index4a.html? tw5e-business

Robert Hughes, Jr. is Professor in the Department of HumanDevelopment & Family Studies and Associate Dean of Outreach& Extension at the University of Missouri, Columbia, MO.

Received 10-19-99Revised & Resubmitted 3-7-00Second Revise & Resubmit 10-25-00Accepted 11-21-00