Embed Size (px)
Citation preview
Analysis of web-based tutorials created byacademic librariesby Marta Somoza-Fernández and Ernest AbadalAvailable online 20 February 2009
This paper aims to analyse thecharacteristics of tutorials created by
academic libraries. It evaluates a sample of180 tutorials by applying thirty basic
indicators referring to generalcharacteristics, content, teaching
methodology, usability and technology.The general conclusion is that most of
the tutorials are at an early stageof development.
Marta Somoza-Fernández, Facultat de Biblioteconomia iDocumentació Universitat de Barcelona Melcior de Palau, 140,
08014 Barcelona Spain<[email protected]>;
Ernest Abadal, Universitat de Barcelona Melcior de Palau, 140,08014 Barcelona Spain
126 The Journal of Academic Librarianship, Volume 35, Number 2, pages
BACKGROUND
The training of library users has received much atten-tion in recent years. The literature includes manystudies of tutorials created by libraries for this purpose,most of which analyse samples by applying a set of pre-established indicators. One of the first studies of thistype, which is widely consulted and cited, is that ofNancy Dewald.1 Dewald first establishes seven funda-mental indicators for a tutorial to be effective: it mustbe directly related to the content of the subject thatit supports, foster active learning, stimulate collabo-rative learning, be offered in more than one medium,teach by objectives, teach concepts, and include thepossibility of contacting or consulting the librarian ifhelp is needed. She then evaluates a sample of 19 tuto-rials of the Library Instruction Round Table to determinethe degree of compliance with the indicators. In con-clusion, she states that tutorials can never replace face-to-face training.
Kornelia Tancheva2 carried out a descriptive study of45 tutorials. After going over the advantages anddrawbacks of web-based tutorials in some detail, sheanalyses 45 tutorials based on a variety of charac-teristics (general or specific content, degree of inter-activity, feedback, technological aspects, etc.). Sheconcludes that the tutorial must contain audiovisualelements, animation and charts; it must be interactiveand modular; it must be able to evaluate the previousknowledge of the student; it must teach concepts; itmust be linked to a specific discipline; it must provideadvice; and it must record the activities of the studentduring the session.
Taking as a basis the Dewald’s article, Paul Hrycaj3
extends the methodology by going further into theconcept of active learning, and also analyses a greaternumber of tutorials, 65 resources created by librariesforming part of the Association of Research Libraries. Hestudies the active learning in these tutorials, basedmainly on whether they include exercises (which mayconsist of questions at the end of the tutorial or at theend of each module, exercises involving two separatebrowser windows or the option of sending the answersto an instructor). In conclusion, the study notes theincrease in elements of active learning, which Hrycajfound in 39 tutorials out of 65 (55%), compared withonly 7 out of 19 (37%) found by Dewald.
126–131
Finally, the recent paper by Rozalyn Anderson et al.4
studies 274 tutorials used by the medical libraries ofthe Association of American Medical Colleges (mostof them created by the libraries, though a few werecreated by third parties). These authors considerwhether the tutorials were created by the librariesor by third parties, the subject, the software used tocreate the tutorials, the level of interactivity, encour-agement of active learning, inclusion of exercises or aquestionnaire at the end of the module, and the userprofile.
OBJECTIVES AND METHODOLOGY
Following these precedents, the aim of this study is toanalyse the characteristics of the tutorials created byacademic libraries. We applied thirty indicators refer-ring to general characteristics, content, teaching meth-odology, usability and technology to a sample of 180tutorials. The data gathering for the study ended in April2008.
The SampleOver 200 tutorials were consulted and 180 were
selected for the analysis because they met all thefollowing criteria:
– Producer: academic libraries. We did not includetutorials created and designed by departments orfaculties or those linked to companies related to theproduction or distribution of products of the libraryenvironment, such as companies selling libraryautomation software (catalogues or databases).
– Subject: without restriction.
– Languages: English, Spanish, Italian, French or Cata-lan. Those that had a translation in any of the abovelanguages were also selected.
– Platform of dissemination: Internet. DVDs, for exam-ple, were not included, nor were materials designedas part of educational platforms and published onintranets.
– Purpose or orientation: educational and not exclu-sively informative. They had to present trainingobjectives or include exercises aimed at consolidat-ing what was explained in the content. Therefore,presentations that explain the functioning of aresource in which the student browses or interactsmerely by clicking on a button or arrow to continuethe explanation were excluded. This was one of thecriteria that excluded the highest number of tutor-ials, because many of them were exclusively infor-mative rather than formative.
Several databases specialised in this type of informa-tion were consulted in order to locate the tutorials:
– PRIMO (Peer-Reviewed Instructional Materials OnlineDatabase)5
A database produced, maintained and updated bythe American Library Association (ALA) and the
Association of College and Research Libraries(ACRL). Its main aim is to share educationalmaterials created by librarians. It includes differenttypes of material but tutorials are predominant. InApril 2008 it had 178 records. It has very strictinclusion criteria that are clearly explained. Itincludes materials created all over the world, butconcentrates on the USA.
– Instruction resources6
A database of materials for training drawn up by theLibrary Orientation Exchange (LOEX). It collectsmaterials for fostering the training of users andinformation literacy. It has a repository with materialon this subject.
– InfoLit Global7
A portal created in 2005 by the InformationLiteracy Section of the IFLA and UNESCO. Itsobjectives include gathering reports on informationliteracy at an international level and creating theInternational Information Literacy Resources Direc-tory on materials for training and literacy of users.The database of resources includes two categories,tutorials in general (141 records in April 2008) andtutorials on use of specific resources (281 records inApril 2008).
– ANTS (Animated Tutorial Sharing Project) 8
A collection of the interactive tutorials of theCouncil of Prairie and Pacific University Libraries(COPPUL), a consortium of twenty Canadian aca-demic libraries deposited in a repository of theUniversity of Calgary. In April 2008 it had 46tutorials.
Furthermore, an exhaustive consultation of the pagesof academic libraries in the USA, Canada, Australia,France, Germany, Great Britain and Spain was carriedout in order to extend the geographic coverage of thetutorials selected. The distribution lists on educationand information literacy were also consulted,9 as weremany personal pages and blogs such as Alfin Red,10
FORMIST informations,11 Information Literacy,12 theweblog of Sheila Webber and Stuart Boon and Informa-tion Literacy in Canada.13
IndicatorsFor the analysis a set of 30 indicators (shown inTable 1)
were divided into five main areas: general charac-teristics, content, educational features, browsing anddesign, and technological features. These indicatorswere drawn up specially for this study based on theliterature on evaluation of teaching resources (e.g. T.Bickney-Holmes14 and D. Charney15) and on usability(e.g. Shneiderman16 and Bevan17).
A MySql database was created to systematise andprocess the 180 tutorials based on the indicators. Theinformation unit was the tutorial (one record for eachtutorial), and a field was created for each of the 30indicators. Many fields were of the “Yes/No” type toindicate the presence or absence of the indicator
March 2009 127
Table 1Indicators Used
Area Indicator Description
General characteristics Country
Language
Date of creation
Date updated
User profile Type of target user.
Content Type Textual, images, etc.
Subject
Structure Whether the content is divided into modules or linear
Levels The possibility of adapting to different levels of userknowledge (general, advanced, etc.).
Presence of a glossary
Indication of objectives A foreword on the educational objectives.
Presence of a summary
Rights
Educational features Timing A statement of the time estimated for studying eachsection.
Pretest Some system for evaluating previous knowledge.
Teaching methodology Presentation of content, guided demonstration,problem-solving.
Presence and type of exercises Questionnaire, practice, games, etc.
Evaluation of the exercises Whether there is some form of evaluation or feedbackfor the exercises.
Contact with the librarian Queries and comments can be made.
System of evaluation of thetutorial
Whether the tutorial can be evaluated or provides thepossibility of suggesting improvements.
Browsing and design Browsing guide
Site map
Multimedia elements Use of audio and video.
Status bar
Personalisation of screen andcolours
Change of letter size
Need for password
Technological features Programming language PHP, HTML, etc.
Animation Whether the tutorial facilitates interactivity with thesystem.
Download of plug-ins
Different versions The possibility of consulting the tutorial in differentversions (without animation, basic text, etc.).
represented (for example, “Presence of glossary”,“Indication of objectives”, “Need for password”); otherfields had a list of variables (for example, the field “Typeof exercises” included the values, “games”, “question-naires” and “guided practice”).
128 The Journal of Academic Librarianship
ANALYSIS OF THE RESULTS
General CharacteristicsMost of the tutorials were produced at universities
in English-speaking countries on three different con-
Figure 1Levels of Content (N=180)
tinents—the USA (71.67%), Australia (8.33%) and GreatBritain (7.78%). The rest were divided among ten othercountries. English was the language used for the con-tent of most tutorials (92.22%). Only eight tutorialswere developed in two languages; in these casesEnglish was used as the second language in order toachieve wider dissemination of the content.
Only 43.33% of the tutorials stated the date ofcreation. Though this is not an essential indicator, it isvery useful for tracing the trajectory of the tutorial. Only46.7% indicated the date of updating, which is ofinterest for detecting changes in content and variationsin the design or the technology used. Only 48.9% spe-cified the profile or type of persons for whom thetutorials were intended, which is very important infor-mation for orienting potential users.
ContentSixteen different subject types were identified, the
predominant one being information literacy (42.78%),18
followed by the process of searching in informationsources (12.78%) and then—highly fragmented—the re-maining subjects. There are also 13 tutorials about morespecific subjects (e-mail, online discussion groups, useof EndNote, etc.) which has not been detailed in Table 2.
Images, icons, photographs or combinations of thesame are contained in 81.7% of the tutorials of academiclibraries. The content was presented in linear form in12% of the tutorials and in modular form in the re-maining 88%. The linear tutorials tended to be the olderones. The use of sections or modules allows the stu-
Table 2Subjects of the Tutorials
SubjectNumber andPercentage
Information literacy 77 (42.78%)
Searching in information sources 23 (12.78%)
Location of books or articles 9 (5%)
Databases 9 (5%)
Thematic resources 7 (3.89%)
Search strategies 6 (3.33%)
Catalogue searching 6 (3.33%)
Internet 5 (2.78%)
Library services 5 (2.78%)
Evaluation of Internet information 4 (2.22%)
Plagiarism and copyright 4 (2.22%)
Library services and resources 4 (2.22%)
Citation 2 (1.11%)
Browsers 2 (1.11%)
Plagiarism and citation 2 (1.11%)
Academic journals 2 (1.11%)
Other 13 (7.2%)
Total 180 (100%)
dents to structure and plan the learning of the contentbetter, and to resume the trainingmore easily if they areunable to follow the tutorial in a single session.
Only 4 of the tutorials (3%) showed distinctions bet-ween levels (basic or expert, and in some cases basic,intermediate and advanced) (Fig. 1). The inclusion oflevels is important because it fosters flexible learningand adapts to the needs of user with different levels ofknowledge and different ways of learning.
“Only 4 of the tutorials (3%) showeddistinctions between levels (basic or expert,
and in some cases basic, intermediateand advanced).”
Only 69 tutorials (38%) specified the objectives ofthe tutorial or of each section. This is another elementthat favours autonomous education, and may refer tothe whole tutorial or to specific sections. The objectiveshelp to situate the students in the learning process,indicating the knowledge or skills that they should havelearned at each stage of the process and what remainsto be learned. Furthermore, only 43% of the tutorialshad a glossary of the most important terms related tothe content. Only 25 (14%) included a summary, whichreinforces the learning and informs of the sections thathave and have not been consulted.
With regard to publication rights, 88 tutorials (49%)included the copyright notice, 67 (43%) gave no infor-mation, 7 (4%) used Creative Commons licences and 7used Open Publication Licences.
Educational FeaturesOnly 37 (21%) of the tutorials indicated the time in
which the tutorial should be studied (either generally orin sections). Most tutorials did not include systems forevaluating the previous knowledge of the students. Only10 (6%) contained a test or pre-test to determine stu-dents’ previous knowledge and skills. The option chosenby these was a pre-test in the form of a questionnaire.
The teachingmethodology used inmost tutorials wasto present the content and consolidate it throughexercises. This was the case in 102 tutorials (56.67%).Guided demonstration—a very valid option for teachingprocesses such as searching for information in
March 2009 129
Figure 2Whether Users’ Opinion Solicited on the
Tutorial (N=180)
catalogues or databases—was the second most usedmethodology (12.22%). Problem-solvingmethodologieswere far behind, with only 2 tutorials (1.11%), becausethey are the most difficult to develop in thisenvironment.
It is important for the tutorial to have exercises(games, guided practice or a questionnaire) that rein-force what has been learned or what is to be taught.These were found in 138 of the tutorials (77%).With regard to the type of exercises, the figures wereas follows: 58 tutorials (42.03%) contained a ques-tionnaire, 34 (24.64%), a questionnaire with practicalexercises, 29 (21.01%) only practical exercises and 17(12.32%) interactive elements in the form of games.
With regard to the evaluation of the exercises, 77tutorials (42.22%) had no scoring but indicated someinformation on whether the answers were right orwrong and provided supplementary information onthis. The second most common option (17.22%) was theuse of exercises with scoring and feedback.
Most tutorials included some system of assistance forcontacting the librarian. Of these, 133 (73.89%) usede-mail and 8 (4.44%) used a form. Surprisingly, 31tutorials (17.22%) offered no possibility of contactingthe librarians. We considered whether users were ableto give their opinion on the usefulness of the tutorial, thelearning process and the formal aspects of the tutorial.Only 21% of the tutorials contained a mechanismallowing them to do so (Fig. 2). The system most usedwas the survey, followed by an online form and e-mail.
Browsing and DesignThe aim of this section is to determine the extent to
which the tutorials met the basic recommendations ofusability (browsing and design) of the interface. Only28 of the tutorials (16%) contained a guide on how tobrowse through the tutorial. Only 32 (18%) had a sitemap—a very useful element for obtaining an overviewof the content. In most cases it was textual. Only 11tutorials (6%) included a status bar, which helps users tofind their way in the tutorial. Only 27 tutorials (15%)included an audio feature.
In terms of accessibility and usability, the tutorialsin the sample rated very poorly. None of them offeredthe possibility of personalising screens or colours, andfew the possibility of changing the font size. Finally,the vast majority of the tutorials (94%) did not requirea password.
130 The Journal of Academic Librarianship
Technological FeaturesThe programming languages most used in the draft-
ing of the tutorials were HTML (83.83%), followed at agreat distance by PHP (7.19%). In cases in which anima-tion was offered, the vast majority (96%) used Macro-media Flash and the rest (4%) Quicktime. The downloadof plug-ins to view the tutorial correctly was offered byonly 45 of the tutorials (25%). Only 6 of the tutorials(3%) offered different versions.
CONCLUSIONS
The situation portrayed in this descriptive study indi-cates in general that web-based tutorials offered byacademic libraries are at an early stage of development.Though statistical data may conceal particular cases,only 12 tutorials satisfied most of the indicators—arelatively low proportion of the sample (7%).
“. . .in general ... web-based tutorialsoffered by academic libraries are at an
early stage of development.”
One of the minimum improvements that tutorialsshould include is the explicit indication of the educa-tional objectives, which was found in only 38% of thesample. Another important element that favours auton-omous learning is information on the time that studentswill need in order to follow the tutorial, which wasfound in only 21% of the sample. Increasing insistence isplaced on the fact that not all students know and learnin the sameway, so it is important to include some formof evaluation of students’ previous knowledge in orderto indicate the part of the tutorial that they shouldfollow. This was found in only 6% of the sample. It wouldalso be useful for the tutorials to include basic andadvanced levels of the same subject, which were foundin only 3% of the sample. However, it must be stated thatthis is not always easy for the subjects of informationliteracy and information searching taught by manypresent-day tutorials. Another fundamental element isthe implementation of some form of evaluation of thetutorial, which was found in only 21% of the sample.
These are the aspects of the tutorials that show roomfor improvement. At a future stage of development,progress must be made towards diversifying the teach-ing methodologies rather than using only the expositivemethod. Furthermore, the exercises should incorporatemore elements of feedback because the existing onesare too simple. More game-type exercises should beincluded to complement the questionnaire-type exer-cise. In addition to these elements that directly affectthe learning process, the tutorials should developelements that favour usability and accessibility, andthat allow greater interactivity of the students with thesystem. Finally, they should incorporate informationaimed at distance-learning students, who are increas-ingly the target of university education offerings.
If academic libraries wish to increase their role in allaspects of education, they must provide high-qualitytutorials. There are some clear signs of progress, as inthe Spanish university libraries studied by Somoza-Fernández and Abadal,19 which show clear progressfrom teaching the use of bibliographic sources toteaching information literacy, including training objec-tives, exercises and other teaching elements. Never-theless, much work remains to be done before the web-based tutorials created by academic libraries reach amature stage of development.
APPENDIX A. SUPPLEMENTARY DATA
Supplementary data associated with this article canbe found, in the online version, at doi:10.1016/j.acalib.2009.01.010.
NOTES AND REFERENCES
1. Nancy H. Dewald, “Transporting Good Library InstructionPractices into theWeb Environment: an Analysis of OnlineTutorials,” The Journal of Academic Librarianship 25 no. 1(1999): 26–32.
2. Kornelia Tancheva, “Online Tutorials for Library Instruc-tion: An Ongoing Project Under Constant Revision,” ACRLEleventh National Conference: Learning to Make a Difference2003. Available: http://www.ala.org/ala/acrl/acrlevents/tancheva.PDF
3. Paul L. Hrycaj, “Elements of Active Learning in the OnlineTutorials of ARL Members,” Reference Services Review 33no. 2 (2005): 210–218.
4. Rozalynd P. Anderson, Steven P. Wilson, Mary BrigetLivingston, and Allison D. LoCicero, “Characteristics andContent of Medical Library Tutorials: a Review,” Journal ofMedical Library Association 96 no. 1 (2008): 61–63.
5. Association of College & Research Libraries, “PRIMO (Peer-Reviewed Instructional Materials Online Database,” ALA.Available:, http://www.ala.org/apps/primo/public/search.cfm
6. Library Orientation Exchange (LOEX), “Instruction Re-sources,” LOEX. Available: http://www.emich.edu/pub-lic/loex/resources.php
7. IFLA and UNESCO, “InfoLit Global,” IFLA and UNESCO.Available: http://www.infolitglobal.info/index.php?pg=10
8. Council of Prairie and Pacific University Libraries(COPPUL), “ANTS (Animated Tutorial Sharing Project),”COPPUL. Available: https://dspace.ucalgary.ca/handle/1880/43471
9. Association of College & Research Libraries, “ILI-L (Infor-mation Literacy Instruction Discussion List), ALA. Avail-
able: http://www.ala.org/ala/acrlbucket/is/ilil.cfm.RedIris, “ALFAINFOR”, RedIris. Available: http://www.rediris.es/list/info/alfainfor.es.html Grup d’Alfabetitzacióinformacional,”ALFINCAT”, Col.legi Oficial de Bibliotecarisi Documentalistes de Catalunya. Available: http://www.cobdc.org/grups/alfincat/index.html
10. Ministerio de Cultura, “Alfin Red,” Ministerio de Cultura.Available: http://www.alfinred.org/blog
11. “FORMIST informations”. Available:, http://blogformist.enssib.fr/index.php
12. Sheila Webber and Stuart Boon, “Information Literacy”.Available: http://information-literacy.blogspot.com/
13. Canadian Library Association, “Information Literacy inCanada,” Canadian Library Association. Available: http://blog.uwinnipeg.ca/ilig/
14. Tracy Bicknell-Holmes and Paul Seth Hoffman, “Elicit,Engage, Experience, Explore: Discovery Learning in LibraryInstruction,” Reference Services Review 28 no. 4 (2000):313–322.
15. Davida Charney, Lynne Reder, and Gail W. Kusbit, “GoalSetting and Procedure Selection in Acquiring ComputerSkills: A Comparison of Tutorials, Problem Solving andLearner Exploration,” Cognition and Instruction 7 no. 4(1990): 323–342.
16. Ben Shneiderman, Designing the User Interface: Strategiesfor Effective Human Computer Interaction. 3 ed. (Boston:Addison-Wesley, 1998).
17. Nigel. Bevan, “International Standards for HCI and Usabi-lity,” International Journal Human-Computer Studies 55(2001): 533–552.
18. Tutorials on information literacy teach the users torecognise the need for information—to locate, searchfor, evaluate and communicate the information effi-ciently. They explain the different types of sources(dictionaries, databases, catalogues, etc.), how to delimitand extend the query, and how to effectively apply theinterrogation language (Boolean searches, masks, trunca-tion, etc.). They also include skills and knowledge oninformation retrieval from resources such as databases,catalogues and the Internet. They present criteria forevaluating and effectively communicating the informa-tion, with emphasis on legal aspects such as plagiarismand citation conventions. Some of them deal with com-munication (alerts, RSS feeds, blogs), some deal with theinformation cycle, and some include sections on the lib-rary services.
19. Marta Somoza-Fernández and Ernest Abadal, “La forma-ción de usuarios en las bibliotecas universitarias españo-las,” El Profesional de la Información 16 no. 4 (2007):287–293.
March 2009 131
