Quit smoking information available for general public on the www in Italian: a cross sectional survey M.C.Mazzoleni, I.Giorgi, F.Boveri Medical Informatics

Quit smoking information available for general public on the www in Italian: a cross sectional survey

M.C.Mazzoleni, I.Giorgi, F.Boveri

Medical Informatics Unit, Psychology Unit, Salvatore Maugeri Foundation, IRCCS - Pavia Italy

Aim

To evaluate the information available on the www for general public

in Italian focusing on a specific topic

in order to

- characterize the products that Italian web publishers offer today

- gain an insight into how to overcome the present situation

Background

A certain number of heath-related sites in Italian is available on the www

20 millions of Italian speaking persons use internet

Background


20 milions of Italian speaking persons use internet

Background


Italian population has interest for heath and well being - related information

20 millions of Italian speaking persons use internet

sometimes

often

never

Practice to seek health related information

14%Snapshot on an unpretentiuos

sample

internet usersadverse

internet usersdisposed

non internetusers

Disposition to the usage of internet to seek health related information

33%

100 relatives in an out-patient pediatric clinic

75% aged between 20 and 40

67% female33% male

31% mandatory educational level63% higher educational level

Formal evaluation: “quality” of the sites in terms of transparency of communication1

Evaluation of the content as regards quit smoking programmes2

Evaluation of accessibility in terms of readability of the texts3

Identification of a sample of sites dealing with smoking related problems as if we were a lay-person

Evaluation of the sample

Map of the study

Collection of the sites

Keywords

smoke damages, quit smoking,quit smoking centers,

quit smoking programmes

First 50 items

215 addresses

Dead, duplicate, non pertinent sites

215 addresses

91 addresses

3 sites were explicitly intended for physicians

16 sites were exclusively promotional *

72 sites contained information

* 12 of them with miraculous systems to quit smoking

The sample


Formal evaluation: “quality” of the sites in terms of transparency of

communication1



Criteria for Assessing the Quality of HealthInformation on the Internet - Policy Paper

Supported in part by the Agency for Health Care Policy and Research,

Grid for site evaluation

Evaluation of the formal quality of the sites

a stated responsible (person or group) of the content;

e-mail of the responsible;

verifiable credentials of the content responsible;

stated aim of the site

stated intended reader;

currency of the site;

clear disclaimer

HON symbol;

funding policy

Rough grid for reliability evaluation

Evaluation of formal quality of the sites

Analysis of formal aspects

0%

20%

40%

60%

80%

100%

Addressee

Responsible

E-mail

Credentials

Last Update

FundingAim

Disclaim

er

HON code

presence absence

02468

101214161820

N°

of s

ites

0 po

ints

1 po

int

2 po

ints

3 po

ints

4 po

ints

5 po

ints

6 po

ints

7 po

ints

8 po

ints

9 po

ints

Formal Score ( range 0 - 9) : Total number of satisfied requisites

50%

0

5

10

15

20

25

30

Private person

Physician

Non medical assoc.

Medical assoc./University

Scientific assoc.

Press agency

Not verifiable

Nu

mb

er o

f si

tes

for

typ

e of

sou

rce Formal Score range

4-4 2-5 1-6 2-5 4-5 2-4 0-5

They have very few aids to evaluate site reliability and adequacy to their needs. A similar situation is depicted in [1, 2, 3,4]

Few information doesn’t mean low authorship but only low visibility

Rules for correct publishing on the www are scarcely known

Quality seals are not used, in spite of the information campaigns

Italian health consumers have a few chances to find information on smoking related problem

[1] Shon J, Musen MA The low availability of metadata elements for evaluating the quality of medical information on the World Wide Web. Proc AMIA Symp1999;:945-9[2] Latthe M., Latthe PM, Charlton R. Quality Information on Emergency contraception on the internet. Br J Fam Plann 2000 Jan; 26(1): 39:43[3] Latthe M., Latthe PM, Khan KS Quality of Medical Information about menorraghia on the worlwide web. BJOG 2000 Jan; 107(1); 39-43.[4] Tamm EP, Raval BK, Huynh PT. evaluation of the quality of self-education mammography material available for patients on the Internet. acad radiol 2000 Mar; 7(3): 137-41

Synthesis of the results



communication1



Quit smoking programmesContent Evaluation

Sample25 sites

Three dimensions approach - two independent observersP. Saccheti,P.Zvara, MK Plante. The internet and patient education resourcesand

their reliability: focus on a selected urologic topic. Urology 53 (1999), 117-20

1=incorrect, 2=some incorrect, 3=correct, 4=correct and referenced statements

ACCURACY

1=isolated, 2=several, 3=most, 4=all issuesCOMPREHENSIVENESS

1=disagreement with accepted psycho-medical practice,2=partial agreement, 3=full agreement,4=full agreement including alternatives

OBJECTIVITY

Content score = accuracy + comprehensiveness + objectivity range 3 - 12

Dimensions Scale

Quit smoking programmesContent Evaluation

Golden standard for comparison

02468

101214161820

N°

of

site

s

Accuracy Comprehens. Objectivity

Accuracy, Comprehensiveness and Objectivity of the content

lev.1lev. 2lev.3lev.4

Accuracy•lev. 1 incorrect•lev. 2 some incorrect•lev. 3 correct•lev. 4 correct and referenced statements

Comprehensiveness•lev. 1 isolated•lev. 2 several•lev. 3 most•lev. 4 all issues

Objectivity•lev. 1 disagreement with accepted practice•lev. 2 partial disagreement•lev. 3 agreement•lev. 4 agreement and alternatives

02468

101214161820

N°

of s

ites

wit

h

Comprehensiveness: Guidelines issues presence in the sites

Content Score vs Formal Score

3456789

101112

0 1 2 3 4 5 6 7 8 9

Formal Score

Con

ten

t S

core

The content of the inspected sites (25) - compared with AHCPR consumers guideline - is fairly good

Coverage of the topics is, in general, the most problematic aspect

The results are in agreement with the ones published in [5] for anglophone sites on Viagra.

Italian sites are not an isolate example.

An editorial project seems often to be lacking.

[5] P. Saccheti,P.Zvara, MK Plante. The internet and patient education resourcesand their reliability: focus on a selected urologic topic. Urology 53 (1999), 117-20


Let us remember that 12 sites (not analyzed here) have been found promoting miraculous systems to quit smoking



communication1



Intelligibility: easiness with which a text is understood by the reader

Readability: characteristics of a text that make it easily readable

Fry Readability Graph (FRG) English and Spanish

Flesh and Flesh -Vacca English and Italian

Gulpease Italian

Evaluation of text readability

Reading grade level = f ( number of words, syllables, sentences, letters)

Gulpease Index = 89 - (Lp/10) + (3+Fr)

Lp = (100 x total number of letters) / total number of wordsFr = (100 x total number of sentences) / total number of words

Educational level

Ind

ex v

alu

es

The higher G.I. value

the easier the text

** *

*

*** * * * *

***

* * * * *

Primary school

Mandatory school

High school

Almost incomprehensible

Very difficult

Difficult

Very easy

Easy* ** * * *

Gulpise index values of the sample

Readability is only a necessary condition for intelligibility

The analyzed texts are easily readable in a high percentage (72%) by people that have attended

only mandatory school

In contrast with [1] this study, with all its limits, shows that internet could serve as a “leveler”

across different socio-economic backgrounds for Italian speaking people

[1]Gretchen K. Bertrand et al. Health information on the internet. Accessibility, Quality, and Readability in English and Spanish. JAMA, May 23/30, 2001 -Vol 285, No 20


The www doesn’t contain either only garbage or only high quality information.

The information published on the web may potentially reach a great number of users with few evaluation tools

Publishing on the web does not only satisfy our narcissism, but can also contribute to satisfy other

people information needs

….. Trying to conclude

Give visibility to the authorship and be transparent and clear, so that health consumer can reasonably

trust the web and exploit all the potentialities

Educate the surfer on the problem of good quality heath-related information while publishing on the web

Publishing on the web is easy, modifiable and can be made by oneself: don’t be naive and have a project in

your mind, as if you were publishing on very expensive glossy paper

As soon as the web is matureI will stop smoking

Today I promise solemnly

Documents

Quit smoking information available for general public on the www in Italian: a cross sectional survey M.C.Mazzoleni, I.Giorgi, F.Boveri Medical Informatics