DIABETES TECHNOLOGY & THERAPEUTICSVolume 5, Number 3, 2003© Mary Ann Liebert, Inc.

Why People Download the Freeware AIDA v4.3aDiabetes Software Program: A Proof-of-Concept

Semi-Automated Analysis

ELDON D. LEHMANN, M.B. B.S., B.Sc.1,2

ABSTRACT

AIDA is a diabetes-computing program freely available at www.2aida.org on the Web. Thesoftware is intended to serve as an educational support tool and can be used by anyone whohas an interest in diabetes, whether they be patients, relatives, health-care professionals, orstudents. In previous “Diabetes Information Technology & WebWatch” columns various in-dicators of usage of the AIDA program have been reviewed, and various comments from usersof the software have been documented. The purpose of this column is to overview a proof-of-concept semi-automated analysis about why people are downloading the latest version ofthe AIDA educational diabetes program. AIDA permits the interactive simulation of plasmainsulin and blood glucose profiles for teaching, demonstration, self-learning, and researchpurposes. It has been made freely available, without charge, on the Internet as a noncom-mercial contribution to continuing diabetes education. Since its launch in 1996 over 300,000visits have been logged at the main AIDA Website—www.2aida.org—and over 60,000 copiesof the AIDA program have been downloaded free-of-charge. This column documents the re-sults of a semi-automated analysis of comments left by Website visitors while they weredownloading the AIDA software, before they had a chance to use the program. The Internet-based survey methodology and semi-automated analysis were both found to be robust andreliable. Over a 5-month period (from October 3, 2001 to February 28, 2002) 400 responses werereceived. During the corresponding period 1,770 actual visits were made to the Website sur-vey page—giving a response rate to this proof-of-concept study of 22.6%. Responses were re-ceived from participants in over 54 countries—with nearly half of these (n 5 194; 48.5%) orig-inating from the United States, United Kingdom, and Canada; 208 responses (52.0%) werereceived from patients with diabetes, 50 (12.5%) from doctors, 49 (12.3%) from relatives of pa-tients, with fewer responses from students, diabetes educators, nurses, pharmacists, and otherend users. The semi-automated analysis adopted for this study has re-affirmed the feasibil-

477

1Academic Department of Radiology, St. Bartholomew’s Hospital, and Department of Imaging (MRU), ImperialCollege of Science Technology & Medicine, NHLI Royal Brompton Hospital, London, United Kingdom.

2Dr. Lehmann is a co-developer of the AIDA diabetes simulator.The AIDA software referred to in this report is an independent, noncommercial development which is being made

available free-of-charge via the Internet—at a dot org (.org) not-for-profit Website—as a noncommercial contributionto continuing diabetes education.

Diabetes Information Technology & WebWatch

INTRODUCTION

AIDA IS A FREEWARE COMPUTER PROGRAM thatpermits the interactive simulation of

plasma insulin and blood glucose profiles fordemonstration, teaching, self-learning, and re-search purposes. It has been made freely avail-able, without charge, via the Internet as a non-commercial contribution to continuing diabeteseducation. In the 71 years since its originalWorld Wide Web launch in March/April 1996over 300,000 visits have been logged to themain AIDA Web pages at www.2aida.org, andover 60,000 copies of the program have beendownloaded, gratis.

The AIDA software has been previously de-scribed in detail in this journal,1 and elsewhere.2

Figure 1 demonstrates a little of what the

program can do. In the representation of anAIDA simulation shown, a baseline blood glu-cose simulation is displayed by the grey linefor a twice-daily short- and intermediate-acting[Actrapid® and Lente® (Novo Nordisk)] insulininjection regimen. Superimposed in black,shown by the solid line on the top graph, is asimulation for educational use demonstratingthe hyperglycaemia that would result from de-laying the morning insulin injection until afterbreakfast.3 AIDA comes with 40 such samplecase scenarios for simulation. Further cases canbe created by users, and more examples of thesort of simulations that AIDA can offer can befound at www.2aida.org/demo on the Inter-net.

While other interactive simulators of glu-cose–insulin interaction in diabetes have been

INFORMATION TECHNOLOGY & WEBWATCH478

ity of using the Internet to obtain free-text comments, at no real cost, from a substantial num-ber of medical software downloaders/users. The survey has also offered some insight intowhy members of the public continue to turn to the Internet for medical information. Fur-thermore it has provided useful information about why people are actually downloading theAIDA v4.3a interactive educational “virtual diabetes patient” simulator.

FIG. 1. In the representation of an AIDA simulation shown, a baseline blood glucose simulation is displayed by thegrey line for a twice-daily short- and intermediate-acting (Actrapid and Lente) insulin injection regimen. Superim-posed in black, shown by the solid line on the top graph, is a simulation for educational use demonstrating the hy-perglycaemia that would result from delaying the morning insulin injection until after breakfast (1 mmol/L 5 18mg/dL). Reproduced from Lehmann3 with kind permission © The Lancet. Further examples of the sort of simula-tions that AIDA can offer can be found at www.2aida.org/demo on the Internet.

described in the literature,4–11 to date these donot seem to have been extensively distributedvia the World Wide Web, or been made par-ticularly widely available. Furthermore for anumber of these6,8–10 it would seem that read-ers are totally dependent on the authors’ owndescriptions of their prototypes, since no ver-sions appear generally available for usage byothers. This is not the case with AIDA, whereall the AIDA simulators are freely available onthe Internet.

In order to understand more about whatAIDA users like and dislike about the diabetessimulation program, and in order to identifywhat people are finding useful with the soft-ware, various preliminary audits and surveyshave been undertaken via the AIDA Website(www.2aida.org), and via the program appli-cation itself. These are yielding useful infor-mation about what sort of people are down-loading and using the software.12,13 In a recentpilot study we have also sought to try and iden-tify why people have in the first place beenturning to the Internet to download AIDA.14

A mention of study terminology may behelpful here. A pilot study is well recognisedas being the initial experimental experiencewith a new approach. By contrast we regard a“proof-of-concept” study as aiming to repro-duce the experimental approach of an initial pi-lot study, retrospectively, before proceedingonto larger-scale/longer-term studies—whichmay be conducted either retrospectively orprospectively. Connected with this, a proof-of-concept survey can be regarded as a second pi-lot study; the idea is to try and expand experi-ence with running and analysing data fromsuch an audit, before applying the approachmore widely.

In order to continue to make efficient useof data from such audits and surveys, it is im-portant for the analysis procedure adopted tobe as straightforward, streamlined, and ob-jective as possible. To achieve this a semi-au-tomated process with as few manual or sub-jective steps as possible should be theoptimum method of analysing such data.Therefore to ensure that the approach was asrobust as possible, we have also now soughtto semi-automate the data analysis process forthis study.

METHODOLOGY

The data collection methodology has beenpreviously described in detail elsewhere in thisjournal.14 For the current study the data col-lection approach was applied unchanged.However, what was refined was the data anal-ysis methodology that was adopted. In the pre-vious study a more manual, “hands on” ap-proach was used. Potentially, though, thismight be misconstrued as being prone to “bi-ases” on the part of the researcher doing theanalysis. By contrast, a semi-automated tech-nique of data analysis should be less prone toany such potential errors, and therefore shouldallow more robust and objective comparisonsto be made between surveys over time. The ra-tionale behind this approach is that hopefullyby removing any possible subjective compo-nent from the analysis, it will become more ob-jective and robust.

Therefore for the current work we havesought to confirm the feasibility of undertak-ing a free-text survey about downloading ofAIDA v4.3a via the Internet. For this proof-of-concept survey we were particularly keento confirm the findings of an earlier pilotstudy14 and test out our approach to confirmwhy people are actually turning to the WorldWide Web and downloading the AIDA soft-ware.

AIDA Website visitors were surveyed fora period of 5 months between October 3, 2001and February 28, 2002. Responses were sub-mitted by end users using a “guestbook”-type of free-text CGI-BIN script operating onthe AIDA Web servers, as described previ-ously.14 All comments were each individu-ally stored in a dedicated HTML database atthe AIDA Websites. The raw, source data forthis pilot study—from both the main AIDAWebsite (www.2aida.org) and from the former AIDA U.S. mirror site (http://us.2aida.org)—can be viewed directly on the In-ternet at www.2aida.org/why2 (Fig. 2).(Please note that since this study was under-taken the AIDA U.S. mirror site has movedfrom http://us.2aida.org—which was basedin Baltimore, MD—to a new separate facility at http://www.2aida.net based inCalifornia.)

INFORMATION TECHNOLOGY & WEBWATCH 479

SEMI-AUTOMATED DATA ANALYSIS

A number of steps were taken to facilitate thesemi-automated analysis of this dataset:

First, the why2.htm HTML Web page con-taining the downloader responses was saved inASCII text format. This could then be importedinto a standard spreadsheet like Microsoft Ex-cel.™ A priori, from previous studies,12–14 eightcategories of respondent had been predefined:(i) a patient with diabetes, (ii) a doctor, (iii) arelative of a patient, (iv) a student, (v) a dia-betes educator, (vi) a pharmacist, (vii) a nurse,and (viii) none of the above. These corre-sponded to the predefined options that weremade available on the HTML data submissionform. As the data were being imported into anExcel spreadsheet, the descriptive category ofthe respondent could be separated from theother data using Excel import wizard delim-iters. Similarly the respondent’s country could

be separated from the free-text comments. Inthis way it was easy to sort the Excel databaseon the descriptive heading to establish thenumber of patients with diabetes, relatives,doctors, etc., who had been responding to thissurvey. In the same way, sorting the data onthe country category enabled the number of re-spondents per country to be established.

Some other categories of description of adownloader’s response could not be automatedas they were more subjective, and therefore itnecessarily remained a value judgement on thepart of the person analysing the data as towhich particular additional category could beapplied. These alternative subcategories can bedescribed as: (ix) “techies”—downloadingAIDA for technical/information technologypurposes; (x) people who missed the point asto what AIDA is about (e.g., wanting a programto download/graph blood glucose test data);(xi) people who just want to “try it and see”;

INFORMATION TECHNOLOGY & WEBWATCH480

FIG. 2. Shows some of the respondents’ comments as stored in the HTML database at the AIDA Website. These canbe viewed directly at www.2aida.org/why2 on the Internet.

and (xii) people who intended to try and ad-just their carbohydrates and/or insulin dosesusing the program (AIDA is not meant for this).

For the purpose of selecting some free-textresponses for publication (see Results below),a decision was taken only to consider com-ments that were $2 lines in length. This wasnot a completely arbitrary decision, as it wasfelt that longer items are usually more inter-esting and informative than more “monosyl-labic” comments.

RESULTS

Four hundred proof-of-concept survey re-sponses were received during the 5 months be-tween October 3, 2001 and February 28, 2002.During this period there were 1,770 visitslogged in total to the survey Web pages at themain AIDA Website, and the former AIDA U.S.mirror site, giving a response rate to this proofof concept survey of 22.6%.

Based on the automated data analysis, 208responses (52.0%) were received from patientswith diabetes, 50 (12.5%) from doctors, 49(12.3%) from relatives of patients, 36 (9.0%)from students, 18 (4.5%) from diabetes educa-tors, six (1.5%) from pharmacists, and four(1.0%) from nurses. Sixteen responses (4.0%)were received from people who regardedthemselves as being in none of the aforemen-tioned categories, and 13 (3.3%) respondentsdid not answer this question.

Based on the manual review of the re-sponses, some three respondents (0.75%)could be described as “techies” downloadingthe software for some technical use, whilesome 13 respondents (3.25%) missed the pointas to what AIDA was about, mainly wishingto use the software for downloading/graph-ing blood glucose data. Approximately 40 re-spondents (10.0%) were just downloadingAIDA to “try it and see,” with only one re-spondent (0.25%) possibly reporting planningto use the software for a nonintended purpose(e.g., to try adjusting his/her carbohydrate in-take and/or insulin regimen).

Responses were received from end users inover 54 countries. However, nearly one-thirdof the responses (n 5 123; 30.8%) came fromthe United States with a further 13.0% (n 552) coming from the United Kingdom. Table1 summarizes the number of responses percountry with five or more respondents. For92 responses (23.0%) no country was speci-fied.

These summary statistics, although interest-ing, do not, however, tell the personal story asto why people have actually been turning tothe Internet and downloading the AIDA v4.3asoftware. Limited space precludes publicationhere of all the free-text comments—althoughthese can be read directly at www.2aida.org/why2 on the Web. Nevertheless, a selection of the more substantial responses are docu-mented below under separate headings foreach of the groups of respondents.

INFORMATION TECHNOLOGY & WEBWATCH 481

TABLE 1. SUMMARY OF THE NUMBER OF RESPONSES

PER COUNTRY WITH FIVE OR MORE RESPONSES

Country Number of responsesa Percentage (of 400 total)

United States 123 30.8United Kingdom 52 13.0Canada 19 4.8India 13 3.3Australia 7 1.8Denmark 6 1.5Greece 5 1.3Brazil 5 1.3

aOf 230 responses (57.5%) from eight countries. The remaining 170 re-sponses (42.5%) from over 46 other countries (with four or fewer responses per country) are not shown. No country was specified for 92 responses (23.0%).

HOW DO YOU SEE YOURSELF MAYBEMAKING USE OF THE AIDA

SOFTWARE?

Downloader comments from patients with diabetes

From the USA: I am always looking for waysto improve my life. I wish to use this programto see what benefits it may provide.

From Chile: I think will use them to analysemy glycemic results & take rational decisionsabout my insulin therapy.

No country specified: Take care of myselfand distribute the knowledge to other peoplewho are suffering.

From the USA: Recently put on medicationfor type II diabetes. Told to lose weight, etc. Iknow I have a reaction to certain “hiddensugar” foods—often in “low-fat” or “lite”foods. So losing weight, reducing sugar, eatingright, taking medication—even with advicefrom a doctor specializing in diabetes, and askinny dietician who basically says “don’t eat,”I want a better handle on myself as I go throughthe highs and lows, ups and downs . . . “a levelplaying field?”

From Australia: Run simulated models forplanned changes to my routines, etc., alongwith consultation with my health profession-als.

From the USA: Simulate action of 4 insulintypes ranging from Humalog through Ultra-lente humulin. To help me better predict re-sponse with fairly complicated routine. I’mtype II with a glycohemoglobin of 7.2 and ab-solutely no diet compliance. I’ve just added theUL [ultralente] to work toward an under 7.0glycohemoglobin number.

From New Zealand: I wish to control my di-abetes a lot better as I have been having badepisodes and my helpers are not sure what iscausing them so I am keen to try anything thatwill help me.

From the UK: Run simulations prior to dis-cussion with my diabetic adviser specialistnurse as a basis for discussion and advice.

From the USA: I need some encouragementin my therapy for diabetes and any help is ap-preciated. I think this will give me that help.

From the USA: I’m new to diabetes and newto this software, so I’m looking for ANY assis-

tance in dealing with my blood sugar testingand my diabetes in general, so I honestly can-not tell you how I plan to use this sofware ex-cept that I’ll use it to do all I can to help my-self survive diabetes! I also am interestedbecause the software for my Blood Sugar Mon-itor is WAY out of my reach financially and Ihope this may do some of the same things, evenif I have to enter the information from my mon-itor manually. Thanks!

From the USA: Learn more about interac-tions between dosages of insulin, timing of in-sulin and carbohydrate counting.

From the USA: Get information about the ef-fect of different types and doses of insulin onblood sugar levels.

From the USA: I hope I can balance my car-bohydrate intake to the amount of insulin I amtaking. I may be alerted to the timing of my in-sulin injections.

No country specified: I would like to see asto how it can help me with running a normallife. I have been a diabetic now for 38 years andam beginning to get trouble with my legs andfeet. Also sexual problems. I’m only 55 yearsold thus I’m trying to get some help. Thank youfor your interest.

From the UK: I have type 2 diabetes, and amat the moment on unsuccesful oral medication,I expect to be switching to insulin soon. Anyinformation about the way a body (generic)body reacts is interesting to me.

From Costa Rica: By using it I’ll try to un-derstand better the interaction between differ-ent insulins and the food I eat, and then findthe best dose to maintain the best control pos-sible in my blood glucose levels.

From Canada: I am a patient who was ex-tremely active all of his life. I competed na-tionally and internationally in wrestling andother combative arts (kickboxing only locally).The physical stress is tremendous and tooksome dietary planning even prior to my diag-nosis with diabetes. At 25 years of age I was di-agnosed with Diabetes type 1 Mellitus. I ampretty disciplined and test myself regularly andnever miss my injections. I take 3 Humalog in-jections per day and 1 Humulin at night. I basemy injections on what I eat (and that maychange day to day). I am now 26 and 1/2 (I’vehad diabetes for a little over a year), and it’s

INFORMATION TECHNOLOGY & WEBWATCH482

out of control. At dinner time, based on carbo-hydrate counting I took ten units of insulin, andit took my reading from 12 mmol/L [216mg/dL] at dinner time to 14 mmol/L [252mg/dL] at supper time. For supper, based oncarbohydrate counting again, I took a littlemore to supplement the fact that I was “twice”over normal level, and 5 hours later, I felt pleas-ant and great but my readings were off thescale. I was above 33.3 mmol/L [599 mg/dL].I did quality assurance tests on my glucome-ter, and re-tested myself, but everythingpointed to me being extremely high. I nowdon’t train as much or as heavy because I havetaken extreme lows while exercising heavy. Iam new to this and I hope AIDA can help mewith the following: 1.) Help me take better con-trol of my glucose readings. 2.) Provide infor-mation to predict based on carbohydrate counting, and history of current readings, ap-proximations of how much Humalog to take.3.) How much Humulin (nightime slow acting)to take (bolus rate stuff). 4.) How to predictchanges when I exercise. 5.) How I couldchange my diet to reach normal weight levelsagain (I’ve lost over 10 lbs, and 26 at 120 lbs isbothersome). BASICALLY 6.) Give me some ofmy freedom back through control. I feel like di-abetes is holding me hostage to everything I’veever done.

From Australia: I am having trouble control-ling my blood sugar levels, they are erratic andoften too high or hypo[glycaemic]. My special-ist tells me to do one thing or the other and Iwould like to satisfy myself on the effects ofdifferent insulin regimens. I have been a type1 diabetic for 24 years now and had no troubleuntil about 5/6 years ago when my glucose lev-els started to play up. I hope to learn somethingfrom this programme. Thank you. Kindest Re-gards.

From the USA: Get a better understanding ofthe relationship between insulin dosage andglucometer readings so that I can discuss in-sulin type and dosage more effectively with myinternist for tight control of my diabetes.

From the USA: I have lots of experienceworking with and building simulations of com-plex systems and they have been a wonderfulway to understand the system.

From the USA: Have a finer tuning of the

amount of insulin I need to avoid blood glu-cose levels that are too high or too low.

From the USA: I would like to learn and ap-ply a Pocket PC capable software for diabeticathletes that would enable them to calculateglucose clearance rates, consumption needsand BG [blood glucose] control during longdistance, multiday events where nutrition in-takes are limited to endurance sport type bars,drinks, etc.

From the UK: I am testing my blood sugarabout 6 times a day to reduce HbA1c [glyco-sylated hemoglobin] as much as possible. I amhoping that this simulation gives me somemore insight to better adjust my dosages of In-sulatard and Humalog.

From the USA: Learn more about insulincontrol. I also need to learn more about the diet.As usual everything I like is probably off lim-its. Ha! Thank you for making these thingsavailable.

From the USA: I am an out of control dia-betic, whose doctor is of little help in this mat-ter. I hope your software will help me to con-trol my diabetes.

From Spain: It is a tool that I was searchingfor a long time ago. I think it can help me tocontrol my diabetes and improve my knowl-edge of the actions of the different variables. Iwill do a correct use of this software and I thankyou very much. Yours sincerely.

From Canada: See how (physician-pre-scribed) changes to my insulin dose and regi-men look on the graph. Also, run “what if” sce-narios that I can discuss with my physician ifany look promising.

From the UK: Try and work out the best in-sulin regime. At the moment I just inject Novo-rapid 43 a day and I am hyper[glycaemic]. Pre-viously I injected Humulin I 23 a day and washypo[glycaemic]. I am going to try a mix of thetwo to see what it comes up with.

No country specified: I’d be curious to seehow this software compares to the type pro-vided by the blood glucose meter manufactur-ers.

From Botswana: I download it for a friendwho discovered 2 weeks ago the diabetes. Ionly want to help him. The funniest thing is,he works in a hospital laboratory and did thou-sands of tests for diabetes but he never thought.

INFORMATION TECHNOLOGY & WEBWATCH 483

. . . Anyway I only try to help him with some-thing.

From France: I began a basal-bolus insulintreatment last Friday (October, 5th) and Iwould like to understand the action of my newinsulins better. Thanks a lot.

From the UK: Not quite sure at the mo-ment—only been diabetic for about 3 months.Probably to get a better understanding of theinteraction of diet and insulin and see how itcompares with what I have experienced so far.

From the USA: I’ll use it mostly for educa-tional purposes in learning how to better con-trol my diabetes.

While these comments are clearly from in-terested and motivated patients, it is also in-formative to hear why relatives of patients withdiabetes have been turning to the Internet todownload AIDA:

Downloader comments from relatives of patients

From the USA: I have a foster child diag-nosed with juvenile diabetes. I want her to trythis to see what could happen under certain cir-cumstances.

From Canada: To compare daily results withsimulation, in order to attempt to improvecause/effect relationships and better controlblood sugar levels.

From the USA: Married to a man with 2 di-abetic Type 1 girls. Plan to work with the girlsto educate ourselves in order to start control-ling their diabetes. This software is one of themany resources we will be and are using to ed-ucate us.

From the USA: To do some modeling of mydaughter’s glucose levels and insulin usage. Iwill not be using this to necessarily plan herdosage or carbohydrate intake.

From Canada: Understand the effects ofchanges in diet and insulin. With a better un-derstanding of the interactions, we can staywithin the range better and hopefully findways to lower the total amount of insulin used.

From Belgium: Do some experimenting Ican’t do on living patients. After all I don’twant anyone to get hurt.

From the UK: My father has just been diag-nosed with diabetes. He has begun medicationbut I thought, seeing as he is quite down about

it, I could provide him with some informationto look at. He likes using the computer. It willhopefully keep him busy and provide practicalinformation.

From South Africa: This is actually for mywife who has been a diabetic since she was 12years old. Her specialist told her that she ob-tained a glucometer in the [United] States thatdownloads her sugar readings on a computerand the computer automatically projects agraph based on the old sliding scale. I wouldlove to get her something like that but even ifit were available here in South Africa, we prob-ably wouldn’t be able to afford it. So I wouldlike to see what this programme can do, be-cause if it can help my wife, she will tell herspecialist and we could help a lot more peoplehere in South Africa with it.

From Denmark: Study it and discuss it withfriends and relatives who have diabetic chil-dren and so on. Mostly to learn somethingabout the subject.

From the USA: Use as basic education tool fora family member’s caregiver. Will review withthe patient’s internist overseeing management ofher diabetes to obtain his recommendations.

From Romania: Try to understand workingsof various types of insulin and ask doctor tomanage my results. I have a child 4 years oldand measurements go up and down, almostunpredictable. Thank you.

From Iceland: See if it can be of help for myhusband and son who both have type II dia-betes mellitus.

From Yugoslavia: To help in teaching mychild and to compare with my evaluation of in-sulin application.

However, it is not just patients’ relatives whohave been leaving interesting comments at theWebsite. Healthcare professionals’ reasons fordownloading the software have also been in-formative:

Downloader comments from doctors

From the USA: Want to try it and see whatit can do. Have had various patients come tome with printouts from the program, and ask-ing various questions about diabetes, based onwhat the software has showed, so I thought Ishould check this out myself.

INFORMATION TECHNOLOGY & WEBWATCH484

From India: Check out if AIDA medical software really helps doctors, as my colleaguessay.

From the UK: Firstly I look into the softwareand its level of helping the general public. Ba-sically now there is a very much general pub-lic awareness about diabetes and heart dis-eases. I will teach my patients and the youngdoctors about the levels of insulin and alsoabout its effects.

From Bahrain: I would like first to try it onmyself, as I am diabetic (type 1) on multiple in-jections regimen. Then as a family doctor, I willuse it for educating my type 1 patients.

From the USA: I had gestational diabeteswith both my pregnancies and I’m at risk of developing diabetes. I would like to takecontrol of my diet and watch my glucose levels.

From Canada: Seeing if this might help somepatients . . . seeing how I do in adjusting in-sulin . . . look at the limitation of mixed insulin.

From Estonia: To get a new information tomyself. To get new tips treating and educatingpatients. Maybe get some practical exercises forteaching students.

No country specified: Train and instructnurses as to the proper use of managementskills and time and good practice towards dia-betes patients.

Students’ reasons for downloading the AIDAsoftware have also been interesting:

Downloader comments from students

From Pakistan: Understand the procedureinvolved in diabetes . . . experiment with in-sulin doses . . . help my relatives who are dia-betics, and believe me there are many. . . .

From the USA: I will show it to my class. I’mdoing a presentation on children with diabetesand this software sounds excellent. I reallythink it will help them understand what dia-betes is.

From Greece: I am studying in Greek uni-versity to be a dietician for diabetic patientsand I consider that this software will be veryuseful to me.

Comments from other healthcare profes-sionals—diabetes educators, nurses, and phar-macists—have also been informative:

Downloader comments from diabetes educators

From the USA: I’m recently certified as a di-abetes educator and my job will be expanded.I plan to use this as a tool to practice insulindose changes.

From the USA: Test my brain against this formy problem patients! Educate patients.

Downloader comments from nurses

From the USA: I am planning on viewing thesoftware simulator and evaluating its effec-tiveness for a course I am taking in graduateschool (university).

From the USA: Examine the information tosee if it could be recommended as an adjunctto diabetes education.

From Belgium: My girlfriend is a nurse whospecializes in diabetes patients. I want to showher the software, she might learn from using it.

Downloader comments from a pharmacist

From India: Basically I am a B.Pharm. grad-uate running a WHO GMP standardised fac-tory near Chennai. I have excellent rapportwith medical profession in South India, whereas most of the interiors/towns of South Indialack Internet facilities, therefore I am planninginteractive website for doctors and so that theycan educate the patients through your AIDAsoftware. In fact we are rice-eating populationand the incidence of diabetes is more in SouthIndia. Because of lack of facilities in towns,where few computer, browsing centres are justcoming-up, I thought if you can support me byyour esteemed software I can make doctors touse the simulation for patients benefits, it is notonly a service but 2,000 strong elite doctors arebecoming subscribed members in this pro-posed site, here-in some doctors participate inequities. I just want to initiate the use of yoursoftware by so many medical practitioners whoare unaware in the interior towns of South In-dia. I will be grateful if you can give optionsfor customizing according to the patients con-dition/case profile. Though you are promotingfor patients, but in the present circumstances Iwill ask the doctors to do this for the patientswhich then will catch like a fire. Anyhowthanks for the useful software. Because of this

INFORMATION TECHNOLOGY & WEBWATCH 485

most of the doctors/members will visit yoursite would give way for furthering our ties foruseful services.

Not all the downloaders fitted into the aboveprespecified categories. There follows a com-ment from a downloader who was not a pa-tient, relative, or health-carer—illustrating theuse that people have found for the software,even in schools:

Downloader comment from none of theprespecified categories

From the UK: I am a computer and biologyteacher and am looking for software to im-prove courses and educate pupils on a varietyof topics.

Unfortunately, despite all the informationgiven at the Website, a number of download-ers did seem to miss the point as to what AIDAis all about, as the software cannot provide adatabase for storing blood glucose readings oroffer charting functions:

Downloader comments from people who “missedthe point” about AIDA

From Germany: Hello! Please excuse my badEnglish, but I’m German. I use the diabetessoftware named CAMIT. The reports are verydifficult and my doctor could not understandits reports. I look for better reports. Also I lookfor a possibility to export my list to MicrosoftExcel.

Also AIDA has not been designed for usagewith diabetic pets!:

Usage with pets

From the USA: To try to assist in regulatingmy dog who has diabetes. While it won’t serveas my main guide, it will help in providinganalyses based on the different variables I canplug into the program.

From the USA: I am working with severalmedical schools on various training projects,and would like to see what you have done withthis. As a byline, I have a cat with a serious di-abetes condition. Thus, I rather fall into two ofthe categories below.

Interestingly there were some respondentswho reported having downloaded AIDA before:

Downloader comments from people who weredownloading a new/updated copy of AIDA

From the USA: I have been a user of the soft-ware in the past and it worked to lower myblood sugar from 11 mmol/L [198 mg/dL]down to 6.2 mmol/L [112 mg/dL]. Thank you.

From the UK: Hi again! It’s been a long timesince I was in touch with you but am VERY im-pressed with the recent developments. The on-line version is excellent and saves a lot of timewhen travelling (as basically you don’t have towait till home to try a solution) as well as be-ing well laid out & easy to use. I know we arenot supposed to use this for personal regimenplanning but for my friend it has stopped bu-limia and anorexia and returned her to me fromher “brittle” status two years ago. I really feelthat it gave her her only hope through blind-ness and has allowed her much better recoverythan she could ever have had. It also has al-lowed me to help specialists and nurses andGPs here from mistreating her (I don’t meanshouting or bullying!) I cannot thank youenough.

From Turkey: Examine the nature of this“wonderful” disease. I had learnt about the ef-fective timings of the carbohydrate intake withthe previous DOS version (v4.0). . .and othervariables that affects the blood sugar level.Thank you for making this available for free.

From Denmark: I have used previous ver-sions of AIDA to check simulated results rela-tive to clinical research data and alternativemodel based analysis.

From the USA: I plan to use the AIDA diabetessoftware simulator to help give me an idea abouthow and when my insulin is working in relationto food eaten, and exercise undertaken. I haveused it before and it is a great help to me. Ofcourse I do not go by the simulation alone in re-lation to insulin dosages, but like I said it helpsme to understand certain qualities of different in-sulins in different situations.

DISCUSSION

Generally in medicine there is a growingmovement to involve patients more in theirhealth care. This is particularly the case in di-abetes. Such moves to involve patients, in the

INFORMATION TECHNOLOGY & WEBWATCH486

case of AIDA, involve securing feedback fromusers of the program, which can assist in thefurther refinement and development of thesimulator. A large number of downloads of theAIDA software have been logged on the Inter-net. However, up to now, apart from user tes-timonials about the program15–18—and ad hoccomments received by the system developersvia electronic mail19–21—there has been rela-tively little formal assessment as to who has ac-tually been downloading or making use of thesimulator or, more importantly, why.14

In the section above selected comments leftby visitors at the AIDA Websites have beendocumented. These illustrate the wide varietyof ways in which people from a large numberof different backgrounds expected to make useof the software. Interestingly, in the previouspilot study,14 some of the respondents clearlymissed the point about AIDA, and what it ismeant for. As a result, the information—par-ticularly the caveats—provided at the AIDAWebsite have been strengthened. Since thenthere seem to have been fewer people down-loading the software quoting reasons that arenot supported by the program.

The majority of respondents in this proof-of-concept survey seemed to expect to derivesome benefit from their use of the simulator—and for the correct, intended purpose of theprogram. Their comments also mirror those ofindependent healthcare professionals, as wellas people with diabetes and their relatives, andothers who have actually used the simulator.A range of these comments can be found atwww.2aida.org/reviews on the Web. How-ever, while intuitively the benefits of such aninteractive educational diabetes simulation ap-proach may seem self-evident, as before,14 it isacknowledged that formal evaluation studiesare still required, as for any other medical in-tervention, to demonstrate a definite clini-cal/educational utility for the use of such soft-ware.22–24

For the current survey it was felt to be in-formative to confirm why people are in the firstplace actually turning to the AIDA site anddownloading the program. In this respect, thecurrent study has re-affirmed the feasibility ofusing the Internet to solicit free-text commentsfrom substantial numbers of users/download-

ers of diabetes software. It has also provideduseful and interesting information—highlight-ing that over half the respondents [257 (64.3%)]are people with diabetes or their relatives. Wedo not wish to overinterpret these findings—but it is a recurring observation being made inseparately conducted audits/surveys12,13 thatso many patients and their relatives are turn-ing to the Internet for diabetes-related infor-mation and guidance.

No survey would be complete without someassessment of the limitations of the approach.The most obvious limitation in this study is thefact that, like many surveys, it is based uponself-reported data, although the substantialnumber of responses received do go some wayto offset this. However, a relatively major lim-itation of the current study is that while it of-fers an indication as to why people have beendownloading the software—we do not knowhow much these people have actually usedAIDA. For instance, whether people downloadthe install file and then do nothing with theprogram—or use it a great deal—cannot be for-mally established from a survey conducted, asthis one has been, at the point of download.However, “word of mouth” referrals, and re-peat downloads, as have been overviewedabove, do imply that people are making use ofthe program—and that is why they are tellingothers—and visiting the Website again todownload the latest release of AIDA.

Furthermore, as has been outlined previ-ously elsewhere25 we do have various othermethods of assessing use of the simulator—andall these different indicators need to be con-sidered together to compile an overall per-spective of usage, which does seem to indicatesubstantial on-going interest in the program.

Semi-automated analysis

Some interesting issues for consideration—which may be useful for future surveys—havealso come out of the current semi-automateddata analysis. In this work it was not possibleto automatically identify “missed the point”and “try it and see” replies—as these were not(and in some cases could not be) precoded.Therefore this involved manually reviewingthe data. We found during this work that au-

INFORMATION TECHNOLOGY & WEBWATCH 487

tomated methods are much more robust andamenable to repeat implementation if as muchof the data as possible has been pre-cate-gorised. This implies that planning ahead toclassify data items correctly is of some impor-tance. However, in addition to this there willalways be an interpretation step required, ifone seeks to get the maximum amount possi-ble out of the data that has been collected.

Nevertheless one issue to be considered withtrying to categorise and semi-automate theanalysis of such survey data is that some of thesurvey responses may cover more than one cat-egory and it may well not be possible to “pi-geon-hole” these just one way. Given this, asintimated above, some human review of thedata—albeit subjective—may be necessary sim-ply to extract as much useful information aspossible from the dataset.

Therefore perhaps two levels of analysis areneeded—the first semi-automated, and the sec-ond slightly more subjective. In this respect thehuman review of the dataset obviously permitsgreater flexibility, which in turn allows inter-esting insights and comments to be extractedfrom a larger dataset, in a way that may not beso practicable using a semi-automated method.By contrast the semi-automatic process basi-cally aims to establish as much information aspossible—particularly about respondent de-mographics which do not require any specialinterpretation—with the minimum user/re-searcher interaction.

As such, this report also provides some in-sight into how much it is possible to extrapo-late a survey analysis beyond the bounds of itsoriginal data collection, by interpreting some ofthe free-text comments. Furthermore it showsthat although free-text data collection offers themaximum freedom for the respondent, it doeslead to some issues that need to be consideredwhen it comes to analysing such data, implic-itly placing some constraints on the data anal-ysis/interpretation that can be adopted. In ad-dition some of the responses clearly require anelement of human interpretation to draw use-ful conclusions from what is being said bydownloaders.

In summary, it is a learning point from thisstudy that if the data collection methodologyincorporates sensible data coding into appro-

priate categories at the beginning, prior to thestudy, then more appropriate semi-automatedanalysis becomes possible later, with less “re-searcher involved” steps.

As an aside, it is interesting that a number ofpeople with non–insulin-dependent (type 2) di-abetes mellitus (NIDDM) report downloadingand using the AIDA v4 program—even thoughNIDDM is not explicitly catered for in this ver-sion of the AIDA model. We have observed ina number of previous surveys of user com-ments17,21 that people with NIDDM have beenusing AIDA and report finding it of benefit.This perhaps is not surprising, as to some ex-tent the basic principles of learning about bal-ancing insulin and diet in diabetes are verysimilar in insulin-dependent (type 1) diabetesmellitus and insulin-treated NIDDM.

A future version of AIDA aims to incorpo-rate the option of including endogenous insulinsecretion in the model to improve the realismof the simulations for patients with NIDDM.Further information about this, and future ver-sions of AIDA, can be obtained by sending ablank e-mail note to subscribe@2aida.org tojoin the low volume AIDA registration/an-nouncement list.

Possible self-selection biases

The author has heard a number of times fromdiabetes health-care professionals as to whatthey think patients do, or do not, require froma program like AIDA. These comments, al-though entirely well intentioned, interestinglydo not seem to correspond so well with whatpatients themselves are saying they want orneed from AIDA.

In this respect, one thing that comes out ofthese surveys,12–14 repeatedly, is the desire forinformation that some people with diabetesmellitus, and their relatives, have. Occasionallywe—as health-care professionals—may notfully appreciate this. Although there areclearly, unfortunately, some people with dia-betes who attend diabetes clinics who have lit-tle or no interest in their disease or its compli-cations—there are also many people withconsiderable interest. How self-selected thegroup who visit the AIDA Websites and down-load the AIDA program are, remains difficult

INFORMATION TECHNOLOGY & WEBWATCH488

to assess. But from the absolute number of re-sponses received there are clearly a substantialnumber of people around the world that seemto have interest in this approach.

One issue of importance to us is whether, inthese surveys,12–14 we are simply hearing fromthe relatively enthusiastic and positive down-loaders with less enthusiastic users simply notbothering to reply? We think not, given thelarge numbers of responses received (1,360 inour first download pilot survey in 1999/200012

and 2,437 in our second survey in 2000/2001,13

plus 642 free-text comments in a subset pilotstudy,14 as well as 400 further comments in thiscurrent proof-of-concept study). Therefore,there would appear to be a sizeable number ofenthusiastic AIDA downloaders/users visitingthe AIDA Websites. This observation is sup-ported by the large (and growing) number ofmembers of the opt-in AIDA registration/an-nouncement list. This is accessible directly viawww.2aida.org/register on the Web and cur-rently has .2,400 subscribers.

Given the above, consideration of any possi-ble respondent “bias” would only seem to re-ally affect estimates of the proportion of AIDAdownloaders/users that our surveys represent.(Attempts to extrapolate the findings of a par-ticular audit study to all AIDA downloads dur-ing that period will always have difficulties.)By contrast, the surveys do seem to reveal asubstantial number (in absolute terms) of en-thusiastic downloaders/users. Therefore all wecan say for sure is that there are thousands ofpeople around the world who have been mak-ing use of AIDA, and seeming to find it of ben-efit. However, exactly how many thousands ofactive users is very much harder to establish,given our current survey tools. In this respectall we can easily monitor is the total number ofprogram downloads from the AIDA Websites,which seems to carry on rising month onmonth.

Lastly we may also wish to consider whetherthe responses we are receiving in these surveysare honest and truthful. However, once again,it is the absolute number of positive respon-dents that seems to highlight the honest natureof the replies. In this respect, if we only had asmall number of downloaders, perhaps a fewself-selected respondents could bias the over-

all perspective—but with thousands of repliesbeing received this of course becomes verymuch more difficult—common themes show-ing through in any analysis of the re-sponses.12–14

Obviously there remains much that we needto learn about how to evaluate computer pro-grams like AIDA. However, it is not just aboutrandomised controlled trials (RCTs). WhileRCTs are clearly of importance,22 and remainthe gold standard method of evaluation of sucha program,23,24 we also need to survey users andlearn from their feedback to close theaudit/feedback loop. In this respect under-standing the issues surrounding the analysis ofdata provided by such surveys is of interest andimportance to us, in order to get the most outof such data collection exercises. Clearly simi-lar survey approaches could be adopted forother diabetes software, and we would en-courage other program developers to considerthis for their own applications.

Finally, this study obviously only answerssome questions that we may wish to ask aboutwhy people are continuing to download theAIDA software. Further studies are very clearlyrequired to provide additional insight into us-age of the program. Nevertheless it has beeninteresting to establish how people think theymight benefit from downloading and using theexisting AIDA v4.3a software.

FURTHER TOPICS

If you would like to suggest further topics orWebsites for future “Diabetes InformationTechnology & WebWatch” columns, please e-mail information—with a brief description ofthe site/suggestion—to Dr. E.D. Lehmann:info-www@2aida.org (please write DiabetesWebWatch in the subject line). You can also faxinformation to: (503) 218-0828, quoting Dia-betes Information Technology & WebWatch.

REFERENCES

1. Lehmann ED: Experience with the Internet release ofAIDA v4—an interactive educational diabetes simu-lator. Diabetes Technol Ther 1999;1:41–54.

INFORMATION TECHNOLOGY & WEBWATCH 489

2. Lehmann ED, Deutsch T: A physiological model ofglucose-insulin interaction in type I diabetes mellitus.J Biomed Eng 1992;14:235–242.

3. Lehmann ED: Diabetes moves onto the Internet.Lancet 1996;347:1542.

4. Berger M, Rodbard D: Computer simulation of plasmainsulin and glucose dynamics after subcutaneous in-sulin injection. Diabetes Care 1989;12:725–736.

5. Lehmann ED: Interactive educational simulators indiabetes care. Med Inf (Lond) 1997;22:47–76.

6. Sivitz WI, Davidson PC, Steed D, Bode B, RichardsonP: Computer-assisted instruction in intense insulintherapy using a mathematical model for clinical sim-ulation with a clinical algorithm and flow sheet. Dia-betes Educ 1989;15:77–79.

7. Biermann E, Mehnert H: DIABLOG: a simulation pro-gram of insulin glucose dynamics for education of di-abetics. Comput Methods Prog Biomed 1990;32:311–318.

8. Hedbrant J, Ludvigsson J, Nordenskjold K: Sarimner:a computer model of diabetes physiology for educa-tion of physicians and patients. Diabetes Res ClinPract 1991;14:113–122.

9. Rutscher A, Salzsieder E, Fischer U: KADIS: model-aided education in type I diabetes. Comput MethodsProg Biomed 1994;41:205–215.

10. Plougmann S, Hejlesen OK, Cavan DA: DiasNet—adiabetes advisory system for communication and ed-ucation via the internet. Int J Med Inf 2001;64:319–330.

11. Lehmann ED: Usage of a diabetes simulation systemfor education via the Internet [letter]. Int J Med Inf2003;69:63–69.

12. Lehmann ED: The freeware AIDA interactive educa-tional diabetes simulator—http://www.2aida.org—(1) A download survey for AIDA v4.0. Med Sci Monit2001;7:504–515.

13. Lehmann ED: Who is downloading the freewareAIDA v4.3 interactive educational diabetes simula-tor? An audit of 2,437 downloads. Diabetes TechnolTher 2002;4:467–477.

14. Lehmann ED: Why are people downloading the free-ware AIDA diabetes computing software program: apilot study. Diabetes Technol Ther 2002;4:793–808.

15. Lehmann ED: Preliminary experience with the Inter-net release of AIDA—an interactive educational dia-betes simulator. Comput Methods Prog Biomed 1998;56:109–132.

16. Wilson DM: Diabetes simulators: ready for primetime? Diabetes Technol Ther 1999;1:55–56.

17. Lehmann ED: User experience with the AIDA inter-active educational virtual diabetes patient simulator.Diabetes Technol Ther 2000;2:165–171.

18. Chausmer AB: Diabetes management tools for pa-tients and physicians. Med Software Rev 2002;11(5):1–16.

19. Lehmann ED: Spontaneous comments from users ofthe AIDA interactive educational diabetes simulator.Diabetes Educ 2000;26:633–643.

20. Lehmann ED: Short user comments (‘sound bites’) regarding usage of AIDA v4—http://www.2aida.org—an interactive educational diabetes simulator.Diabetes Technol Ther 2000;2:663–666.

21. Lehmann ED: Further user comments regarding us-age of an interactive educational diabetes simulator(AIDA). Diabetes Technol Ther 2002;4:121–135.

22. Tatti P, Lehmann ED: A randomised-controlled clin-ical trial methodology for evaluating the teaching util-ity of interactive educational diabetes simulators. Di-abetes Nutr Metab 2001;14:1–17.

23. Tatti P, Lehmann ED: Preliminary results from a ran-domised controlled clinical trial for evaluating theteaching utility of an interactive educational diabetessimulator (AIDA) [abstract]. Diabetes 2001;50(Suppl2):A25.

24. Tatti P, Lehmann ED: A prospective randomised-con-trolled pilot study for evaluating the teaching utilityof interactive educational diabetes simulators. Dia-betes Nutr Metab 2003;16:7–23.

25. Lehmann ED: Usage of the www.2aida.org AIDA di-abetes software website: a pilot study. Diabetes Tech-nol Ther 2003;5:75–89.

Address reprint requests to:Dr. Eldon D. Lehmann

c/o www.2aida.org Diabetes SimulatorDevelopment TeamP.O. Box 46104

London, EC2Y 8WN, UK

E-mail: info-www@2aida.org

Web: http://www.2aida.org/lehmann

INFORMATION TECHNOLOGY & WEBWATCH490