View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Measuring Changes in Service in an Established
Telemedicine ProgramElizabeth A. Krupinski, PhD
Arizona Telemedicine Program
University of Arizona
RationaleWhen a telemedicine program first begins, it
is often sufficient to simply report frequencies
of numbers of patients, types of consults, sub-
specialties offered etc. Frequencies are very
useful and can be used as a straight-forward
measure of program success - the higher
the numbers, the greater the success.
For Example From its inception in the 2nd Quarter (Qtr) of 1997
through the 1st Qtr of 2000 the Arizona Telemedicine Program (ATP) conducted 1610 telemedicine consults
60% were Store-Forward (SF) and 40% were Real-Time (RT) interactions
75% were initial and 25% were follow-up consultations
These frequencies give a general idea of what the ATP has done over the past 3 years
RationaleHowever, as a program becomes more
established it is possible to examine these
simple frequencies in more sophisticated ways.
For example, looking at changes over time can
reveal whether consult volume has stabilized or
has it continued to grow, whether certain sub-
specialties are being used more
consistently than others, etc.?
Graphing Graphing is an easy way to illustrate data as a
function of time It is necessary to choose a meaningful unit of
time We typically use yearly quarters
• Provides enough data for statistical analyses• Parallels the seasons, which might affect volume• Easily tracks the Fiscal Year schedule
For Example
0
40
80
120
160
200#
Cas
es
2nd
Qtr
97
3rd
Qtr
97
4th
Qtr
97
1st
Qtr
98
2nd
Qtr
98
3rd
Qtr
98
4th
Qtr
98
1st
Qtr
99
2nd
Qtr
99
3rd
Qtr
99
4th
Qtr
99
1st
Qtr
00
Color coding each year helps make trends stand out * represent points of significant change - see ANOVA panel
*
*
Basic Statistical Analyses There is a wide variety of statistical techniques
that can be used to analyze frequency data Basic Summary Statistics can describe
Central Tendency & Dispersion• These are useful because they give an overall
picture of the data• But, because they summarize the data the time
variable is lost
For Example An examination of the consults provided by the
ATP on a monthly basis reveals:– Mean # consults = 46.00 – Standard Deviation = 21.75– Median = 54.00 – Inter Quartile Range = 35.00– Minimum = 7 consults– Maximum = 87 consults– Skew = -0.342– Kurtosis = -1.023
Advanced Statistical Analyses To examine the data and include the time
variable two other statistical techniques are useful:• Correlation & Regression Techniques• Analysis of Variance (ANOVA) Techniques
More complicated techniques also exist, but the two above can reveal a significant amount of information about changes in data
Regression Plot
0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10 11 12 13
# C
ases
Month 1
Month 2
Month 3
2nd Qtr 3rd Qtr 1st Qtr 97 98 00
QuarterlyBreakdown
Regression Statistics Regression Equation: Y = 12.91 + 4.97 X
• The value of 4.97 indicates the type (+ or -) & degree of slope of the regression line
Correlation Coefficient: r = + 0.782• Values closer to + 1 indicate a strong linear
relationship Conclusion: The ATP has continued to see a
linear increase in the number of consults with each quarter
Data Dispersion & Regression sesty = 13.76 (average dispersion of data
around the regression line) Heteroscedasticity: dispersion around the
regression line is not constant• Dispersion in the middle of the plot is much
greater than at either end• New sites were being added to the network so
there was a period of significant change and adjustment in the number of consults
# Consults by Site
0
20
40
60
80
100
120
140
160
# C
onsu
lts
1997 1998 1999 2000*
Douglas 3-21-99
Ganado 6-5-98
Nogales 5-2-97
Patagonia 1-13-98
Payson 7-9-97
Springerville 12-8-97
Tuba City 5-27-97
White River 3-4-99
DOC 12-4-97
Site & Date ofFirst Consult
* 1st Qtr 2000 only
ANOVA Analysis Initial omnibus F-test compares between
and within variances to determine if there is an overall difference among the quarters
Post-hoc Fisher’s Protected Least Squares Difference Tests determine exactly which quarters differ
Allows for identification of specific points of difference or change
For Example* F = 7.926, df = 11,23 p < 0.0001 2nd Qtr 1998 represents the first significant
increase in ATP case volume 1st Qtr 1999 represents another significant point of
increase in ATP case volume Otherwise case volume has been fairly consistent
since 3rd Qtr 1998
* see quarterly volume bar graph for illustration
# Medical Sub-Specialties Analyzing the number of sub-specialty
consults provided is another valuable measure of program success
Does a program start out offering lots of services then narrow down to a few, does it continue to offer a variety of services over time or is there another pattern of services provided?
ATP Top 10 Sub-Specialties
0
100
200
300
400
500
600
700
800
Der
m
Psy
ch
Car
d
Ort
ho
Neu
ro
Rhe
um
Ob/
Gyn
End
oc
Oto
rhin
Hem
/Onc
# C
onsu
lts
# Sub-Specialties Analysis Basic Statistics
• Mean # sub-specialties / month = 11.86• Standard Deviation = 3.56• Median = 13.00• Inter Quartile Range = 3.75• Minimum = 5• Maximum = 20• Skew = -0.28• Kurtosis = -0.77
Regression Plot
0
5
10
15
20
25
0 1 2 3 4 5 6 7 8 9 10 11 12 13
# S
pec
ialt
ies
Month 1Month 2Month 3
QuarterlyBreakdown
2nd Qtr 3rd Qtr 1st Qtr 97 98 00
# Sub-Specialties / Quarter
02468
1012141618
# Su
b-Sp
ecia
ltie
s
2nd
Qtr
97
3rd
Qtr
97
4th
Qtr
97
1st Q
tr 9
8
2nd
Qtr
98
3rd
Qtr
98
4th
Qtr
98
1st Q
tr 9
9
2nd
Qtr
99
3rd
Qtr
99
4th
Qtr
99
1st Q
tr 0
0
*
*
* significant increase
Sub-Specialty Analyses Regression: Y = 8.021 + 0.576 X
r = 0.554
sesty = 3.05
The fairly flat slope and moderate r-value suggest a constant relation between quarters & # of sub-specialties
ANOVA: F = 3.758, df = 11,23 p = 0.0036• Post-hoc tests: Significant increase in # of sub-
specialties in 1st Qtr 1998 & 1st Qtr 1999
Sub-Specialty Conclusions The ATP has provided consults in 53 different
sub-specialty services Although dermatology and psychiatry are the
services provided most often, the ATP has consistently provided consults in about 13 different* sub-specialties each month since the program’s inception
*The individual sub-
specialties vary each month
Summary The ATP has had a significant and
consistent increase in teleconsult volume since the program began
Numbers from the 1st Qtr 2000 suggest the trend will continue
The ATP has maintained its goal of being a multi-specialty telemedicine provider
This work was supported by
1) US Dept. Agriculture, Rural Utilities Service Distance Learning and Telemedicine Grant2) US Dept. Commerce, National Telecommunications and Information Administration TIIAP Grant3) Office of Rural Health Policy, HRSA Dept. Health & Human Services Rural Telemedicine Grant Program4) The State of Arizona