Embed Size (px)
Citation preview
ava i lab le at www.sc ienced i rec t . com
journa l homepage : www.e lsev ie r . com/ loca te /cose
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8
Expected benefits of information security investments
Julie J.C.H. Ryana,*, Daniel J. Ryanb
aDepartment of Engineering Management and System Engineering, The George Washington University, Washington, DC 20052, USAbInformation Resources Management College, National Defense University, Washington, DC 20319, USA
a r t i c l e i n f o
Article history:
Received 27 January 2005
Revised 11 June 2006
Accepted 3 August 2006
Keywords:
Security
Information security
Attack probabilities
Return-on-investment
Benefits of security investments
a b s t r a c t
Ideally, decisions concerning investments of scarce resources in new or additional proce-
dures and technologies that are expected to enhance information security will be informed
by quantitative analyses. But security is notoriously hard to quantify, since absence of
activity challenges us to establish whether lack of successful attacks is the result of good
security or merely due to good luck. However, viewing security as the inverse of risk en-
ables us to use computations of expected loss to develop a quantitative approach to mea-
suring gains in security by measuring decreases in risk. In using such an approach, making
decisions concerning investments in information security requires calculation of net ben-
efits expected to result from the investment. Unfortunately, little data are available upon
which to base an estimate of the probabilities required for developing the expected losses.
This paper develops a mathematical approach to risk management based on Kaplan–Meier
and Nelson–Aalen non-parametric estimators of the probability distributions needed for
using the resulting quantitative risk management tools. Differences between the integrals
of these estimators evaluated for enhanced and control groups of systems in an informa-
tion infrastructure provide a metric for measuring increased security. When combined
with an appropriate value function, the expected losses can be calculated and investments
evaluated quantitatively in terms of actual enhancements to security.
ª 2006 Elsevier Ltd. All rights reserved.
1. Introduction
Making decisions concerning investments in information se-
curity requires calculation of net benefits expected to result
from the investment. Gordon and Loeb (2002) suggest that
expected loss provides a useful metric for evaluating whether
an investment in information security is warranted. They pro-
pose that since expected loss is the product of the loss v that
would be realized following a successful attack on the systems
comprising our information infrastructure and the probability
that such a loss will occur, one way of accomplishing such
calculations is to consider for an investment i the probabilities
p0 and pi of the losses occurring with and without the
investment, respectively. The expected net benefit of the
investment i is, then,
ENB½i� ¼ p0n� pin� i ¼�p0 � pi
�n� i: (1)
A positive expected net benefit characterizes an attractive
investment opportunity.
In addition to being subject to a number of simplifying as-
sumptions, such as the loss being constant rather than a func-
tion of time, the ability to use this equation depends upon the
ability to obtain probability distributions for information secu-
rity failures. The notion of an information security ‘‘failure’’ is
itself a concept that requires careful consideration. ‘‘Failure’’
may not necessarily mean the catastrophic destruction of in-
formation assets or systems. Failure in this context means
some real or potential compromise of confidentiality, integrity
or availability. An asset’s confidentiality can be compromised
by illicit access even if the integrity and availability of the
* Corresponding author.E-mail addresses: [email protected] (J.J.C.H. Ryan), [email protected] (D.J. Ryan).
0167-4048/$ – see front matter ª 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.cose.2006.08.001
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8580
asset are preserved. Assets may have their integrity compro-
mised even if their confidentiality and availability are un-
changed. Obviously, destruction of an asset compromises its
availability, even when confidentiality and integrity may be
inviolate. Degradation of performance can be a form of failure,
even when the system continues to operate correctly, albeit
slowly. Successful installation of malicious code can be
a form of information security failure even when the mali-
cious code has not yet affected system performance or com-
promised the confidentiality, integrity or availability of
information assets. Consequently, detection of failures for
use in the models discussed in this paper requires that, as
part of our experimental design, we carefully define what
types of failures are to be considered. Nevertheless, it is only
by examining failures that we can begin to understand the
actual security states of our information infrastructures.
Counting vulnerabilities we have patched or numbers of
countermeasures implemented may provide evidence to indi-
rectly reassure us that we are making progress in protecting
our valuable information assets, but only metrics that
measure failure rates can inform us how well we are actually
abating risk. Security is the inverse of risk, and risk is
measured by expected loss.
Unfortunately, little data are available upon which to base
an estimate of the probabilities of failure that are required for
expected loss calculations (Ryan and Jefferson, 2003). Mariana
Gerber and Rossouw von Solms (2001), in describing the use of
quantitative techniques for calculation of annual loss expec-
tancies, say dryly, ‘‘The only factor that was somewhat
subjective was determining the likelihood of a threat mani-
festing.’’ Other authors explicitly or implicitly assume the
availability of these probability distributions in models they
propose to apply to risk management, cost-benefit or return-
on-investment decisions (e.g., see Carroll, 1995; Ozier, 1999;
Wei et al., 2001; Iheagwara, 2004; Cavusoglu et al., 2004, among
others). John Leach (2003) says, ‘‘The data is there to be gath-
ered but it seems we are not in the practice of gathering it. We
haven’t started to gather it because we haven’t yet articulated
clearly what questions we want the gathered data to answer.’’
This paper will develop a mathematical approach to risk man-
agement that will clarify the data that need to be collected,
and will explore methods for non-parametric estimation of
the probability distributions needed for using the resulting
quantitative risk management tools.
2. Failure time distributions
The mathematics of failure time distributions is explored
thoroughly in several texts. (See Kalbfleisch and Prentice,
2002; Collett, 2003; Bedford and Cooke, 2001; Crowder, 2001;
Therneau and Grambsch, 2000 for excellent coverage of the
field.) In the case of information security, the exploration of
time to failure is interesting in that it provides a contextual
basis for investment decisions that is easily understood by
managers and financial officers. Showing an economic benefit
over a given period of time can be used to help them to under-
stand investment strategies that take into account capital
expenditures, risk mitigation, and residual risk in an opera-
tional environment.
If T is a non-negative random variable representing times
of failure of individuals in a homogenous population, T can
be specified using survivor functions, failure functions, and
hazard functions. Both discrete and continuous distributions
arise in the study of failure data.
The survivor function is defined for discrete and continuous
distributions by
SðtÞ ¼ PrðT � tÞ; 0 < t < N: (2)
That is, S(t) is the probability that T exceeds a value t in its
range. S(t) is closely related to several other functions that
will prove useful in risk assessment and management, in-
cluding the failure function F(t), which determines the cumu-
lative probability of failure, and its associated probability
density f(t), and the hazard function h(t), which provides
the instantaneous rate of failure, and its cumulative function
H(t). The way in which these functions relate is determined
as follows.
The failure function or cumulative distribution function
associated with S(t) is
FðtÞ ¼ PrðT < tÞ ¼ 1� SðtÞ: (3)
S(t) is a non-increasing right continuous function of t with
S(0)¼ 1 and limt/NSðtÞ ¼ 0.
The probability density function f(t) of T is
fðtÞ ¼ dFðtÞdt¼ d½1� SðtÞ�
dt¼ � dSðtÞ
dt: (4)
Now f(t) gives the density of the probability at t, and so,
fðtÞdzPrðt � T < tþ dÞ ¼ SðtÞ � Sðtþ dÞ: (5)
for small values of d, providing that f(t) is continuous at t. Also,
f(t)� 0,Z N
0
fðtÞ ¼ 1; and SðtÞ ¼Z N
t
fðsÞds: (6)
The hazard function of T is defined as
hðtÞ ¼ limd/0þPrðt � T < tþ djT � tÞ=d: (7)
The hazard is the instantaneous rate of failure at t of indi-
viduals that have survived up until time t. From Eq. (7) and def-
inition of f(t),
hðtÞ ¼ fðtÞSðtÞ: (8)
Integrating with respect to t
SðtÞ ¼ exp
"�Z t
0
hðsÞds
#¼ exp½�HðtÞ�; (9)
where HðtÞ ¼R t
0 hðsÞds ¼ �log SðtÞ is the cumulative hazard
function. Then
fðtÞ ¼ hðtÞexp½�HðtÞ�: (10)
If T is a discrete random variable, and takes on values
a1< a2</ with a probability given by the function
fðaiÞ ¼ PrðT ¼ aiÞ; i ¼ 1;2;.; (11)
then the survivor function is given by
SðtÞ ¼Xjjaj>t
f�aj
�: (12)
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8 581
The hazard function at ai is the conditional probability of
failure at ai provided that the individual has survived to ai.
hi ¼ PrðT ¼ aijT � aiÞ ¼fðaiÞS�a�i�; where S
�a�i�¼ limt/a�
iSðtÞ: (13)
Then the survivor function is
SðtÞ ¼Y
jjai�t
ð1� hiÞ; (14)
and the probability density function is
fðaiÞ ¼ hi
Yi�1
j�1
ð1� hiÞ: (15)
3. Empirical estimates of survivor andfailure distributions
Failure data are not generally symmetrically distributed,
usually tending to be positively skewed with a longer tail
to the right of the peak of observed failures. Hiromitsu
Kumamoto and Ernest J. Henley (1996) show that if the
data were not censored, we could use an empirical distribu-
tion function to model survivor, failure and hazard
distributions.
Let N be the number of individual systems in the study, and
let n(t) be the number of failures occurring prior to time t.
Then, nðtþ dÞ � nðtÞ is the number of systems that can be
expected to fail during the time interval [t, tþ d). N� nðtÞ is
the number of systems still operational at time t.
An empirical estimator SðtÞ for the survivor function SðtÞ is
given by
SðtÞ ¼ Number of individuals operational at time tNumber of individual systems in the study
¼ N� nðtÞN
:
(16)
So, an empirical estimator FðtÞ for FðtÞ is
FðtÞ ¼ 1� SðtÞ ¼ nðtÞN: (17)
Then, we see that
fðtÞ ¼ dFðtÞdt
xFðtþ dÞ � FðtÞ
d¼ nðtþ dÞ � nðtÞ
dN: (18)
For sufficiently small d, the empirical estimator hðtÞ for the
instantaneous failure rate (hazard function) hðtÞ should ap-
proximately satisfy
hðtÞd¼Number of failures during ½t; tþ dÞ
Number of systems still operational at time t
¼ nðtþ dÞ � nðtÞN� nðtÞ
¼�
dnðtþ dÞ � nðtÞ
dN
���N� nðtÞ
N
�
¼ fðtÞdSðtÞ
:
So, hðtÞ ¼ fðtÞ = SðtÞÞ���
, as we would expect from Eq. (8).
Now, consider the notional system failure data given in
Table 1. The entries in the table are the times, in days follow-
ing initiation of operations, at which the computer systems
we have been tracking were observed to fail due to attacks
on information assets they are creating, storing, processing
or communicating. Thus, two systems fail in the first day,
a second fails on the 19th day, and so on until the final failure
on the 126th day. Graphs of the empirical survivor and empir-
ical failure functions are shown in Fig. 1.
The data in Table 1 consist of failure times due to suc-
cessful attacks on systems being studied. Usually, however,
some of the systems entered into our studies either survive
beyond the time allotted for the study and are still going
strong when we cease to track them, or they fail during
the study period for reasons unrelated to security – perhaps
due to reliability problems not resulting from successful at-
tacks, or due to human errors rather than attacks. In either
case, studies that collect failure time data usually have such
cases, and the systems that survive or that fail for reasons
other than those related to the purpose of the study are
said to be ‘‘right censored.’’ Other types of censoring can
occur as well (see Collett, 2003, pp. 1–3; Kalbfleisch and
Prentice, 2002, pp. 52–54).
Unfortunately, the presence of censored data makes use of
the empirical estimators impossible, because the definitions
of the functions do not allow information provided by a system
for which survival time is censored prior to time t to be used in
calculating the functions at t. Fortunately, other estimators
have been developed that can take advantage of censored
data and provide valid and useful estimations of the functions
we need in determining the advantage of a proposed
investment.
Table 1 – Notional computer system failure data
0.5 0.5 19.5 20 23.5 23.5 25.5 25.5 27.5 27.5
27.5 28.5 29 29 29.5 30 30.5 30.5 31.5 33
33 33 33.5 33.5 34 34.5 34.5 34.5 35.5 35.5
36 36 37 37 37 37.5 37.5 38.5 38.5 38.5
38.5 39.5 39.5 39.5 40 40 40.5 40.5 40.5 41
41 42 42 42.5 42.5 43 43 43 43 44.5
44.5 44.5 44.5 45 45 46 46 46 46.5 46.5
47 48 48.5 48.5 48.5 48.5 49 50 50 50
51 51 52 52 53 53 53.5 54.5 55 56
57 57 59.5 61 100 102 109 121 121 126
Values represent days.
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8582
4. Estimates of survival and failuredistributions with censoring
The 100 entries in Table 2 are notional times at which com-
puter systems we have been tracking were observed to fail
due to attacks on information assets they are creating, storing,
processing or communicating. Thus, two systems fail on the
first day, another fails on the 19th day, and so forth until six
systems remain operational when the experiment is termi-
nated on the 100th day. The six surviving systems are right-
censored. Systems that fail on the 27th, 28th, and the other
entries marked as negative times, are also censored, repre-
senting failures due to causes unrelated to successful attacks
such as, perhaps, reliability failures.
We will explore two estimators that provide useful repre-
sentations of the probability distributions underlying such
data.
5. Kaplan–Meier estimators
One such estimator is the product–limit estimator of the
survivor function derived by Kaplan and Meier (1958) using
the method of maximum likelihood, although the estimator
Survival and Failure Distributions
0
0.2
0.4
0.6
0.8
1
1.2
Days
S(t)
F(t)
0
8.5 17 25 34 42 51 59 68 76 85 93 102
111
119
128
Fig. 1 – Empirical survival and failure functions for system
failure data.
was known and used earlier. Let t0 represents the start of
our study, and let t1< t2</< tk, be a complete list of times
at which individual systems being tracked in the study fail.
To construct this estimator, we construct a series of time
intervals, each of which starts at the time ti at which one
or more systems in the study fails, and ends immediately
before the next time of failure tiþ1. Thus, at least one,
and perhaps several, system(s) fail(s) at exactly the start
of the interval, and no other systems fail during the inter-
val [ti, tiþ1). If individuals are censored at times during the
interval [ti, tiþ1), their censoring does not change the con-
struction of the interval, but should one or more systems
be censored at exactly the start of the interval, we will as-
sume that their censoring actually occurs immediately after
ti, and not at ti.
The number of systems that remain functional just before
time tj is denoted as nj, and dj will represent the number of sys-
tems in the study that fail at tj. The probability of failure at tj is
estimated by dj/nj, while the probability of surviving is
1� ðdj=njÞ ¼ ðnj � djÞ=ðnjÞ. Since there are no deaths between
tj and tjþ1, the probability of survival in that interval is unity,
and the joint probability or surviving the interval from tj to
tjþ1 is ðnj � djÞ=ðnjÞ.We assume for now that the failures of the systems in the
study are independent of one another, so the estimated survi-
vor function at any time t, where tj� t< tjþ1, j¼ 1,2,.,k� 1,
where tkþ1¼N, will be the estimated probability of surviving
beyond tj, which is the probability of surviving through the
jth interval and all the preceding intervals. This is the product
SKMðtÞ ¼Yj
i¼1
ni � di
ni
; for tj � t < tjþ1; j ¼ 1;2;.; k� 1; (20)
and we have SKMðtÞ ¼ 1 when t< t1, and SKMðtÞ ¼ 0 when tk� t.
Actually, if the largest recorded time t* is a censored survival
time, SKM(t) is undefined for t> t*.
Note that if there are no censored failure times in the data
set, then nj � dj ¼ njþ1 in Eq. (20), and
SKMðtÞ ¼Yk
i¼1
niþ1
ni¼ nkþ1
n1: (21)
Since n1¼N, and nkþ1 is the number of systems that survive
beyond tkþ1, the Kaplan–Meier estimate of the survivor func-
tion is identical to the empirical estimator of the survival in
the absence of censored data.
Table 2 – Notional computer system failure data
0.5 0.5 19.5 20 23.5 23.5 25.5 25.5 27.5 27.5
�27.5 28.5 29 �29 29.5 30 30.5 30.5 31.5 33
33 33 33.5 �33.5 34 34.5 34.5 34.5 35.5 35.5
36 �36 37 37 37 37.5 37.5 38.5 38.5 38.5
38.5 39.5 39.5 39.5 40 40 40.5 40.5 40.5 41
�41 42 42 42.5 42.5 43 43 43 43 44.5
44.5 44.5 44.5 45 �45 46 46 46 46.5 46.5
47 48 48.5 48.5 48.5 48.5 49 50 50 50
51 �51 52 52 53 �53 53.5 54.5 55 56
57 7 59.5 61 �100 �100 �100 �100 �100 �100
The sign � indicates censoring. Values represent days.
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8 583
The estimator for the failure function that corresponds to
Eq. (20) is
FKMðtÞ ¼ 1� SKMðtÞ: (22)
It is natural to estimate the hazard function by the ratio of
the number of systems that fail at a given time divided by the
number at risk at that time. More formally, because
HðtÞ ¼ �log SðtÞ, we have
HKMðtÞ ¼ �log SKMðtÞ ¼ �Xj
i¼1
log
ni � di
ni
; for tj � t < tjþ1: (23)
and
hKMðtÞ zHKM
�tj
��HKM
�tj�1
��tjþ1 � tj
� zdj
nj
�tjþ1 � tj
�; for tj � t < tjþ1:
(24)
This equation can be applied to all intervals except the
interval that begins at t
k
, since that interval is infinite in
length.
Returning our attention to the data in Table 2, we set out
the calculations in Table 3 for the Kaplan–Meier estimators
of the survivor, failure and hazard functions. Fig. 2 shows
the Kaplan–Meier estimator for the survivor function.
Table 3 – Kaplan–Meier calculations
j tj tjþ1� tj dj cj (nj� dj)/nj SKM(t) FKM(t) hKM(t) fKM(t)
0 0 0.5 0 0 1 1 0 0.0000 0.0000
1 0.5 19 2 0 0.9800 0.9800 0.0200 0.0011 0.0011
2 19.5 0.5 1 0 0.9897 0.9699 0.0301 0.0206 0.0200
3 20 3.5 1 0 0.9896 0.9598 0.0402 0.0030 0.0029
4 23.5 2 2 0 0.9789 0.9396 0.0604 0.0106 0.0100
5 25.5 2 2 0 0.9785 0.9194 0.0806 0.0109 0.0100
6 27.5 1 2 1 0.9780 0.8992 0.1008 0.0225 0.0202
7 28.5 0.5 1 0 0.9886 0.8890 0.1110 0.0227 0.0202
8 29 0.5 1 1 0.9885 0.8787 0.1213 0.0233 0.0205
9 29.5 0.5 1 0 0.9882 0.8684 0.1316 0.0235 0.0204
10 30 0.5 1 0 0.9881 0.8581 0.1419 0.0238 0.0204
11 30.5 1 2 0 0.9759 0.8374 0.1626 0.0244 0.0204
12 31.5 1.5 1 0 0.9877 0.8270 0.1730 0.0082 0.0068
13 33 0.5 3 0 0.9615 0.7952 0.2048 0.0769 0.0612
14 33.5 0.5 1 1 0.9867 0.7846 0.2154 0.0263 0.0206
15 34 0.5 1 0 0.9863 0.7739 0.2261 0.0267 0.0207
16 34.5 1 3 0 0.9583 0.7416 0.2584 0.0417 0.0309
17 35.5 0.5 2 0 0.9706 0.7198 0.2802 0.0571 0.0411
18 36 1 1 1 0.9848 0.7089 0.2911 0.0147 0.0104
19 37 0.5 3 0 0.9524 0.6752 0.3248 0.0923 0.0623
20 37.5 1 2 0 0.9667 0.6527 0.3473 0.0317 0.0207
21 38.5 1 4 0 0.9310 0.6076 0.3924 0.0678 0.0412
22 39.5 0.5 3 0 0.9444 0.5739 0.4261 0.1071 0.0615
23 40 0.5 2 0 0.9608 0.5514 0.4486 0.0741 0.0409
24 40.5 0.5 3 0 0.9388 0.5176 0.4824 0.1176 0.0609
25 41 1 1 1 0.9783 0.5064 0.4936 0.0204 0.0103
26 42 0.5 2 0 0.9545 0.4834 0.5166 0.0851 0.0411
27 42.5 0.5 2 0 0.9524 0.4603 0.5397 0.0889 0.0409
28 43 1.5 4 0 0.9000 0.4143 0.5857 0.0650 0.0269
29 44.5 0.5 4 0 0.8857 0.3670 0.6330 0.2162 0.0793
30 45 1 1 1 0.9677 0.3551 0.6449 0.0286 0.0102
31 46 0.5 3 0 0.8966 0.3184 0.6816 0.1875 0.0597
32 46.5 0.5 2 0 0.9231 0.2939 0.7061 0.1333 0.0392
33 47 1 1 0 0.9583 0.2816 0.7184 0.0345 0.0097
34 48 0.5 1 0 0.9565 0.2694 0.7306 0.0714 0.0192
35 48.5 0.5 4 0 0.8182 0.2204 0.7796 0.3333 0.0735
36 49 1 1 0 0.9444 0.2082 0.7918 0.0435 0.0091
37 50 1 3 0 0.8235 0.1714 0.8286 0.1500 0.0257
38 51 1 1 1 0.9286 0.1592 0.8408 0.0556 0.0089
39 52 1 2 0 0.8333 0.1327 0.8673 0.1250 0.0166
40 53 0.5 1 1 0.9000 0.1194 0.8806 0.1429 0.0171
41 53.5 1 1 0 0.8750 0.1045 0.8955 0.0769 0.0080
42 54.5 0.5 1 0 0.8571 0.0895 0.9105 0.1667 0.0149
43 55 1 1 0 0.8333 0.0746 0.9254 0.0909 0.0068
44 56 1 1 0 0.8000 0.0597 0.9403 0.1000 0.0060
45 57 2.5 2 0 0.5000 0.0298 0.9702 0.1000 0.0030
46 59.5 1.5 1 0 0.5000 0.0149 0.9851 0.0952 0.0014
47 61 1 0 0.0000 0.0000 1.0000 0.0000 0.0000
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8584
6. Nelson–Aalen estimators
The Nelson–Aalen estimator was first proposed by Nelson
(1969, 1972). Altshuler (1970) also derived the estimator. It
has been shown that the Nelson–Aalen estimate of the sur-
vivor function is always greater than the Kaplan–Meier
estimate at a specified time. For small samples, the Nelson–
Aalen estimator is better than the Kaplan–Meier estimator
(Collett, 2003, p. 22).
The Nelson–Aalen estimator is given by
SNAðtÞ ¼Yk
j¼1
exp��dj=nj
�: (25)
Since expð�dj=njÞz1� ðdj=njÞ ¼ ðnj � djÞ=nj, whenever dj is
small compared to nj, which it is except toward the end
of the study, the Kaplan–Meier estimate given by Eq.
(20) closely approximates the Nelson–Aalen estimate in
Eq. (25).
Then, as usual, we can obtain the Nelson–Aalen failure
function directly from the survivor estimator:
FNAðtÞ ¼ 1� ~SðtÞ: (26)
The cumulative hazard H(t) at time t is, by definition, the
integral of the hazard function. Because HðtÞ ¼ �log SðtÞ, we
find
HNAðtÞ ¼ �log SNAðtÞ ¼Xr
j¼1
dj
nj; (27)
the cumulative sum of estimated probabilities of failure in the
first r time intervals. Since the differences between adjacent
values of HNAðtÞ are estimates of the hazard function hNAðtÞafter being divided by the time interval (Collett, 2003, p. 33).
Thus,
hNAðtÞ ¼ dj
nj
�tjþ1 � tj
�; (28)
exactly as in Eq. (24).
The calculations for the Nelson–Aalen estimators for the
data from Table 2 are shown in Table 4, and a graph of the
survivor function is shown in Fig. 3. To obtain the probability
density functions associated with the Kaplan–Meier and
Nelson–Aalen estimators, we can use Eq. (8), or Eq. (16).
The Kaplan-Meier Estimate of the Survivor Function
0
0.2
0.4
0.6
0.8
1
1.2
0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96
Time in days
S(t)
Fig. 2 – The Kaplan–Meier estimator for the survivor func-
tion underlying the data in Table 2.
7. The advantage of an investmentin information security
We hope and expect, of course, that an investment in infor-
mation security will provide us with some advantages. If it
did not, we would be foolish to make the investment. More
specifically, we expect that an investment in information se-
curity will result in greater freedom from successful attacks
on the systems being protected, so that the systems survive
longer before succumbing to an attack. Since no security is
perfect, we know that eventually the systems will succumb,
but our investment should delay that time.
The result of an investment should, then, be to move the
survivor curve to the right. Suppose that we track 100 systems
which are protected by an investment in additional informa-
tion security that mirrors the investment proposed for our
information infrastructure. This might occur contemporane-
ously with the gathering of the data in Table 2, so that the sys-
tems represented in Table 2 represent a control group. Thus,
in the enhanced group, one system fails on the first day,
another fails on the fifth day, and so forth until 34 systems
remain operational when the experiment is terminated on
the 100th day. The 34 surviving systems are right-censored.
Systems that fail on the 65th, 93rd, and the other entries
marked as negative times, are also censored, representing
failures due to causes unrelated to successful attacks such
as, perhaps, reliability failures.
Tracking the enhanced systems might also take place later,
and could even be the same systems as were followed in pre-
paring Table 2, suitably repaired following the attacks that
provided the data in Table 2 and enhanced according to the
proposed investment. In any event, suppose that following
protection of the new set of systems with the improved secu-
rity, we observe the attacks described by the failure times in
Table 5. Fig. 4 shows the survivor curve S0 that we experience
without the investment (from Table 2), and the survivor curve
Si that occurs following the investment (from Table 5). The
benefit produced by our investment is the area between the
curves.
Thus, the advantage we gain from our investment i is
given by
AðiÞ ¼Z N
0
SiðtÞ �Z N
0
S0ðtÞ ¼ Ei½T� � E0½T�; (29)
since, as is well known,
E½T� ¼Z N
0
tf ðtÞdt ¼Z N
0
SðtÞdt: (30)
Of course, it may be that SiðtÞ � S0ðtÞ is not always true. The
two curves may cross one or more times, but if AðiÞ > 0 the
benefits of the investment will eventually outweigh any short
term detriments. If, however, we restrict our attention to
the near term, say one year or a few years, it is possible that
AðiÞ > 0 but the value ofR t
0 SiðtÞ �R t
0 S0ðtÞ is less than zero for
t restricted to the period of interest.
Eq. (29) is a useful metric for measuring the advantage
of an investment, but it still fails to address the impact of
a successful attack on those information assets that have
their confidentiality, integrity or availability compromised
at the observed failure times. We know that the impact of
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8 585
Table 4 – Nelson–Aalen calculations
j tj dj nj exp(�dj/nj) SNA(t) FNA(t) hNA(t) fNA(t)
0 0 0 100 1.0000 1.0000 0.0000 0.0000 0.0000
1 0.5 2 98 0.9798 0.9798 0.0202 0.0011 0.0011
2 19.5 1 97 0.9897 0.9697 0.0303 0.0206 0.0200
3 20 1 96 0.9896 0.9597 0.0403 0.0030 0.0029
4 23.5 2 94 0.9789 0.9395 0.0605 0.0106 0.0100
5 25.5 2 92 0.9785 0.9193 0.0807 0.0109 0.0100
6 27.5 2 89 0.9778 0.8989 0.1011 0.0225 0.0202
7 28.5 1 88 0.9887 0.8887 0.1113 0.0227 0.0202
8 29 1 86 0.9884 0.8784 0.1216 0.0233 0.0205
9 29.5 1 85 0.9883 0.8682 0.1318 0.0235 0.0204
10 30 1 84 0.9882 0.8579 0.1421 0.0238 0.0204
11 30.5 2 82 0.9759 0.8372 0.1628 0.0244 0.0204
12 31.5 1 81 0.9877 0.8269 0.1731 0.0082 0.0068
13 33 3 78 0.9623 0.7957 0.2043 0.0769 0.0612
14 33.5 1 76 0.9869 0.7853 0.2147 0.0263 0.0207
15 34 1 75 0.9868 0.7749 0.2251 0.0267 0.0207
16 34.5 3 72 0.9592 0.7433 0.2567 0.0417 0.0310
17 35.5 2 70 0.9718 0.7224 0.2776 0.0571 0.0412
18 36 1 68 0.9854 0.7118 0.2882 0.0147 0.0105
19 37 3 65 0.9549 0.6797 0.3203 0.0923 0.0627
20 37.5 2 63 0.9688 0.6585 0.3415 0.0317 0.0209
21 38.5 4 59 0.9345 0.6153 0.3847 0.0678 0.0417
22 39.5 3 56 0.9478 0.5832 0.4168 0.1071 0.0625
23 40 2 54 0.9636 0.5620 0.4380 0.0741 0.0416
24 40.5 3 51 0.9429 0.5299 0.4701 0.1176 0.0623
25 41 1 49 0.9798 0.5192 0.4808 0.0204 0.0106
26 42 2 47 0.9583 0.4976 0.5024 0.0851 0.0423
27 42.5 2 45 0.9565 0.4759 0.5241 0.0889 0.0423
28 43 4 41 0.9070 0.4317 0.5683 0.0650 0.0281
29 44.5 4 37 0.8975 0.3875 0.6125 0.2162 0.0838
30 45 1 35 0.9718 0.3766 0.6234 0.0286 0.0108
31 46 3 32 0.9105 0.3429 0.6571 0.1875 0.0643
32 46.5 2 30 0.9355 0.3207 0.6793 0.1333 0.0427
33 47 1 29 0.9661 0.3099 0.6901 0.0345 0.0107
34 48 1 28 0.9649 0.2990 0.7010 0.0714 0.0213
35 48.5 4 24 0.8465 0.2531 0.7469 0.3333 0.0844
36 49 1 23 0.9575 0.2423 0.7577 0.0435 0.0105
37 50 3 20 0.8607 0.2086 0.7914 0.1500 0.0313
38 51 1 18 0.9460 0.1973 0.8027 0.0556 0.0110
39 52 2 16 0.8825 0.1741 0.8259 0.1250 0.0218
40 53 1 14 0.9311 0.1621 0.8379 0.1429 0.0232
41 53.5 1 13 0.9260 0.1501 0.8499 0.0769 0.0115
42 54.5 1 12 0.9200 0.1381 0.8619 0.1667 0.0230
43 55 1 11 0.9131 0.1261 0.8739 0.0909 0.0115
44 56 1 10 0.9048 0.1141 0.8859 0.1000 0.0114
45 57 2 8 0.7788 0.0889 0.9111 0.1000 0.0089
46 59.5 1 7 0.8669 0.0770 0.9230 0.0952 0.0073
47 61 1 6 0.8465 0.0652 0.9348 0.0000 0.0000
a successful attack on an information asset varies over
time. A compromise of the confidentiality of the planned
landing on Omaha Beach on D-day would have had
enormous impact on June5, 1944, and a negligible impact
on June 7th. The impact of a successful attack on integrity
is much larger following creation and prior to making and
storing a backup of an information asset than the impact
that would result from a compromise after the backup
is safely stored. Such a collapse of the loss function is
characteristic of information security, although whether
the decline takes place very rapidly, as in the D-day
example, or degrades gradually over time, depends upon
circumstances.
Alternatively, consider the loss functions associated with
an organization that keeps customer accounts in a database
that is updated once per day, say at midnight, based on
a transactions file accumulated throughout the day. The loss
function for the database itself is constant throughout the
day:
0; prior to midnight and the creation of the databasen ¼ n; between midnight and the following midnight0; after midnight when the database has been replaced
by an updated database
:
(31)
If a duplicate copy of the accounts database is made con-
currently with the following midnight’s update, and stored
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8586
securely offsite, the exposure for destruction or corruption of
the accounts database can be as little as the cost of recovering
and installing the backup – much less than the cost of creating
the database from scratch. The loss function for the transac-
tion file, on the other hand, is a sawtooth function, the value
of which is zero at midnight and increases monotonically as
transactions accumulate throughout the day, but which
returns to zero value the following midnight when the trans-
actions are used to update the accounts database and a new
transactions file is created for the following day. The loss
function nðtÞ for all the information assets contained in our in-
formation infrastructure is, of course, a sum of the individual
loss functions for each asset.
Our estimation processes have provided us with the proba-
bility densities f0 and fi we need, so, given a loss function nðtÞrepresenting the loss we would experience from a successful
attack on our unimproved infrastructure as a function of
time, the expected loss at t without the proposed investment is
E0½n� ¼Z N
0
nðtÞf0ðtÞdt; (32)
and, similarly, for the expected loss following making the pro-
posed investment in information security,
Ei½n� ¼Z N
0
nðtÞfiðtÞdt: (33)
Since we expect the loss to be less following our invest-
ment, the expected benefit from our investment is
The Nelson-Aalen Estimate of the Survivor Function
0
0.2
0.4
0.6
0.8
1
1.2
0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96
Time in days
S(t)
Fig. 3 – The Nelson–Aalen estimator for the survivor func-
tion underlying the data in Table 2.
E0 nðtÞ �Ei nðtÞ > 0�½�½ , and the expected net benefit is
BðiÞ ¼ E0½nðtÞ� � Ei½nðtÞ� � i: (34)
Of course, if nðtÞ ¼ n is constant, then
E0½nðtÞ� � Ei½nðtÞ� ¼ E0½n� � Ei½n� ¼ n� n ¼ 0;
so BðiÞ ¼ �i, confirming our intuition that no security is ever
perfect and eventually a compromise will occur. But, if our in-
vestment i is such that the survivor curve moves sufficiently
far to the right, then the occurrence of a successful attack
could be delayed beyond the collapse of the loss function. As
SiðtÞ moves to the right, so does fi(t). If tc is the time at which
the loss function collapses, then, since nðtÞ ¼ 0 for t > tc,
E0½nðtÞ� � Ei½nðtÞ� ¼Z tc
0
nðtÞf0ðtÞdt�Z N
tc
nðtÞf0ðtÞdt
�"Z tc
0
nðtÞfiðtÞdt�Z N
tc
nðtÞfiðtÞdt
#
¼Z tc
0
nðtÞf0ðtÞdt�Z tc
0
nðtÞfiðtÞdt : ð35Þ
Then, if fi has moved to the right, making fi(t) small when
t< tc, the second integral is small, and
BðiÞ¼ E0½nðtÞ� � Ei½nðtÞ� � i
z
Ztc
0
nðtÞf0ðtÞdt� i: ð36Þ
Thus, the collapse of the loss function will make our invest-
ment worthwhile if i is less than the near-term loss expected if
the investment is not made.
8. Future research
To enable the use of expected loss in these mathematical
models for evaluation of proposed investments in information
security, and the use of the metrics in tracking the evolution of
security in information infrastructures as investments are
implemented, more research is needed.
Studies of failure time data can involve epidemiological
studies of the entire information infrastructure, or can use
a cross-sectional study of a representative sub-infrastructure
based on concurrent measurements of samples of individual
systems followed prospectively (a cohort study), or of several
Table 5 – Notional computer system failure data following an investment i in information security
0.5 5.5 26.5 57 57.5 60 60 62 62 65.5
�65.5 67.5 68.5 70.5 72.5 72.5 72.5 72.5 72.5 75
75.5 77 77 78 78 79 80 80 80.5 81
81 82.5 82.5 84.5 84.5 84.5 85.5 85.5 86 86
87.5 87.5 87.5 89 89 89 91 91.5 91.5 91.5
93 93 93.5 �93.5 94.5 94.5 96 96 �96 97.5
98 98 99 99 99 99 �100 �100 �100 �100
�100 �100 �100 �100 �100 �100 �100 �100 �100 �100
�100 �100 �100 �100 �100 �100 �100 �100 �100 �100
�100 �100 �100 �100 �100 �100 �100 �100 �100 �100
The sign � indicates censoring. Values represent days.
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8 587
samples with retrospective measurements (a case–control
study). Alternatively, we can use a randomized controlled
trial to study two or more parallel, randomized cohorts of
systems, one of which (the control group) receives no
security enhancements, and the others of which are en-
hanced by a proposed investment or investments in infor-
mation security. Such studies should be undertaken to
evaluate the overall utility of various approaches to using
failure time data in an information security environment,
and addressing a wide variety of different practices and
technologies to determine if this approach is more effective
in some settings or for some types of practices or
technologies.
9. Conclusion
Too often information security investment decisions are
based on criteria that are at best qualitative, and at worst little
more than fear, uncertainty and doubt derived from anecdotal
evidence. Security is the inverse of risk. Because security is at
its best when nothing happens, it is notoriously difficult to
measure. But we can measure risk by calculating expected
loss, and a reduction in expected loss is a measure of the
change in security posture that accrues to an information
infrastructure following an investment in a new security
practice or technology. Unfortunately, there are little data
available to allow us to understand the probabilities of
successful attacks that we need in order to calculate expected
loss.
We sought a way to obtain the probabilities needed so we
can actually use the expected net benefit Eq. (1). By collecting
data on the experiences of two separate populations, one
which has the benefit of the investment and the other – a con-
trol group – which does not, we can compute the Kaplan–
Meier or Nelson–Aalen estimates of the probability density
functions with and without the investment. Having the prob-
abilities, and knowing the value of our information assets, we
can calculate the respective expected losses, and to use those
Survivor Functions Before and
After an Investment i
0
0.2
0.4
0.6
0.8
1
1.2
Time
Pro
bab
ility
S0(t)
Si(t)
0 7.5 15 22.5 30 37
.5 45 52.5 60 67
.5 75 82.5 90 97
.5
Fig. 4 – The survivor function S0(t) before an investment and
Si(t) following an investment i in Information security.
expected values to measure the advantage that will be
realized from the investment. We will then be able to make
an informed decision as to whether a proposed investment
is wise. Implemented in an operational environment, this
method can change the way security investments are consid-
ered and made.
r e f e r e n c e s
Altshuler B. Theory for the measurement of competing risks inanimal experiments. Mathematical Bioscience 1970;6:1–11.
Bedford Tim, Cooke Roger. Probabilistic risk analysis: foundationsand methods. Cambridge, UK: Cambridge University Press;2001.
Carroll John M. Information security risk management. In:Hutt Arthur, et al., editors. Computer security handbook.3rd ed. NY: John Wiley & Sons; 1995. p. 3.1–320.
Cavusoglu Huseyin, Mishra Birendra, Raghunathan Srinivasan. Amodel for evaluating it security investments. Communica-tions of the ACM 2004;47(7):87–92.
Collett David. Modelling survival data in medical research.2nd ed. Boca Raton: Chapman & Hall/CRC; 2003.
Crowder Martin. Classical competing risks. Washington, DC:Chapman & Hall/CRC; 2001.
Gordon Lawrence A, Loeb Martin P. The economics of informationsecurity investment. ACM Transactions on Information andSystem Security November 2002;5(4):438–57.
Gerber Mariana, von Solms Rossouw. From risk analysisto security requirements. Computers and Security 2001;20:580.
Iheagwara Charles. The effect of intrusion detection manage-ment methods on the return on investment. Computers andSecurity 2004;23:213–28.
Kalbfleisch John D, Prentice Ross L. The statistical analysis offailure time data. 2nd ed. Hoboken, NJ: John Wiley & Sons;2002.
Kaplan EL, Meier P. Nonparametric estimation from incompleteobservations. Journal of the American Statistical Association1958;53:457–81.
Kumamoto Hiromitsu, Henley Ernest J. Probabilistic risk assess-ment for engineers and scientists. 2nd ed. New York: IEEEPress; 1996. p. 266ff.
Leach John. Security engineering and security ROI. p. 2, <http://www.compseconline.com/free_articles/cose_22_6.pdf>; 2003[accessed 7/22/2004].
Nelson Wayne. Hazard plotting for incomplete failure data.Journal of Quality Technology 1969;1:27–52.
Nelson Wayne. Theory and applications of hazard plotting forcensored failure data. Technometrics 1972;14:945–65.
Ozier Will. Risk analysis and assessment. In: Tipton Harold F,Krause Micki, editors. Information security managementhandbook. 4th ed. Boca Raton: Auerbach; 1999. p. 247–85.
Ryan Julie J.C.H., Jefferson Theresa I. The use, misuse, andabuse of statistics in information security research.Managing technology in a dynamic world. In: Proceedings ofthe 2003 American society for engineering managementconference, St. Louis, Missouri, October 15–18, 2003.p. 644–53.
Therneau Terry M, Grambsch Patricia M. Modeling survival data:extending the Cox model. New York: Springer; 2000.
Wei Huaqiang, Frinke Deb, Carter Olivia, Ritter Chris, 2001. Costbenefit analysis for network intrusion detection systems. In:CSI 28th annual computer security conference, Washington,DC, <http://wwwcsif.cs.ucdavis.edu/%7Ebalepin/new_pubs/costbenefit.pdf> [accessed 7/22/2004].
c o m p u t e r s & s e c u r i t y 2 5 ( 2 0 0 6 ) 5 7 9 – 5 8 8588
Julie JCH Ryan is a member of the faculty at George Washing-
ton University in Washington, DC. Earlier she served as Pres-
ident of the Wyndrose Technical Group, Inc., a company
providing information technology and security consulting ser-
vices. She was also a Senior Associate at Booz Allen & Hamil-
ton and a systems analyst for Sterling Software. In the public
sector she served as an analyst at the Defense Intelligence
after beginning her career as an intelligence officer after grad-
uating from the Air Force Academy. She holds a Masters from
Eastern Michigan University and a D.Sc. from George Wash-
ington University.
Daniel J. Ryan is a Professor at the Information Resources
Management College of the National Defense University
in Washington, DC. Prior to joining academia, he served
as Executive Assistant to the Director of Central Intelli-
gence, and still earlier as Director of Information Security
for the Office of the Secretary of Defense. In the private
sector, he served as a Corporate Vice President of SAIC
and as a Principal at Booz Allen & Hamilton. He holds
a Masters in mathematics from the University of Maryland,
an MBA from California State University, and a JD from the
University of Maryland.
