RE 2012

A cross-domain empirical study and legal evaluationof the requirements water marking method

David G. Gordon • Travis D. Breaux

Received: 6 November 2012 / Accepted: 20 March 2013 / Published online: 4 April 2013

� Springer-Verlag London 2013

Abstract Companies that own, license, or maintain per-

sonal information face a daunting number of privacy and

security regulations. Companies are subject to new regu-

lations from one or more governing bodies, when compa-

nies introduce new or existing products into a jurisdiction,

when regulations change, or when data are transferred

across political borders. To address this problem, we

developed a framework called ‘‘requirements water mark-

ing’’ that business analysts can use to align and reconcile

requirements from multiple jurisdictions (municipalities,

provinces, nations) to produce a single high or low standard

of care. We evaluate the framework in two empirical case

studies covering a subset of U.S. data breach notification

laws and medical record retention laws. In these studies,

applying our framework reduced the number of require-

ments a company must comply with by 76 % across 8

jurisdictions and 15 % across 4 jurisdictions, respectively.

We show how the framework surfaces critical requirements

trade-offs and potential regulatory conflicts that companies

must address during the reconciliation process. We sum-

marize our results, including surveys of information tech-

nology law experts to contextualize our empirical results in

legal practice.

Keywords Legal requirements � Requirements

comparison � Requirements reconciliation � Conflicts

1 Introduction

Information systems are increasingly leveraging third-party

services for data processing and storage. These third party

services provide economies of scale that allow companies

with minimal infrastructure to provide rich consumer

experiences at relatively low cost. The emerging com-

modification of ‘‘software as a service’’ amplifies this

phenomenon: Google Maps, Facebook, LinkedIn, and

PayPal provide mapping, social network, and payment-

processing services, to name a few, that can be packaged

by third-parties into new consumer services. Composing

software from services in this new ecosystem amplifies an

old challenge: How do business analysts identify those

system requirements that govern their software in the

presence of trans-border data flows? This problem has

received attention from government and industry with

regard to privacy and security regulation [19, 31, 33].

While successful large companies can assemble global,

interdisciplinary legal and product teams, small companies,

and start-ups, in particular, frequently lack the resources to

resolve this issue through legal guidance alone.

Consider an example scenario with data transfers across

multiple jurisdictions in the United States. A New York

State resident, while visiting relatives in Nevada State,

accesses an online web account she has with a Wisconsin-

based business. The business stores her data using a cloud

service provider (CSP) that maintains the data in their

Connecticut State facility. Each ‘‘step’’ in this data flow

must address provincial laws that govern data access,

retention, and breach notification. The laws are triggered

by legal conditions, such as the geographical location of

the business and data (Wisconsin, Nevada and Connecti-

cut), as well as the location and legal residence of the data

subject (Nevada, New York). These laws are written in

D. G. Gordon (&)

Engineering and Public Policy, Carnegie Mellon University,

Pittsburgh, PA, USA

e-mail: dggordon@cmu.edu

T. D. Breaux

Institute for Software Research, Carnegie Mellon University,

Pittsburgh, PA, USA

e-mail: tdbreaux@cs.cmu.edu

123

Requirements Eng (2013) 18:147–173

DOI 10.1007/s00766-013-0167-6

semi-isolation: in some cases, a law may borrow require-

ments from another, jurisdiction, which is frequently

observed in U.S. data breach notification laws; in other

cases, regulators may compete with other jurisdictions by

‘‘racing’’ to the top or bottom of best practice, such as the

recent India privacy regulations that established stronger

consent requirements than the European Union. While our

examples in this paper are limited to U.S. regulations, the

scope of this problem affects many industrialized countries

worldwide.

We introduce an empirically validated framework that

business analysts can use to reconcile regulatory require-

ments from multiple jurisdictions into a single standard of

care. This reconciliation method, called requirements water

marking, allows an analyst to establish a high- or low-water

mark standard across two or more jurisdictions. The

framework preserves traceability so that a business analyst

can trace observed similarities and differences from

requirements to specific sentences and phrases in the law.

The collection of requirements produced by the framework

can then be further evaluated by legal counsel and experts

familiar with regional legal practices.

We developed and validated our framework in two

empirical case studies. The first study examines U.S. data

breach notification laws (DBNL) that were enacted during

the past 8 years and have effectively created a U.S.

nationwide information system that sends messages (noti-

ces) to consumers and regulatory agencies when a com-

pany discovers a breach of consumer data. While these

laws support legacy systems for sending notices (e.g.,

telephone, postal mail, etc.), they also permit using elec-

tronic notices and many describe functional security

requirements. The benefit of this new system is increased

information sharing of emerging security threats and vul-

nerabilities. The cost of this system, however, is that

products that cross U.S. state lines must address the legal

requirements contained in these laws. The second study

examines U.S. medical record retention (MRR) laws. In the

U.S., there is a movement toward a national health infor-

mation network that aims to enable sharing health infor-

mation across state borders and institutions. There are also

incentives to facilitate the transition from paper-based to

electronic health records, called Meaningful Use, that are

organized by the U.S. Department of Health and Human

Services (see 45 C.F.R. 170 for the Stage 2 final rule).

However, companies that aim to sell a common medical

record system across state lines must contend with the

variety and potentially conflicting medical record retention

laws.

The remainder of this paper is organized as follows: in

Sect. 2, we discuss related work; in Sect. 3, we introduce

the framework, including the new water mark method; in

Sect. 4, we introduce our case study design that we used to

validate our framework; in Sect. 5, we discuss the design of

our legal expert survey to validate the water marking

method; in Sect. 6, we present our summary findings from

the two case studies; in Sect. 7, we provide a discussion of

multi-jurisdictional conflicts discovered by the process

presented in Sect. 6, along with differences between

domains that affect application of the water marking

method; in Sect. 8, we discuss threats to validity; and in

Sect. 9, we report on legal expert interviews and separate

surveys to compare our results with legal practice. Finally,

we summarize and discuss future work in Sect. 10.

2 Related work

The role of regulations in legal requirements has been a

continuing topic of research [22]. We consider three related

work topics: techniques for extracting requirements from

legal texts, methods for comparing requirements to find

similarities and differences, and research on the legal

requirements semantics that have logical implications for

reconciling differences across legal requirements sets. We

note differences between our contribution and prior work.

Regulations and laws often conform to a stylized subset

of natural language. Breaux introduced a frame-based

method for systematically extracting requirements from

legal texts [6]. The method includes validated phrase

heuristics and a legal ontology that significantly improve

requirements extraction by human analysts over traditional

methods (p \ 0.001) [6]. Based on this method, Breaux

and Gordon developed a legal requirements specification

language (LRSL) to assist analysts with the framework by

formatting extracted requirements in a standard notation

[5]. Gordon and Breaux combined the LRSL with a set of

qualitative metrics [BAB08] to develop a multi-jurisdic-

tional analysis framework that was applied to data breach

notification laws [16]. In this paper, we describe new

framework validation that includes an additional case study

in a new domain, medical record retention law, and an

additional survey of legal experts to compare the water

marks to legal practice. In addition, we report an extension

to the framework to resolve a new type of conflict that was

discovered during our extended validation.

In order to compare requirements across jurisdictions,

analysts must compare textual requirements pairs to iden-

tify similarities and differences. Prior work to automati-

cally identify equivalent requirements includes research in

applied information retrieval (IR) [10, 36], and machine-

learning [18]. Falessi et al. [10] conducted an empirical

evaluation of multiple IR-based NLP techniques to identify

equivalent requirements pairs. The evaluation compares

different algebraic models, weighting and similarity met-

rics, and term extraction methods. The results found the

148 Requirements Eng (2013) 18:147–173

123

‘‘ideal’’ best technique is a vector-space model with the

Cosine similarity metric, linear weighting, and a Stanford

part-of-speech noun and verb extractor. We evaluated this

technique on our dataset and discuss the results in Sect. 10.

Enhancements to IR-based techniques, such as project

glossaries [36] and machine-learning [13, 18], or multi-

word abstractions [12], may provide better automation to

assist analysts with this step in our process. In particular,

machine-learning methods that rely on training sets [18]

are likely to show promise in multi-jurisdictional analysis

over successive jurisdictions when comparing regulations

from the same domain. Other works on regulatory analysis

and comparison, such as those by Amyot et al., do not

make direct comparisons between regulatory texts, but

between goal models generated from the text [14, 25].

Amyot’s framework also relies on business process models

in order to determine compliance, which is beyond the

scope of this work. We discuss this issue of scalability in

Sect. 6.

Dekhtya et al. [9] studied human performance in

tracing requirements to system tests. They found that no

single analyst was able to achieve the gold standard,

which was the ideal solution, whereas the combined effort

of all analysts did find all traces in the standard. We

believe tracing requirements to test cases (or source code)

is a conceptually different problem than comparing tex-

tual requirement pairs for similarity. To assist human

analysts, we developed metrics that measure types of

differences between requirements [4]. These metrics are

used to measure terms and phrases that conceptually

subsume the meanings of other terms and phrases, or

dissimilar phrases that correspond to changes in modality

(must, should, may). In addition to creating a ‘‘link’’

between two similar requirements, these metrics lead an

analyst to rationalize and explain the similarity or dif-

ference that they observe.

Maxwell and Anton introduce a taxonomy of legal

cross-references to identify conflicting requirements [20].

Cross-references are explicit phrases that appear in reg-

ulations and serve to link regulatory requirements within

and across regulations. These links encode a specific

semantic relationship, such as reusing a previously stated

definition, conferring a priority to reconcile potential

conflicts, or refining a requirement by describing required

or recommended implementation strategies [6]. In our

approach, we encode both explicit cross-references and

implied links between requirements in our LRSL to

identify relational dissimilarities. However, our metrics

further identify phrase differences between requirements

that are not encoded in cross-references. These compari-

sons are similar to work in model merging that examined

inter- and intra-model properties before performing a

merge [26].

3 Water marking framework

The water marking framework process overview appears in

Fig. 1 and consists of three steps performed manually by a

human analyst. Arrows lead from inputs/outputs to each

step, which are individually numbered: (1) The analyst

extracts and encodes requirements from two regulatory

documents R and S using a machine-readable LRSL that is

parsed to yield itemized requirements; (2) the analyst

conducts a gap analysis to compare requirements pairs

across the two requirements sets to yield dissimilarity

measures; and (3) the analyst applies the water marking

constructs (union, disjoint, and minimum) to identify and

reconcile consensus and conflict across these measures and

conducts a post-water marking de-confliction process.

The framework combines and extends prior work to

enable the water marking method. Step 1 is based on the

frame-based requirements analysis method by Breaux [6],

which is implemented in a legal requirements specification

language (LRSL) to improve repeatability [5]. Step 2

extends metrics previously applied in an industry case

study to compare regulatory requirements with product

requirements in the domain of accessibility [4]. We pre-

viously introduced a new metric (SE-PE) and validated the

metric in a new domain (data breach notification), in

addition to introducing Step 3 [16]. In this paper, we

introduce a new process to step 3 to conduct post-water

marking de-confliction. We now briefly describe steps 1–3,

before introducing our new process.

Fig. 1 Overview of regulatory water mark construction

Requirements Eng (2013) 18:147–173 149

123

3.1 Extracting requirements

In Step 1, the analyst copies text directly from a regulatory

document into a legal requirements specification language

(LRSL). The LRSL (see abridged example in Fig. 2) serves

to itemize legal requirements and maintain traceability to

section and paragraph references in the original text. In the

following paragraphs, we will describe the syntax,

semantics, and features of the language, using the text in

Fig. 2. The language grammar is expressed in ISO/IEC

14977-compliant Extended Backus–Naur Form (EBNF)

and appears in ‘‘Appendix’’. For an extended LRSL dis-

cussion with example legal requirements patterns and legal

document styles, see Breaux and Gordon [5].

To begin, each document is assigned a unique index to

the specification, using the DOCUMENT keyword on line 1.

On line 2, the SCHEMA keyword denotes a series of com-

ponents in curly brackets, with each component corre-

sponding to different reference levels in the document

model: In this case, the title level followed by the next

nested level, the chapter level. Two kinds of deeper levels,

sections and nested paragraphs, are expressed in the LRSL

using SECTION and PAR keywords and are followed by a

reference and optional title, as on lines 9 and 10.

Requirements consist of roles, preconditions, and pre-

scriptive clauses, organized into first-order logic expres-

sions using the operator ‘‘|’’ for logical-or and ‘‘&’’ for

logical-and. Each requirement begins with a left justified

stakeholder role (see data collector, line 11), followed by

one or more requirement clauses led by colons (lines 12,

14, 18, 21). Requirements are indicated by the presence of

modal verbs, such as ‘‘must’’ to indicate an obligation or

‘‘may’’ to indicate a right or permission, and requirements

may be categorized or labeled using the ANNOTATE key-

word (line 17).

To preserve part of the requirement’s context, require-

ments are linked to each other by relational keywords as

follows: REFINES indicates that one requirement is a

subprocess or quality attribute of another requirement,

FOLLOWS indicates that one requirement is a post-condi-

tion to another requirement, and EXCEPT indicates that

one requirement has an exception in another requirement

(see line 16, 20, 22, respectively). Additionally, the

REFINES relation allows an analyst to link low-level

system requirements to high-level goals, the latter of which

are written at higher levels of abstraction. The expression

following the keyword indicates the target of the relation;

for example, FOLLOWS 1. #1 on line 16 indicates that the

requirement ‘‘shall disclose the breach…’’ is a post-con-

dition to the first requirement (as indicated by ‘‘#1’’) in

paragraph ‘‘1.’’ These targets can be absolute or relative to

the current requirement’s position and can also target

requirements contained within a paragraph range, such as

‘‘all requirements in (2)(a).’’ Definitions for terms-of-art,

such as ‘‘data collector’’ are preceded by the equals sign

(lines 4 and 5) and are later linked by the parser to uses in

parsed requirements. Definitions apply to the paragraph in

which they occur, but may be used throughout a regulation

using the INCLUDE keyword, not shown here.

The LRSL is complemented by an automated parsing

tool that checks specifications for syntax errors, such as

malformed or unassociated logical expressions, and

semantic errors, such as incorrect references, empty rela-

tions that refer to no rules, unreferenced definitions, and

cycles among relations of the same type, e.g., REFINES,

EXCEPT, FOLLOWS. The parser-constructed model is

1234567891111111111222

12345678910 11 12 13 14 15 16 17 18 19 20 21 22

DOSC

da=

SEPAda

OCUMCHEM

ata gov

ECTAR ata

A

R

MENMA

cover| i//.

ION1. co

: minf: ssysFOLANN: mtimREF: mEXC

NT {c

ollrnmins...

N 6

ollmaiforshasteLLONOTmusme FINmayCEP

NV-chap

ectmentstit

603A

ectnta

rmatall em dOWS TATEst mpos

NES y dePT-T

-60pte

tortaltut

A.2

toraintiodi

dat1.

E bmakssi1.

elaTO

03Aer:

rl atio

210

rns oniscta . #breke ibl. #ay 1.

A603

genn o

//

recof losto 1 achthee a2 the #3

3A}

ncyof

/Se

cora

se th

h-de dand

e r3

.{

yhig

ecur

rds rethe

he r

discdiscd w

requ

sec

ghe

rit

whside bres

clocloith

uir

cti

er

ty

hicdenbresid

osuosuhou

red

on

edu

mea

h cnt oeachdent

ureure ut u

d no

:\d

uca

asu

conof h ot o

inunr

oti

d+}

ati

ure

ntath

of of

n trea

ifi

.{p

on

s

in is thethi

he son

cat

par

peSt

e sis

monab

tio

r:\d

ersotatesecuSta

ost ble

on

d+\

onaeuriate

exde

\.}

al

itye

xpeela

//

y of

dieay.

/..

f t

ent..

.

the

t

Fig. 2 Abridged LRSL Excerpt

from Nevada §603A.210 (1)

150 Requirements Eng (2013) 18:147–173

123

exportable to other formats, such as the HyperText Markup

Language (HTML), the Graph Markup Language (Graph-

ML), and the eXtensible Markup Language (XML). In the

remaining paper, we present post-LRSL-processed

requirements as text statements and graphs automatically

generated by the LRSL parser. The corresponding graph

for Fig. 2 appears in Fig. 3: nodes map to requirements,

and arrows map to relations as follows: REFINES (solid

line), FOLLOWS (finely dotted line), or EXCEPT (dashed

line). Each requirement has a unique identifier: a shared

label, e.g., the two-letter abbreviation NV for Nevada, and

numerical index.

Requirements described in legal texts may contain pre-

conditions embedded in the statement. Step 2 requires the

analyst to separate pre-conditions into implied permissions

when they describe separate actions; a technique that we

call non-modal adaptation. For example, consider the fol-

lowing excerpt from Connecticut §36A-701B(e)(4):

The entity who demonstrates that the affected class of

subject persons to be notified exceeds five hundred

thousand persons may send notice using substitute

notice.

The above excerpt maps to the LRSL in Fig. 4, as fol-

lows: The underlined clause above is separated into a

requirement clause (line 2) with the modal verb ‘‘may’’ and

the annotation ‘‘implied-permission’’ (line 3) to indicate a

non-modal adaptation produced this permission. Next, the

instruction PRECEDES (line 4) indicates the prior

requirement (line 2) is a pre-condition to the second

requirement (line 5). In this work, requirements from

medical record retention laws will be differentiated from

those extracted from data breach notification laws through

the addition of an ‘‘M’’ to the requirements label. For

example, NY-6 refers to the sixth requirement from New

York’s data breach notification law, and NY-M6 refers to

the sixth requirement from New York’s medical retention

law.

3.2 Comparing specifications

After encoding two regulations in the LRSL, the analyst

performs a ‘‘gap analysis’’ using metrics to identify and

rationalize similarities and differences between require-

ments pairs. For comparing two requirements A and B, the

metrics in Table 1 are used; A refers to the first require-

ment, and B refers to the second requirement.

The gap analysis is used to discover salient differences

between two requirements sets. These differences occur

between statements, called relational dissimilarity, and

within statements, called phrase-dissimilarity. Relational

dissimilarity is measured when one requirement set con-

tains a requirement not present in the other set (i.e., a

requirement without an S-NE or S-PE metric) and phrase-

dissimilarity is measured when two near equivalent

requirements (S-NE) are differentiated using the phrase-

level metrics (e.g., P-G1, P-G2). If an organization operates

information services in two jurisdictions governed sepa-

rately by these requirements sets, resolving these differ-

ences is necessary to determine a single standard of care.

For example, consider the following requirements from

regulations CT and WI, respectively. Comparison of these

requirements by the analyst yields the measurements

shown in Fig. 5.

Because some portions of the requirements describe the

same action, they are first asserted as being near equivalent

(S-NE). Phrases in the requirements generalize one

another; ‘‘owns, licenses, or maintains’’ is more general

than ‘‘maintains or licenses,’’ because it includes the extra

action ‘‘owns’’ (P-R1) and ‘‘personal information’’ is a

more general term than ‘‘computerized data containing

personal information,’’ because this data potentially con-

tain other types of information (P-G1). Lastly, the P-R2

metric measures the new constraint ‘‘in this state’’ that does

not appear in the CT-4.

3.3 Generating water marks

In prior work, we hypothesize that the differences made

salient during a gap analysis could be generally resolved

through three water mark techniques, called union, disjoint,

and minimum [15]; in this paper, we implemented and

evaluated this proposal. The union water mark technique

yields a single practice from multiple jurisdictions, whereas

the disjoint water mark technique maintains separate

practices for each jurisdiction. The minimum water mark

Fig. 3 GraphML representation

of Nevada §603A.210 (1)

Requirements Eng (2013) 18:147–173 151

123

describes the lowest standard of care across multiple reg-

ulations. We now describe how to implement the water

marks using the previously obtained measures.

3.3.1 Union reconciliation

The union water mark consists of systematically merging

requirements from multiple jurisdictions while addressing

conflicts. The merger proceeds in two steps: (1) The analyst

reviews the relational dissimilarities to identify require-

ments that are valid in both jurisdictions; and (2) the ana-

lyst merges phrase dissimilarities from two near equivalent

requirements to yield a single, combined requirement.

The analyst identifies relationally dissimilar require-

ments by finding requirements in either requirement set

that are not measured with S-NE or S-PE metrics. These

requirements are reconciled by two techniques: preserva-

tion, which means practicing the requirement in both

jurisdictions, and omission, or choosing to not practice a

requirement in either jurisdiction. Preservation is typically

applied to refinements linked by REFINES that describe

how to implement a practice or to post-conditions linked by

FOLLOWS that describe follow-on permissions, obliga-

tions, or prohibitions. In Fig. 6, we preserve New York’s

(NY) requirement NY-25 to log notices in Connecticut’s

(CT) jurisdiction using a dashed-border node and main-

taining the same refinement relation (a solid arrow).

Omission is typically applied to exceptions linked by

EXCEPT that appear in one jurisdiction and not another. In

Fig. 6, the omission of Mississippi’s (MS) requirement

MS-23 appears as a red cross through a node. The key in

Fig. 6 applies to subsequent figures in this paper.

The intuition for preservations is that relationally dis-

similar requirements linked with REFINES or FOLLOWS

are sub-tasks, quality attributes, or additional tasks an

organization performs to achieve compliance with one

jurisdiction and that compliance with these requirements is

permissible in another jurisdiction where they have no

23

123456

eenttity : su hu AN PR :

yma

ubjundNNORECma

ay jecdreOTACEDay

demt pd t

ATEDES sen

monperthoim#2

nd

nstrsonousmpl2no

ratns andied

tic

te to

d pd-p

ce

thao bepersperm

usi

at e nsonmis

ing

thnotnsssi

g s

he aifi

on

ubs

affied

sti

fecd ex

itu

tedxce

te

d ceed

no

clasds f

otic

ss fiv

ce

ofve

Fig. 4 Example of non-modal

adaptation to map pre-

conditions to implied

permissions

Table 1 Requirements

comparison metricsMetric Metric description

S-NE (Near equivalent): requirements A and B are equivalent, with some portions of the requirements

describing the same or a similar action

S-PE (Pure equivalency): requirements A and B are equivalent and do not need further refinement

through phrase metrics

P-G1 (Generalized concept): the ‘‘phrase in B’’ describes a more general concept than the ‘‘phrase in A’’

P-G2 (Missing constraint): the ‘‘phrase in A’’ is missing from Requirement B

P-R1 (Refined concept): the ‘‘phrase in B’’ describes a more refined concept than the ‘‘phrase in A’’

P-R2 (New constraint): the ‘‘phrase in B’’ is missing from Requirement A

P-M (Modality change): the ‘‘phrase in A’’ has a different modality than the ‘‘phrase in B’’

C

W

S-Nn

P-Rr

P-GgenP-R

CT

WI

MNEneaR1 refiG1 neraR2 new

-4:

-2:

MeE (Car eq

(CTned(Caliz(CT

w co

AcA

easuCT-4quivT-4d coT-4

zed T-4ons

A pcontA pe

ure 4, Wvale4, Wonc4, Wcon

4, Wstra

perstainerso

WI-ent

WI-2ept

WI-2nce

WI-2int

sonns pon m

2)

2)t 2)

ept2)

n opersmai

ownonainta

–

“owor “petha“in

ns, al inains

wnsliceersoat con th

licnfos or

s, lienseonaontis s

ensrma

r lic

icenes”

al intainstate

ses atio

cens

nse

nforns pe” i

oron ses

s, o

rmaersois m

r m

per

or m

atioona

miss

main

rson

main

on” al insing

ntai

nal

Ph

ntai

gennforg fro

ins

inf

hras

ins”

nerarmaom

co

form

se

” ge

alizatio

m CT

omp

mati

ene

zes on”T-4

pute

ion

rali

“co

eriz

n in

izes

omp

zed

thi

s “m

pute

da

s st

mai

eriz

ata

tate

nta

zed

th

e

ains

dat

hat

ta

Fig. 5 Phrase-dissimilar

Requirements from CT §36a-

701b and WI §134.98

152 Requirements Eng (2013) 18:147–173

123

observed conflicts or near equivalent counterparts. This

may incur an additional burden for those transactions

covered by the second jurisdiction, but it may also

streamline an organization’s business practices. Contrarily,

relationally dissimilar requirements linked using EXCEPT

describe alternatives or optional requirements from one

jurisdiction that do not appear in a second jurisdiction.

Thus, practicing such exceptions in the second jurisdiction

may lead to violating a near equivalent obligation in that

jurisdiction.

Finally, the analyst merges near equivalent requirements

by carefully combining dissimilar phrases into a single

statement designed to encompass the details specified by

both requirements. To facilitate this process, we developed

heuristics (see Table 2) based on the phrase metric type

(concept, constraint, modal) as well as the phrase topic in

question: the subject of the requirement, such as ‘‘a person

or business;’’ the action or a quality of the action, such as

‘‘notify the attorney general’’ or ‘‘notify expeditiously;’’ or

the object of the action, including to whom or for whom the

action is performed, such as ‘‘affected residents.’’ The

heuristics in Table 2 are intended to ‘‘take the union’’ of the

meanings of two phrases, effectively yielding a requirement

that covers two previously separated sets of circumstances.

Applying the heuristics yields a single requirement that

maintains the original legal text with changes that can be

traced back to the selected measures and heuristics. Fur-

ther, the analyst must be aware of negations in the text,

which reverses the above heuristics, as discussed in Sect. 9.

3.3.2 Minimum and disjoint reconciliation

For relational dissimilarity, the minimum water mark

technique consists of ‘‘omitting’’ requirements from one

jurisdiction that do not appear in another jurisdiction.

Omissions are excluded from consideration for the affected

system implementation. Alternatively, the disjoint tech-

nique preserves these requirements. For example, the NY

data breach law §899-aa(8)(a) specifies that an organiza-

tion shall notify the state attorney general and other state

entities regarding the ‘‘timing, content, and distribution of

the notices and approximate number of affected persons’’

following notification of the affected individuals (see

Fig. 7); CT’s data breach law §36a-701b has no such

requirement. If an organization chooses the minimum

standard, they will follow CT’s lower standard of care and

not notify the state attorney general in either jurisdiction, as

shown in Fig. 6. Otherwise, the organization may keep

their practices disjoint and only notify the state attorney

general in New York, where it is prescribed.

The analyst applies the minimum technique to phrase-

dissimilar requirements by omitting P measured phrases

that appear in one regulation but not another. In Fig. 8,

both CT and MS specify requirements covering entities

that possess data on their consumers. However, nuances in

the phrases affect the object that each requirement refers to,

as shown in Fig. 8. Adopting the minimum technique

means preferring the more specific phrase from CT in the

P-G1 measure ‘‘computerized data that contains personal

information’’ for both jurisdictions over MS’s more general

phrase ‘‘personal information’’ that covers non-computer-

ized information. The disjoint standard retains these

requirements separately for data covered by each state.

3.3.3 Post-water mark de-confliction

The union water mark process is designed to eliminate

conflicts between requirements by taking the higher stan-

dard of care. In rare cases, the union may introduce new

Fig. 6 Union reconciliation of relational dissimilarity between MS-

HB-583 and CT-36a-701b

Table 2 Heuristics for union reconciliation of phrase-dissimilar requirements

Conceptual measures

(P-G1, P-R1)

Constraint measures

(P-G2, P-R2)

Modal measures

(P-M)

Subject Preserve more general subject phrase Preserve constrained subject Preserve obligations over permissions

(e.g. ‘‘shall’’ or ‘‘must’’ over ‘‘may’’)Action Preserve more specific action Preserve constrained action

Object Preserve more general object phrase Preserve less constrained object

Requirements Eng (2013) 18:147–173 153

123

conflicts. Consider Fig. 9: after a data breach has occurred,

Connecticut obligates the breached organization to conduct

an investigation to determine the scope of the incident, to

identify affected individuals, or to restore the integrity of

the system (CT-6). Because this requirement has no

equivalent in Wisconsin, the requirement is preserved and

consequently practiced in both jurisdictions. Within the

Wisconsin law, another requirement (WI-20) describes the

timing of the notification and WI-20 also becomes part of

the union. Although the two requirements pertain to dif-

ferent actions and with a S-NE metric, a potential conflict

arises: conducting the investigation (CT-6) may introduce a

delay into the notification process that could result in a

violation of the expedited delivery (WI-20).

In general, a preserved relationally dissimilar require-

ment or merged phrase-dissimilar requirement can conflict

with a quality attribute specified in a separate requirement

that was also preserved in the union. Conflicts such as this

are addressed by performing a second pass over the union

water mark, called Quality Attribute Validation (QAV). In

QAV, the analyst compares each newly preserved or

merged requirement with any quality requirements to dis-

cover potential conflicts. For example, one jurisdiction may

have a requirement that notification to affected individuals

be made before the breach becomes public knowledge

through the press, while another jurisdiction requires

notification of the breach be sent to the state attorney

general. Although not in direct conflict with one another,

there is a possibility the state attorney general has a duty to

notify public media of the breach, which would conflict

with the quality requirement that individuals must be

notified first. Quality requirements will not always conflict

with preservations and mergers. However, in the event of a

likely conflict, the analyst may choose to instead keep these

requirements disjoint, or note the case for later review by a

designer who can better evaluate its likelihood based on

design choices provided by the technology.

3.4 Water mark chaining

The water mark method is a binary operation that accepts

two sets of requirements and produces a single, reconciled

requirement set for two jurisdictions. To analyze three or

more jurisdictions, the analyst combines the output from two

jurisdictions with the third requirements set using the same

binary operation. These combinations produce ‘‘chains’’ that

raise the question: Is this process commutative? That is, does it

matter which order we apply the operation over three or more

jurisdictions to compute the outcome?

Consider an organization that has data on residents from

three jurisdictions to which the organization is subject to

their regulations: A, B, and C. Preferring to determine a

single standard of care (if one exists), the organization’s

business analyst applies the water mark method. First, the

analyst compares requirements sets A and B (denoted A/B)

and generates the A–B water mark for the aggregate of two

jurisdictions. The A–B water mark can then be reconciled

with the requirements set C (denoted A–B/C), which

Fig. 7 Minimum and disjoint reconciliation of relational dissimilar-

ity between NY §899-aa and CT-36a-701b

C

M

S-N(neP-G(ge

CT

MS

NE ear eG1 ener

-4:

S-2:

M(CTequ(CTraliz

PpP

MeaT-4uivaT-4,zed

PerspersPers

sur, M

alent, MScon

son sonason

re MS-2

t) S-2ncep

owal inow

2)

) pt)

wnsnfo

wns,

, licorma, lic

–

“pth

cenatiocens

pershat c

nseson ses

sonacont

s or

or

al intain

r m

ma

nforns pe

main

ainta

rmaterso

ntain

ain

tiononal

ns c

s pe

P

n” gl inf

com

erso

Phr

geneform

mpu

ona

rase

eralimati

uter

al in

e

izesion”

rize

nfor

s “co”

ed d

rma

omp

data

atio

pute

a th

on…

eriz

hat c

…

zed d

con

data

ntain

a

ns Fig. 8 Phrase-dissimilar

requirements from CT §36a-

701b and MS-HB-583

Fig. 9 CT/WI investigation/timing quality attribute conflict

154 Requirements Eng (2013) 18:147–173

123

reflects a comparison between the water mark A–B and

regulation C and yields the A–B–C water mark. This left-

associative notation is used throughout our paper to

describe the order of operations. In Sects. 9 and 10, we

discuss interview findings about how legal experts order

jurisdictions in their analysis and our results from evalu-

ating the commutative property, respectively.

4 Case study design

We now discuss our case study design, including research

questions, dataset selection criteria, units of analysis, and

analysis procedure. To guide our research, we established

the following research questions:

R1: What techniques exist to align requirements from

multiple jurisdictions?

R2: How do these techniques scale?

Regarding question R1, we discovered that business and

legal analysts presently lack a systematic method for

comparing requirements across jurisdictions. To discover

such a method, we employed grounded analysis [8], in

which a theory is derived from a dataset, and then we chose

to evaluate the method using additional datasets and sub-

ject matter expert review. We conducted two case studies

using this design: the dataset for case study #1 consists of

U.S. data breach notification laws that have been enacted

across 46 U.S. states and territories from 2002 to 2011,

each governing personal information about state residents;

the dataset for case study #2 consists of U.S. medical

record retention laws that have been enacted across all 50

U.S. states and two territories (D.C. and Puerto Rico)

starting as early as the 1970s. These sections were made to

observe variation within these two themes and to test

whether we could generalize our observations from one

domain to a second domain. For case study #1, we down-

selected to eight data breach notification laws based on

guidance that we received from a legal expert with 7 years

of privacy and security law expertise to highlight regula-

tions that have been a priority for U.S. companies:

AR: Personal Information Protection Act, Arkansas

Chapter 4.110. Enacted 2005

CT: Breach of Security Regarding Computerized Data

Containing Personal Information: Connecticut

Chapter 669, §36a-701b. Enacted 2006

MA: Security Breaches, Massachusetts Chapter 93H.

Enacted 2007

MD: Personal Information Protection Act, Maryland

Subtitle 14–35. Enacted 2008

MS: (No title given) Mississippi House Bill 583.

Enacted 2011

NV: Security of Personal Information, Nevada Chapter

§603A. Enacted 2006

NY: Notification of Unauthorized Acquisition of

Personal Information, New York General

Business Law §899-aa. Enacted 2005

WI: Notice of Unauthorized Access to Personal

Information, Wisconsin §134.98. Enacted 2006

For case study #2, we down-selected to four medical

record retention laws by identifying the highest estimated

personal health care expenditures for all payers by state of

provider in 2009, the latest year for which this data was

available [30]. This yielded the states: California, New

York, Texas, and Florida. Within the U.S., state health care

law is distributed through different chapters of each state’s

law. Thus, to standardize our search and collection process,

we obtained the relevant medical record retention laws by

consulting a brief issued by the American Health Infor-

mation Management Association (AHIMA) that identifies

the relevant sections of each state’s body of law [1]. This

yielded the following laws:

CA: California Code Regs. tit. 22 §70751—Medical

Record Availability

FL: Florida Admin. Code 59A-3.270 Health Information

Management

NY: N.Y. Comp. Codes R. & Regs. tit. 10, §405.10—

Medical Records Requirements in Hospitals

TX: Texas Health and Safety Code §241.103—

Preservation of Records

In case study #1, the selected laws all follow a similar

structure: They outline who is covered, what constitutes

personal information, who must be notified and under what

conditions. Because medical record retention law has

evolved at different times for each state and over a longer

period of time (over 40 years, as opposed to 9 years), we

had to supplement the sections that we identified in the

AHIMA brief with additional sections. These supplements

were identified using explicit references to definitions and

other paragraphs that were frequently adjacent to those

cited in the AHIMA brief. This approach allowed us to

provide important context that was needed for the

requirements to be understood.

In practice, an analyst may select relevant regulations

with the aid of legal consultation; however, this option may

not be available to small firms. To guide their selection, we

recommend that the analyst begins by searching for orga-

nizations—such as non-profits or governmental agencies—

that prepare and maintain aggregate lists of such regula-

tions. For example, in our case study #1, we consulted with

a list of state security breach notification regulations

maintained by the National Conference of State

Requirements Eng (2013) 18:147–173 155

123

Legislatures [21], and in our case study #2, we relied on the

aforementioned AHIMA brief of medical record retention

laws. Regulatory bodies may also provide keyword search

tools for inspecting their legal codes in electronic formats.

Because the cost of overlooking a relevant regulation and

missing potentially significant requirements is greater than

the cost of coding an additional regulation, we recommend

that the analyst err on the side of caution and include more

regulations in their analysis to ensure greater coverage.

After the selection process was completed, all legal

documents were mapped into the LRSL by the investiga-

tors (the authors), separately, and co-reviewed. The first

author designed the reconciliation process with feedback

from the second author to identify and address errors or

concerns that arose throughout the process. The investi-

gators kept a research notebook to record comments about

unusual or notable artifacts in the translation; during

comparison and reconciliation, a list of strategies was

recorded to reflect how the investigator handled unusual

cases, and upon acceptance of a new strategy, all previous

resolutions were reviewed to ensure consistency across the

dataset. A law expert was consulted on legal questions that

arose during the process.

The units of analysis consist of the translated require-

ments and their relations as expressed in the LRSL and the

measures of relational- and phrase-dissimilarity produced

by the gap analysis. In the analysis procedure, we first

compared definitions and then requirements between the

regulations, applying the metrics outlined in Sect. 3. After

near and pure equivalencies were determined, we applied

phrase-level metrics to further differentiate constraints

between the requirements. After determining the differ-

ences, we constructed the union and disjoint water marks

by applying the water mark generation techniques to the

measures to identify trade-offs. Next, we applied the post-

water marking de-confliction process to identify potential

conflicts that can be introduced by the union water mark.

Finally, we invited three legal experts (two law scholars

and one attorney) to review the final process and a subset of

the generated water marks.

5 Legal expert survey design

Based on the reviews with our legal experts, we designed

an online survey to compare our findings from case study

#1 to the current state of legal practice. The results of our

survey aimed to supplement the remarks and assertions put

forth during the interviews with opinions from a larger

sample. Participants were solicited using fliers distributed

at two conferences, the 2012 Global Privacy Summit held

by the International Association of Privacy Professionals

(IAPP) and the 2012 Privacy Law Scholars Conference

(PLSC), both leading conferences for privacy professionals

from all sectors. Participants were offered a $20 Amazon

gift card upon completion of the survey. Before taking the

survey, participants were screened using preliminary

questions regarding legal background and experience,

including their current position or job title, a description of

their typical job duties, an overview of their educational

background (including degrees obtained and areas of con-

centration, if applicable), and their legal background,

including types of law practiced and years of experience.

Participants were required to have a law degree and

experience in corporate law (minimum of 5 years), or

significant exposure to IT regulatory compliance based on

written descriptions of job duties and legal backgrounds.

We conducted our survey as follows: 1) We presented

brief descriptions of the methods of union, disjoint, and

minimum; and 2) we presented six scenarios that consist of

a pair of requirements from two separate jurisdictions and

with all legal cross-references removed, as well as the

reconciled third requirement produced by our union

method. The following scenarios were selected by both

analysts after reviewing comparisons collected during case

study #1. These scenarios represent a stratified sample of

the kinds of reconciliations that an analyst might encoun-

ter. We sought examples based on the following dimen-

sions: (1) regulation phenomena, actions, and attributes,

such as data breach notice thresholds, acceptable notice

media, and so on; (2) means of reconciliation, such as

reduction in options, merging of nuanced phrases; and (3)

relative isolation from outside context (e.g., few linkages to

other definitions and requirements). While the first two

criteria establish breadth of types, the third criteria are

necessary to conduct the survey in a timely fashion and

have the drawback of limiting our sample to potentially

less complex examples. The number of scenarios was kept

to a minimum, given that each scenario requires three

open-ended responses as explanations, as discussed in the

next paragraph. The six scenarios are as follows:

Scenario 1: Reduction in options—one requirement

disallows an option present in another

requirement

Scenario 2: Preservation of relationally dissimilar

requirement—one requirement addresses

an issue the other requirement is silent on

Scenario 3: Merging of multiple phrases—each

requirement contains constraints not

present in the other requirement

Scenario 4: Merging of nuanced phrases—

requirements have subtly different con-

straints that address the same issue

Scenario 5: Handling negatively framed pre-

conditions—pre-conditions are given in the

156 Requirements Eng (2013) 18:147–173

123

negative, meaning they cover an organization

that does not meet a certain criteria

Scenario 6: Handling non-modal requirements—

requirements are implied permissions that

function as pre-conditions to other

requirements

The requirements pairs themselves cover many different

areas found in data breach notification laws, including the

conditions under which notice must be sent, whether or not

an organization is obligated to provide individuals with the

information that was affected by the breach, the criteria

under which certain types of notice (e.g. as electronic

notice) may be used, the priority with which the notifica-

tion must be delivered, and the different mechanisms under

which an organization may be covered by the laws. For

each case, the participants were asked to answer, and

elaborate upon their answers to, the following questions:

(a) Do you believe the given ‘‘union’’ option may put an

organization in violation of the law? (Yes/No/

Uncertain)

(b) If you were hired as a consultant for a business or

organization facing this decision, what would you

recommend? (Union/Disjoint/Minimum)

(c) Which option do you think most businesses or

organizations are choosing, based on your experi-

ence? (Union/Disjoint/Minimum)

The results of the two case studies and the survey are

discussed in the following sections.

6 Summary findings

Applying the method to the eight data breach regulations

produced a total of 338 requirements with Maryland

yielding the most (60 requirements) and Arkansas and

Wisconsin the fewest (36, each) for an average 42

requirements per regulatory document. Requirements

extraction from the eight regulations required approxi-

mately 2.2 h per regulation. Additional time was expended

to develop and refine the extraction method. Applying the

method to the four medical record retention laws produced

a total of 168 requirements, with Florida and Texas pro-

ducing the most and least (103 vs. 3), respectively, aver-

aging again at 42 requirements per document. Extraction

took 1.9 h per regulation.

The gap analysis to produce the measures required a

total of 30.8 h for the eight data breach notification regu-

lations and 8.5 h for the four medical record retention laws.

This effort required pairwise comparisons between the

union of previously measured regulations and the entire

next regulation (shown for data breach notification in

Fig. 10: The size of the union grows slower, as a function

of the total number of requirements covered). Figure 11

summarizes the number of requirements contained in the

union water mark (a single standard) and the disjoint water

mark (separate standards). Above each water mark, we

display the average time in minutes required to analyze

each requirement in the union water mark. Although this

number rises moderately as each new jurisdiction is added,

this increase suggests the process is linear. Note that our

process employed no additional efficiencies over succes-

sive jurisdictions.

In MRR case study #2, we found the percent reduction

achieved by the union method was far less that the

reduction observed in the original DBNL case study #1.

The final MRR union water mark (TX-NY-CA-FL) con-

tains only 15 % fewer requirements than if the require-

ments were kept entirely disjoint. There are a number of

potential explanations for this statistic. First, the DBNL

dataset contains laws that were passed by state legislatures

within a short timeframe of 6 years and it is very likely that

these laws represent a shared legislative focus across the

states to address an emerging issue. In several laws, it

seemed evident that states were in fact borrowing

Fig. 10 Breakdown of

comparison metrics by

specification pair

Requirements Eng (2013) 18:147–173 157

123

legislative text from other states that passed similar laws in

years prior. The MRR dataset, however, contains laws that

were passed over a larger 16-year period. This larger period

means that states may be more likely to vary their focus as

issues in medical record retention evolve with new tech-

nology and industrial practice. Independent of the influence

of time, it was clearly observed in the dataset that the MRR

case study #2 exhibited far more domain variation than the

DBNL case study #1. Second, as a water mark that covers a

related domain adds a new regulation, the size of the

increase (or delta) in number of requirements per regula-

tion will decrease for each new regulation added, as it

becomes increasingly unlikely that a new regulation will

address a domain-related topic that has not been previously

encountered. Where we to reconcile additional medical

record retention laws, we believe that the percentage

reduction would thus increase in this manner, although,

perhaps not to the same extent as a series of laws passed

within a short time frame or covering a narrower topic.

Additionally, we found a few cases where achieving the

union proved to be impossible due to the presence of

irreconcilable differences between states. We discuss this

phenomenon in detail in Sect. 7.

Figure 10 shows the relative breakdown of the com-

parison metrics (S-*, P-*) for each new jurisdiction when

creating the union water mark for our set of data breach

notification laws. The first column, CT/MS, denotes the

comparison between Connecticut (CT) and Mississippi

(MS); the next column, CT-MS/NY, reflects a comparison

between the generated water mark CT-MS and New York

(NY), and so on. We now discuss interesting patterns

observed during reconciliation.

As we performed additional comparisons, we found an

increasing dominance of phrase metrics (P-*) over state-

ment metrics (S-*). Initially, statement measures (S-*)

contributed to over 70 % of the total measures (see CT/

MS); however, as additional regulations were added to the

water mark, phrase measures began to dominate (P-*). As

we seek to reduce comparison times between specifications

for future work, we will begin with techniques that show

promise in reducing phrase-level comparisons.

When adding a new regulation to the existing water

mark, we discovered fewer pure equivalencies (S-PE) rel-

ative to near equivalencies (S-NE). As the union water

mark grew in size (see Fig. 10), we saw fewer pure

equivalencies coupled with a rise in near equivalencies.

The heuristics for reconciling phrase dissimilarities may

produce this effect in the union water mark. The repeated

merging of phrases produced requirements of increasing

scope (e.g. ‘‘owns or licenses’’ changing to ‘‘owns, licen-

ses, maintains, or uses’’) and thus reduced the likelihood of

encountering two purely equivalent requirements. As all

our comparisons occurred between generated specifications

and a single jurisdiction (e.g. CT-MS-NY/NV), this may

reflect the decreasing similarity of single jurisdictions with

the water mark.

Lastly, we found that constraint metrics (P-G2, P-R2)

took increasing prevalence over concept metrics (P-G1,

P-R1) as we added specifications to the union water mark.

This may indicate opportunities for future automation, as

identifying conceptual generalizations is more difficult

than identifying new or missing constraints.

Although these patterns were less observable for our

medical records retention laws (owing to the smaller

number analyzed, as well as the large discrepancy between

number of requirements contained in each document),

we found that two of three still held: near equivalencies

(S-NE) increased relative to pure equivalencies (S-PE), and

constraint metrics (P-G2, P-R2) increased relative to con-

ceptual metrics (P-G1, P-R1).

7 Patterns of dissimilarity

During the water marking process, we observed multiple

cross-regulatory conflicts that affect system design or

organizational processes depending on the reconciliation

technique, union, or disjoint, employed by the analyst.

These conflicts were due to varying legal definitions,

varying outcomes that result from attempting the union,

and varying practices described in regulations.

7.1 Variations among legal definitions

Regulatory definitions affect how an analyst decides cov-

erage, because the definitions are often used in pre-condi-

tions to requirements. In the regulations that we studied,

the definitions for ‘‘personal information’’ produced several

coverage conflicts (see Fig. 12). These definitions have

several overlaps, e.g., all include a first name, or first ini-

tial, and last name in combination with at least one ‘‘data

element’’ as noted. However, individual states also identify

Fig. 11 Requirement counts for union and disjoint high water marks

158 Requirements Eng (2013) 18:147–173

123

special inclusions and exclusions. Some laws cover medi-

cal information, while others more broadly cover any

identifiable information. Furthermore, certain states dif-

ferentiate who is or is not covered by making allowances

for organizations subject to other laws, such as the

Gramm–Leach–Bliley Act (GLBA). These allowances

often enable the covered organization to use compliance

with another law as a proxy for compliance with the state’s

law. Explicit information exclusions appear when a state

identifies a class of information that is not covered by the

law. For example, Maryland excludes information listed

under the HIPAA, which covers medical information. We

omitted this exclusion from our union water mark, because

medical information subsumes information that is included

in the definitions of personal information in other regula-

tions, like Wisconsin who explicitly includes biometric

data. Thus, we interpret certain exclusions as being dis-

cretionary and not mandatory. If the regulator’s intent is

clear to the analyst, such as using exclusions to reduce

regulatory burden, then we believe this choice is reason-

able. However, the intent may be unclear for some exclu-

sions, in which case the analyst may have encountered a

strong conflict, or when two requirements addressing the

same issue have no overlap and cannot be reconciled using

the union technique. We discuss strong conflicts in more

detail in Sect. 7.4.3.

7.2 Variations in reconciliatory outcomes

When faced with different standards of care between two

jurisdictions, the analyst may choose to keep the standards

separate (practicing them only where prescribed) or take

the higher standard for both jurisdictions. Taking the union

can have a number of effects for the jurisdictions in

question, including the preservation of constraints from one

jurisdiction that affect quality attributes in another,

increasing or decreasing the frequency of a performed

action between two jurisdictions, supplementing an

existing action from one jurisdiction with additional steps

required by another, or even the creation of new, higher

standard that exceeds the standards set by both jurisdic-

tions. We discuss these variations in outcomes in this

section.

When one jurisdiction provides additional requirements

that another jurisdiction lacks, taking the union of two

jurisdictions results in preserving these requirements across

both jurisdictions. The CT-MS water mark was reconciled

with New York’s §899-aa, which contains requirements

NY-2 through NY-4 that prescribe the criteria to use to

determine when a data breach have occurred (see Fig. 13).

These requirements refine otherwise ambiguous require-

ments at the cost of flexibility within the organization.

Because the relationally dissimilar requirements are linked

with the REFINES relation, the union water mark requires

retaining and practicing these refinements in both juris-

dictions. If kept disjoint, however, the two jurisdictions

could implement different breach determination criteria:

New York data would be subject to the criteria set forth in

the NY §899-aa, and Connecticut and Mississippi data

would be subject to any practices deemed appropriate by

the covered organization. Such differences could yield

different operational outcomes given the same circum-

stances; for example, complying with New York’s law

could treat a lost laptop as a breach (NY-3), whereas

complying with Connecticut and Mississippi might not

assume this same treatment under a disjoint water mark.

In some cases, taking the union water mark results in the

preservation of a constraint that can affect the quality with

which a certain action is performed. In Fig. 14, for

example, both CT-15 and MS-14 require notice to the data

owner or licensee; however, Mississippi includes an addi-

tional constraint (indicated in italics and measured by the

P-R2 metric): provide notice ‘‘as soon as practicable fol-

lowing [the breach’s] discovery.’’ Using the union heuris-

tics, the additional constraint was preserved in the

reconciled requirement (CT-MS-15), placing a degree of

Fig. 12 Inter-jurisdictional

conflicts in personal information

definitions

Requirements Eng (2013) 18:147–173 159

123

urgency on the process. Taking the disjoint water mark

could yield different priorities for notifying the data owner.

In other cases, taking the union water mark can result in

increased or reduced frequency of an action that organi-

zations already perform. In Fig. 15, for example, CT-MS

contains a requirement (CT-MS-11) specifying that indi-

viduals do not need to be notified of a breach if there is not

a risk of harm. Because this requirement serves as an

exception to the standard notification procedure shared

across jurisdictions (CT-MS-7, NY-10), it cannot be pre-

served and applied to NY-10. Thus, CT and MS residents

will be notified regardless of whether or not harm is likely

as a result of a breach. Our legal experts comment on this

exception in Sect. 9, noting that while sending notification

for all breaches may not be in violation of the law, it

creates a risk of overloading residents with notices.

Within our medical records retention dataset, New York

and Florida specify data elements that must be included in

medical records and do so with different specificity. In

Fig. 16, for example, New York uses broad information

categories, such as ‘‘patient care services’’ (TX-NY-CA-

M20), and Florida uses specific categories, such as ‘‘indi-

vidualized treatment plan’’ (FL-M43) or ‘‘medication and

dosage administered’’ (FL-M35). Variations in the level of

abstraction to which information is described lead to more

than one equivalency between requirements from Texas,

New York, and California and requirements in Florida. As

shown in Fig. 16, the refinement TX-NY-CA-M23 has no

equivalent requirement in Florida: A requirement that all

data elements in a medical record must include additional

meta-data, such as time, practitioner category, etc. This

added meta-data have significant implications for system

design, given that it requires all access points to medical

records to have the capability to log this additional data.

When reconciled, this refinement is preserved and prac-

ticed in both jurisdictions. As a consequence, the analyst

must propagate this requirement as a refinement to all

requirements in the new jurisdiction that the parent

requirement, TX-CA-NY-M20, is linked to as an

equivalent.

Requirements in some jurisdictions are highly coupled

with the practices in that jurisdiction and thus propagating

these requirements to other jurisdictions would not make

sense. In Fig. 17, Florida describes multiple requirements

that specify the content that must be included in medical

records. In particular, requirement FL-M36 requires

including a Florida-specific Emergency Medical Service

Report (HRS Form 1894). While the union prescribes this

requirement be practiced in all jurisdictions, doing so is

irrational and could result in confusion on the part of

hospital personnel or patients operating in a Texas or

California, for example. In this case, the analyst should

choose the disjoint option and not practice this requirement

where it is not explicitly required.

Finally, the union can produce a higher standard of care

than both requirements it reconciles, creating a new

standard entirely. In Fig. 18, the requirements NV-47 and

CT-MS-NY-48 require notifying consumer-reporting

agencies about a data breach. Using the P-G2 and P-R2

metrics, we measured unique constraints in each require-

ment, shown in boldface and italics, respectively. These

constraints affect the action (how to notify), object (notice

content) and the target (notice recipient). The requirement

CT-MS-NY-NV-47 produced by the union operation

Fig. 13 Preservation of

refinement series between

CT-MS and NY §899-aa

(GraphML)

Fig. 14 Phrase-dissimilarity between CT-36a-701b and MS-HB-593

160 Requirements Eng (2013) 18:147–173

123

preserves the highlighted phrases from each input

requirement. Keeping such requirements disjoint may

result in unnecessary duplication of effort in the determi-

nation of a consumer-reporting agency or confusion about

which notification content should be sent to whom. How-

ever, the combination of these constraints yields a higher

standard of care that is not present in either jurisdiction,

alone.

7.3 Variations in practice

During our reconciliation process, we discovered unusual

cases that merited additional care from the analyst. These

cases include uncommon coverage mechanisms that pre-

clude using a reconciliation technique, the use of goal-

based requirements that necessitate simultaneous recon-

ciliation with multiple requirements, and the potential for

reconciled definitions to have unintended implications as

they are propagated throughout a requirements specifica-

tion. These occurrences signify areas of potential future

expansion for both the LRSL itself and the water marking

method.

Fig. 15 Removal of relational

dissimilarity between union

CT-MS and NY §899-aa

(GraphML)

Fig. 16 Preservation of logging requirement from TX-NY-CA-M water mark resulting in multiple refinements

Fig. 17 Omission of Florida-specific requirement in TX-NY-CA

water mark

Requirements Eng (2013) 18:147–173 161

123

7.3.1 Variation in coverage mechanisms

The water mark generation process is used to reconcile

requirements from different jurisdictions. Most of our

regulations studied were limited to residents of the gov-

erned jurisdiction; however, Wisconsin §134.98 requires

organizations that ‘‘have their principal place of business

located in [Wisconsin]’’ (WI-1) to send notices to affected

subjects, regardless of the subject’s state of residence [7].

In this case, individuals are covered by both the law in their

state of residence and Wisconsin’s law. If the individual is

not a Wisconsin state resident, then the organization must

at least meet the Wisconsin’s legal requirements for data

breach notification, which may be a higher standard of

care. In the event that Wisconsin’s requirements are a

lower standard, then the organization may find themselves

in a conflicted situation. In such situations, legal guidance

can be used to assess the risk of complying with one

standard over another. After discovering this finding, we

examined other data breach notification laws in the United

States and found Wisconsin to be the only state to use such

a mechanism.

7.3.2 Goal-based requirements

Goal-based requirements broadly describe what an orga-

nization must do, whereas means-based requirements

describe how to achieve the goal [35]. In law, this dis-

tinction corresponds to legal standards, which are high-

level goals that lack detailed specification, and legal rules,

which are detailed steps that an organization must take to

comply with the law [23]. Reconciling similar goals and

means yields numerous phrase-dissimilar measures when a

single goal can be deemed equivalent to multiple means.

For example, in Fig. 19, Wisconsin allows notification

through a ‘‘reasonable method’’ (WI-27), which permits the

covered organization to determine what method is rea-

sonable. Alternatively, Connecticut and Mississippi define

explicit criteria for the means of notification, including

written (CT-22, MS-21), telephonic (CT-23, MS-22), and

electronic notice (CT-24, MS-25). When performing the

phrase-level comparisons, a separate measure is produced

to link the high-level goal-oriented phrase ‘‘reasonable

method’’ to each of these means-oriented phrases. An

alternative approach is to use the S–R and S–G statement-

level metrics that were introduced in a prior case study [4].

7.3.3 Deference to standard

Regulations may defer to other regulations, such as the

GLBA, as an alternative compliance standard. These

external cross-references are problematic for requirements

engineers because they can yield errors and conflicts [20].

External cross-references can be inconsistently defined, as

shown in Fig. 20. The water marking process can be used

to determine a high standard of care for a set of regulations

that an organization is covered by or anticipates being

covered by. Thus, a company operating in New York and

Massachusetts would code, compare, and reconcile laws

from only these two jurisdictions, even though other

jurisdictions may have higher standards of care. An orga-

nization that is covered by data breach notification laws

and external standards, such as GLBA, would include the

external standards in their analysis along with the data

breach notification laws.

7.4 Variation between domains

Case study #1 focused on the domain of data breach

notification law in the United States, and thus all findings

were limited to this domain. In applying this method in

medical record retention laws in case study #2, we found a

number of pronounced differences that affected the appli-

cation of our method. These differences include a lack of

common document structure, topical similarity and pre-

scriptions between laws of the same domain, and an

increased need for the analyst to have domain knowledge

during the comparison and reconciliation steps.

Fig. 18 Example unification of

phrase-dissimilar requirements

162 Requirements Eng (2013) 18:147–173

123

7.4.1 Document structure and unique practices

As we coded data breach notification laws in the LRSL, we

observed that these laws bear a strong resemblance to one

another with regards to document organization, prescribed

practices and topics covered, and a similar use of rela-

tionships between requirements, such as exceptions and

refinements to permissions or obligations for similar pur-

poses. The eight data breach notification laws that we

encoded all exhibited the following elements:

• Decomposition of personal information into data sub-

categories

• Criteria under which data breach notice must be

delivered

• Acceptable notification means (written notice, tele-

phonic notice, etc.)

• Timing constraints on the notice delivery

• Additional notifications to other legal entities, such as

the state attorney general

These elements were often presented in the same order

across the different laws and in many cases could be

detected upon visual inspection of the LRSL-generated

graphs due to similar relational structures or sub-graphs

(e.g., similar modalities linked together using similar

relational types). Once detected, we learned to compare the

regulations further by examining the number of relationally

dissimilar requirements between the regulations. For

example, generating the union water mark for the data

breach notification laws began with the comparison and

reconciliation of Connecticut and Mississippi, at 37 and 40

requirements, respectively. Between these two regulations,

only four of the total 77 requirements (one from Con-

necticut and three from Mississippi) lacked a near

equivalent requirement in the other jurisdiction. Although

the 73 equivalent requirements require further comparison

using the phrase metrics, the commonality between the two

regulations allowed us to develop a mental model of a

prototypical data breach notification law. In doing so, we

were able to more easily detect the introduction of new

requirements across additional regulations that we studied,

e.g., Maryland requirements describing notice content that

were not previously observed, or requirements that we

observed in previous regulations that were missing from

the new regulation.

With regards to medical record retention, however, the

lack of similarity between different laws was immediately

apparent. Although each law did contain retention periods

for records, the additional areas that were addressed by

each varied considerably, which was confirmed by the lack

of statement-level equivalencies between regulations.

Compare the earlier data breach notification result for

Connecticut and Mississippi (4/77 relationally dissimilar

requirements) with the medical record retention result for

Texas, New York and California, which has 42/62 rela-

tionally dissimilar requirements. In this latter case, each

jurisdiction has its own unique requirements sets: Califor-

nia requires closing existing records and initiating new

records when a patient is transferred within a hospital (CA-

M18 through CA-M20); New York restricts the accept-

ability of telephone and facsimile patient treatment and

care orders from a health care provider (NY-M39 and NY-

M40); and Florida creates an organ and tissue transfer

system (FL-M8—FL-M19), among others. While the water

mark accounts and makes salient these differences, the

accounting requires significant cognitive resources from

the analyst to establish the context in which each require-

ments set is prescribed and to search for potential points of

Fig. 19 Multi-statement

equivalency between WI

§134.98, CT §36a-701b, and

MS-HB-583 (GraphML)

Fig. 20 Inconsistent usage of

external cross-references in

MS-HB-583 and NV §603A

Requirements Eng (2013) 18:147–173 163

123

similarity. We plan to investigate methods for mental

model building going forward that may be able to reduce

this cognitive burden when comparing new and strikingly

different documents.

7.4.2 Conceptual similarity and domain knowledge

In the data breach notification law analysis, we found that

the document text lacked technical jargon and frequently

reused terms across jurisdictions. The lack of jargon

allowed us to identify equivalencies between requirements

with less impediment, as there was little to no uncertainty

over the meaning or usage of a particular word or phrase.

The use of common phrases among different regulations

also helped in the identification of conceptual similarity

and new and missing constraints using the phrase metrics.

However, in our sample of medical record retention

laws, we found that the requirements contained several

uncommon, domain-specific medical terms, which hin-

dered the identification of equivalent statements and

phrase-level similarities. To identify similarities, we relied

on referential domain knowledge (including Taber’s

Medical Dictionary [29]) to make these assertions. For

example, New York and Florida have several requirements

that specify the content of a medical record or the infor-

mation that must be recorded upon discharge of a patient.

Phrases recorded in the P-R1 and P-G1 measures are shown

in Table 3. For diagnostic information, we identified

equivalent statements by looking for requirements that

classified information as being diagnostic in nature (NY-

M23, FL-M31) or for explicit references to a type of

diagnosis (NY-M19, NY-M25, FL-M29, FL-M39). For

information related to discharge summaries, we reviewed

our requirements sets, as well as a medical dictionary, and

constructed a glossary of terms and phrases describing

patient care lifecycle phases (e.g., discharge, release, check

out, and post treatment) and then we searched for these

terms within our dataset, which yielded additional com-

parisons to NY-M24, and FL-M94—FL-M97.

Although we captured these conceptual similarities

using the phrase metrics, our lack of domain knowledge

and reliance on surrogate knowledge sources (e.g., medical

dictionaries) illustrates the ambiguous and uncertain nature

of these measurements among domain outsiders. Incorrect

measures can lead to inaccuracies in the union water mark

especially when one term is replaced by another term. To

reduce this risk, we recommend that for each potential

conceptual similarity the requirements analyst: (1) consult

a relevant standard dictionary and domain-specific dictio-

nary for the term, and (2) annotate the measure with an

uncertainty score, with those above a certain threshold

requiring external validation. After all measures have been

made, the analyst should then (3) consult a domain expert

to review the most uncertain terms and record the expert’s

observations in a final glossary. This procedure reduces the

time commitment from domain experts, prioritizes risk

reduction for the most uncertain terms, and preserves

uncertainty as additional information recorded by the

method. Because analysts eventually acquire significant

domain knowledge, including knowledge of domain-spe-

cific terms-of-art, this procedure will be most important for

analysts approaching a new domain.

Because the consequences of preservation are less than

the consequences of omission, this procedure is designed to

reduce the risk of eliminating a valuable requirement or

requirement phrase during the subsequent reconciliation

phase. If an analyst chooses to preserve a requirement or

phrase that the heuristics in Table 2 indicate should be

omitted, then the analyst only risks including an unneces-

sary obligation or other redundancy in the final water mark.

However, if the analyst omits a requirement or phrase that

the heuristics indicate should be preserved, the risks

include missing an obligation, or losing important details

that are relevant to the satisfaction of the requirement—

both of which could put an organization in violation of the

law.

As a best practice, we recommend that any uncertain

cases be reviewed by a domain expert before proceeding to

the reconciliation phase. Within the MRR dataset, these

uncertain cases of conceptual similarity were infrequent: of

all 146 measures taken using the phrase metrics (P-G1,

P-G2, P-R1, and P-R2), only 12 (8 %) were marked as

being uncertain and would be recommended for review by

a domain expert. While the water mark process cannot

completely remove the need for a domain expert, the

design of the process—as well as the infrequent nature of

these cases—allows the bulk of the work to be handled by

the requirements analyst. If the number of uncertain cases

where frequent, we believe the analyst would need to

quickly acquire relevant domain knowledge before con-

structing the water marks.

7.4.3 Reaching a complete union water mark

For the eight data breach notification laws studied, we were

able to generate a complete union water mark, which is the

repeated use of the union technique for all non-equivalent

and partially equivalent requirements pairs. The complete

union is a single set of requirements that covers all juris-

dictions in the input dataset. For this dataset, the complete

union was possible because all of the conflicts between

requirements were restricted to similar practices that were

different standards of care, and not between practices that

were in true opposition to one another (e.g., an organiza-

tion must use 128-bit encryption or greater vs. an organi-

zation must use encryption of at most 64 bits). These

164 Requirements Eng (2013) 18:147–173

123

conflicts are called weak conflicts, and in general, we

observed that choosing the higher standard of care yields

little to no possibility of causing a legal violation for either

jurisdiction. All examples shown to this point have featured

weak conflicts.

However, as we reconciled the four medical record

retention laws, we observed requirements that prevented us

from generating the complete union. This occurs because

the obligations or permissions imposed by the requirements

have no overlap with one another, and the union method

cannot produce a single practice or single standard of care

that will satisfy both conflicting requirements. These con-

flicts are called strong conflicts, which appear when two

reconciled requirements generated by the union method

yield a high possibility of violating the law in one or both

jurisdictions. In this case, the analyst is forced to use the

disjoint method to keep the requirements separate, which in

turn has consequences for designing separate procedures in

the resulting information system. In extreme cases, these

conflicts can yield entirely separate information systems

when no reuse is possible. We encountered strong conflicts

among 7 requirements, or 5 % of the final 142 require-

ments. We provide one example of a strong conflict below

and offer an explanation for how the analyst can identify

these conflicts.

As medical records may contain contested information,

New York included a permission (NY-M15) for patients

and qualified persons to add statements challenging the

accuracy of these documents, citing another body of law

(New York Public Health) for justification. Within the

scope of our dataset, we found this requirement to be

relationally dissimilar—that is, it had no near- or pure-

equivalent requirement among those from other jurisdic-

tions. In this case, the union method would prescribe pre-

serving this requirement and practicing it in all

jurisdictions, such as in California. Doing so grants a right

to patients in California that California Title 22 §70751

does not provide, and thus patients could not expect that

right to be defended by the California legal system.

Similarly, omission of NY-M15 in an attempt to maintain a

single standard of care infringes on this right as given to

patients in New York. Because both removal and preser-

vation fail in this case, the only outcome is to keep the

requirements disjoint.

8 Threats to validity

We now discuss threats to validity and our mitigations.

8.1 Construct validity

Construct validity reflects whether the construct we pro-

pose to measure is indeed what we measured. In this paper,

we rely on previously validated methods to acquire our

data, including the frame-based method for extracting

regulatory requirements from laws [6], and the nominal

metrics for performing a gap analysis [4]. In this study,

both authors reviewed the extracted requirements for con-

sistency and both authors measured a stratified sample of

requirements and found a 100 % overlap for statement-

level equivalences. Because of the importance of the

comparison phase of the process, we also conducted a

small-scale study in order to determine the consistency

with which raters identify relationships between pairs of

requirements. The study itself was conducted in two parts,

the first focusing on the presence of pure and near equiv-

alencies, and the second on conceptual or constraint rela-

tionships. For each part, the participants were given a short

lesson on how to identify the relationship being tested

(equivalency or concept/constraint) and then presented

with a set of randomly generated requirements pairs, along

with instructions to identify the type of relationship

between pairs.

Responses between participants were tested for inter-

rater reliability using Fleiss’ free-marginal kappa [32]. The

free-marginal Fleiss’ kappa was chosen over the traditional

fixed-marginal kappa proposed by Siegel and Castellan

Table 3 Variations in terms-of-art measured using P-G1 and P-R1 metrics

New York Florida

Diagnostic

information

NY-M19: admitting diagnosis FL-M29: provisional and pre-operative diagnosis

NY-M23: diagnostic orders FL-M31: diagnostic imaging

NY-M25: final diagnosis FL-M39: principal and secondary diagnoses

Discharge

information

NY-M24: outcome of hospitalization, disposition of

case and provisions for follow-up care

FL-M94: recapitulation of patients hospitalization

FL-M95: statement of patients progress and condition upon discharge

FL-M96: facility or person… assuming responsibility for the patient

after discharge

FL-M97: recommendations… for after care, follow-up, referral or other

action necessary to help patient deal with problems

Requirements Eng (2013) 18:147–173 165

123

[27] due to the latter’s susceptibility to prevalence and bias,

leading to a high agreement rating but low kappa score

[24]. In addition, the free-marginal kappa does not assume

that raters are restricted in how cases are distributed across

category types.

For equivalency determination, participants had a kappa

score of .84, and for conceptual and constraint determina-

tion, a kappa score of .69; both indicate substantial

agreement among raters and a high degree of inter-rater

reliability. Based on feedback from participants, we are

adapting the test to an online environment in order to

further strengthen validity. In addition, we plan to conduct

further evaluation on the newly discovered heuristics for

merging phrases that employ the phrase-level metrics, as

reported in Table 2.

8.2 Internal validity

Internal validity is the extent to which observed causal

relationships exist within the data and, particularly, whe-

ther the investigator’s inferences about the data are valid

[34]. Each nominal measure is an inference that some

statement or phrase can be assigned to a corresponding

unary or binary relationship based on the metric’s defini-

tion. Because the binary metrics are asymmetric, an alter-

native explanation for the findings is that the water marks

are due to the order in which the comparisons occur, which

is a threat to internal validity. Thus, we conducted a water

mark chaining evaluation to test the commutative property

and found the same water marks are generated despite the

order of comparisons. We intend to further test this

assumption by examining other domains with less simi-

larity in the domain phenomenology.

In addition, we evaluated the manual method in a

validity study wherein four different analysts applied the

statement and phrase metrics after receiving a brief tutorial.

We found significant inter-rater reliability: using the Fleiss

Free-Marginal Kappa, we observed a score of 0.84 for

different analysts identifying statement-level equivalencies

and a score of 0.69 for identifying conceptual and con-

straint-based differences. These findings suggest the met-

rics in our study produce reliable results and, with more

training, we believe disagreement can be further narrowed

across different analysts.

8.3 External validity

External validity is the extent to which the framework

generalizes. United States data breach notification laws are

largely homogenous, as opposed to comparing laws from

finance to health care, which describe different domains

and different kinds of risk to privacy and security. As part

of our formative, exploratory study, we selected data

breach laws because the very near-similarity would reduce

complexity of developing a new method and prototyping

our water mark process. However, to strengthen external

validity, we conducted a second case study in a second

domain of medical record retention law. As expected, we

observed insightful differences that we discuss in Sect. 7.4.

Future work should continue to examine laws from mul-

tiple, different domains, as well as from different countries

of origin, to assess external validity of our guidance and

further refine and extend our heuristics for this type of

analysis. It is possible that our process generalizes to norms

that are not related to system requirements, which we may

explore in future work.

9 What the legal experts say

In addition to repeatability of the water marking method,

we are interested in how our results are reflected in applied

legal settings. As noted by Siena et al. [28] and Bobkowska

and Kowalska [2], legal and engineering viewpoints differ

and these differences must be accounted for when priori-

tizing compliance decisions. To address legal validity, we

engaged with legal subject matter experts to review our

results through semi-structured interviews, and we used the

responses obtained during these interviews to structure a

follow-on survey in order to reach a wider audience of

legal experts [3, 11].

9.1 Early interviews with legal experts

We conducted semi-structured interviews with three legal

experts to obtain feedback on the water marking process.

Prior to the interviews, the investigator (the first author)

provided each expert with general descriptions of the rec-

onciliation techniques and select conflicts from our dataset.

We then asked the expert which techniques they would

propose or they believed were currently practiced. We also

surveyed the perceived legal validity of reconciliations

produced by the union technique, given that the union often

includes derived requirements that are not present in any of

the input laws. The presented conflicts were chosen to

demonstrate the different heuristics or strategies prescribed

by the union technique, such as duplicating an action from

one jurisdiction in a second jurisdiction (e.g., preserving a

relational dissimilar requirement linked with REFINES).

We organized the expert responses around the following

questions:

How do legal experts identify and resolve conflicts

across jurisdictions? The experts responded that they

employ their past experiences and training to resolve

conflicts, often working directly with clients and within

their limited abilities, budgets, organizational structure, etc.

166 Requirements Eng (2013) 18:147–173

123

Companies may choose experts who are familiar with local

jurisdictional sensitivities, including which requirements

are routinely enforced or ignored, and experts may priori-

tize requirements differently based on their individual

judgment. The prioritization process can include political,

economic and technological issues, such as, is the State’s

Attorney General up for re-election, are the implementa-

tion costs for a requirement unreasonable, and has a tech-

nology changed to invalidate a regulatory requirement.

How do legal experts perceive the different reconcilia-

tion techniques of union, disjoint, and minimum? Our

experts generally responded positively to the reconciliation

techniques (high standard, separate standards, low stan-

dard) and grasped their intent, immediately. Respondents

generally agree that the disjoint water mark may be cum-

bersome, but posed no additional legal concern, as the legal

text in the requirements can remain unmodified. Although

they agreed that the proposed union water marks for the

requirements were ‘‘reasonable’’ and ‘‘legally fine’’, they

offered a number of valuable caveats:

• Sending notice to individuals for every breach may

appear as a higher standard (MA, NV, NY), than

sending it only when there is a risk of harm (AR, CT,

MS, WI); however, the latter approach avoids over-

inundating residents with notices and losing their

effectiveness. The aim of the notice is to encourage

residents to act when there is a risk of identity theft.

Thus, incorporating the rationale for a particular

requirement can aide in resolving these conflicts,

however, the elicitation and documentation costs can

limit the preservation of rationale [17]. In general, the

analyst should examine the underlying intent when

considering trade-offs that involve different frequencies

and not presume that better satisfying is always best,

e.g., more notice means more consumer awareness,

higher encryption key bits means more confidentiality,

etc.

• The union can introduce other parties who have their

own requirements into the business process, such as

obligation CT-10 to consult with law enforcement in

the event of a data breach. Implementing this practice

may further limit company autonomy, because law

enforcement can deliver advice that leads to new

requirements that conflict with existing regulations.

• Preserving an action from one regulation not present in

another may indirectly violate an unrelated requirement

in the other. To reuse the above example, consulting

with law enforcement may introduce an unacceptable

delay in a certain jurisdictions where this practice is not

prescribed. In this particular case, a preserved sub- or

post-process (REFINES or FOLLOWS) may produce an

undetected conflict with a quality attribute, e.g., delay

the notice conflicts with notifying consumers, expedi-

tiously. As mentioned, this particular example moti-

vated the need for an additional step in order to detect

such conflicts, as presented in Sect. 3.3.3.

• Requirements that are particularly difficult to reconcile

may best be resolved by choosing a self-imposed

standard that is higher than both, rather than risk

choosing one or the other and yield a gap in compli-

ance. For example, choosing to provide notice within a

specific time frame (e.g., ‘‘48 h’’) rather than allowing

the system to default to the legally required time frame

‘‘as soon as practicable’’ or ‘‘immediately following

discovery’’.

What do legal experts recommend to businesses? Our

experts indicated that the union technique ‘‘is a familiar

approach in law’’ as an organization will often pick the

most onerous standard, particularly if the regulations are

large. This remark is tempered with the belief that busi-

nesses only take on the more onerous standard provided

that there is not a ‘‘significant cost difference.’’ Regardless,

the organization ‘‘will always back up its decision by

having [a] business justification for [the decision].’’

Although respondents recognize the multiple standards

created using the disjoint technique, they often prefer this

approach over union, because it introduces less risk than

reinterpreting the law. No respondent advocated for the

minimum technique, citing its lack of compliance; how-

ever, one recognized that, due to resource constraints,

organizations may prioritize meeting certain jurisdictional

standards before others; e.g., ‘‘we have affected [individ-

uals] in every state but the majority of them are from [this

state and that state]; we want to avoid legal trouble in these

locations in particular.’’ When asked further about this, the

respondent admirably indicated ‘‘[I] would much rather a

client tries to do the best they can as opposed to saying ‘I

can’t afford this’ or ‘I can’t do anything.’’’ Respondents

acknowledged that differences in experience, past clients,

and area of focus could contribute to different opinions

between legal experts.

9.2 Summary surveys with legal experts

The survey protocol described in Sect. 5 was executed by

the first author and led to several important insights con-

cerning the water mark process. In response to our solici-

tation, we collected 20 responses in total, 5 of which were

incomplete and were thus not considered in our analysis.

The majority (13) of participants work in corporate set-

tings, with the remaining two participants holding positions

in academia. With regards to education, all participants

hold a Juris Doctor (JD), 6 hold other masters degrees, and

one has a PhD. In a free response question, their job duties

Requirements Eng (2013) 18:147–173 167

123

include: legal research; teaching privacy law; privacy and

security counseling; compliance advising; reviewing data

collection and vendor agreements; legal analysis of federal

privacy laws; and general legal and public policy matters.

We now discuss our findings from the survey data.

9.2.1 Perceived validity of union water mark

Participants’ responses regarding the perceived validity of

the union reconciliation can be seen in Table 4. Results for

all scenarios were positive, with over half of all responses

(67 %, or 60 responses) indicating that the requirement

generated by the union method would not put an organi-

zation in violation of the law. Notably, participants were

very divided with regards to scenarios 5 and 6, which

feature pre-conditions with the adverb not, which we called

negatively framed (e.g. ‘‘data the entity does not own…’’),

and non-modal requirements (i.e. ‘‘an entity that…’’),

respectively. Of the remaining 30 responses where the

respondent observed a violation of the law or that the

respondent was uncertain of the outcome of the union,

63 % or 19 responses came from scenarios 5 and 6, with

only 11 coming from the first four scenarios. This divide

can be observed in Table 4. We now discuss these

responses to scenarios 5 and 6.

In our analysis, 20/168 or 12 % of pre-conditions that

we encountered were negatively framed; that is, they were

framed in terms of what an organization does not do rather

than what the organization does. We found that reconciling

these negatively framed pre-conditions to be counterintui-

tive, which was supported during our validity study dis-

cussed in Sect. 8.1. When constructing the survey, we

selected an example that contains negatively framed pre-

conditions in order to see how legal experts would respond.

The text for this example, which was Scenario 5 in our

survey, is shown in Fig. 21, with italics added here for

emphasis.

In this example, observe the italicized negative ‘‘does

not’’ in the pre-conditions to be reconciled, CT-MS-NY-12

and NV-23. The third pre-condition CT-MS-NY-NV-10 is

presented as the reconciliation result of the two earlier pre-

conditions and is identical to pre-condition CT-MS-NY-12.

As part of the union, this reconciled pre-condition is

intended to reflect the broader case, resulting in coverage

for as many entities as possible. Despite this intent, how-

ever, many lawyers stated that this union would be in

violation of the law, because they believe that NV-23 is

broader than the proposed union CT-MS-NY-NV-10. For

the purposes of explanation, we found it beneficial to

visualize these requirements in a Venn diagram (see

Fig. 22). The domain (M) reflects entities that ‘‘maintain

computerized data which includes personal information’’;

(O) reflects entities that maintain computerized information

which includes personal information that they own, and

(L) entities that maintain computerized information which

includes personal information that they license. The cov-

ered entities established by each pre-condition above are

shaded.

When shown visually through a Venn diagram, it is

clear that the reconciled requirement has broader coverage,

given that the shaded area reflecting the entities covered is

larger for CT-MS-NY-NV-10 than NV-23. In positively

framed requirements, which compose the significant

majority of those we encountered, preservation of a clause

will yield a broader requirement: more organizations are

covered by ‘‘clause A, clause B, or clause C’’ than ‘‘clause

A or clause B.’’ When framed negatively, however, the

opposite is true, as ‘‘not clause A or not clause B’’ is less

restrictive than ‘‘not clause A, not clause B, or not clause

Table 4 Legal expert survey results by scenario

Scenario Is the union a violation of law?

(Yes, No, Uncertain)What do you recommend to businesses?

(U-Union, D-Disjoint, M-Minimum)What is the current state of practice

(U-Union, D-Disjoint, M-Minimum)

Yes No Uncertain U D M U D M

S1 3 12 0 6 9 0 5 8 2

S2 1 14 0 10 5 0 8 6 1

S3 1 11 3 7 7 1 9 4 2

S4 2 12 1 8 3 4 7 4 4

S5 7 5 3 7 8 0 7 8 0

S6 8 6 1 6 9 0 6 8 1

Fig. 21 Negatively framed pre-conditions from CT-MS-NY water

mark and NV §603A

168 Requirements Eng (2013) 18:147–173

123

C’’. Using a heuristic-based analysis wherein negatively

framed pre-conditions are merged with this observation in

mind, we believe that analysts can avoid this inconsistent

outcome.

Scenario 6 features two non-modal requirements, or pre-

conditions, that are implied permissions. When pre-condi-

tions are separated into non-modal requirements the

FOLLOWS relation is used to indicate that the subsequent

requirement follows from the separated pre-condition. In

scenario 6, these non-modal requirements describe when a

notification is required. The separate pre-conditions CT-4

and MS-2 and reconciled union option CT-MS-4 that was

presented to respondents can be found in Fig. 23, again

with italics were added here for emphasis.

In the above example, the reconciled requirement CT-

MS-4 preserves the broader term of ‘‘personal information’’

in place of the more specific term, ‘‘computerized data that

contains personal information,’’ and omits the additional

clause, ‘‘in the ordinary course of the person’s business

functions’’, which acts as a constraint on the act of owning,

licensing, or maintaining the information. Many of the

respondents who said the union was not in violation of the

law referenced the ‘‘broader’’ aspect of the union in their

open-ended responses. Dissenters, however, expressed more

caution. Although the union requirement CT-MS-4 covers

both situations described in CT-4 and MS-2, one dissenter

warned that the union can put an organization in violation of

the law, if a follow-on obligation to CT-MS-4 (e.g., to

provide notice) from one jurisdiction is a lower standard of

care than what would have otherwise exist in the other

jurisdiction prior to the union. However, the water mark

process preserves the higher standard of care independently

of pre- and post-conditions and other relations. For example,

if the analyst is reconciling two similar pre-conditions (one

of which is broader than the other) and two corresponding

obligations that follow these pre-conditions (one of which

imposes a higher standard of care than the other), then the

process will preserve both the broader pre-condition and the

higher standard of care. Because the survey only showed

individual requirements without this additional context, the

dissenting respondent was unable to see that the water mark

process responds to this threat to legal validity.

9.2.2 Recommended course of action

As shown in Table 4, respondents recommended the union

option (44/90, or 48 %) and disjoint option (41/90, or 45 %)

fairly evenly for all scenarios. When choosing the union,

respondents’ claimed it was not only ‘‘more manageable,’’

but also the ‘‘best practice.’’ This latter remark was further

supported as follows: ‘‘segregating [practices],’’ in reference

to the disjoint method, ‘‘would create very poor marketplace

optics,’’ or bad publicity for the covered organization. A few

responses commented on an underlying principle of the

union itself. One respondent indicated that practicing an

action outside of its jurisdiction of origin (a common effect of

Fig. 22 Broadening of coverage through negatively framed pre-conditions

Fig. 23 Non-modal

requirements used in legal

expert survey from MS-HB-583

and CT-36a-701b

Requirements Eng (2013) 18:147–173 169

123

the union) would be ‘‘in addition to, and not in conflict with,

the statutes that are silent as to the required response.’’ We

made a similar assumption that we describe in Sect. 3.3.1,

where relationally dissimilar requirements linked with

REFINES or FOLLOWS may be practiced in other jurisdic-

tions because they have no near equivalent requirement that

prescribes, prohibits, or otherwise addresses the same issue.

In two responses the union was chosen because the respon-

dents indicated that the differences between the require-

ments were effectively negligible, with one respondent

stating the requirements would ‘‘not produce differing

operational standards.’’

When respondents chose the disjoint water mark, we

interpreted their responses to be risk adverse. In contrast to

the previous remark that the differences between two

requirements would not produce differing operational

standards in the union, the respondents who preferred the

disjoint water mark indicated that the differences between

the two requirements were not immediately clear and, in

such cases, the disjoint should be taken in place of the

union. In this situation, the disjoint option is aimed at

avoiding the risk that the differences were actually signif-

icant and the union would thus conflict with this potential

interpretation. In addition, the disjoint option would be

chosen because the respondent was uncomfortable pro-

viding an opinion without knowledge of prior practice

within the covered jurisdictions or without further details

of cross-referenced laws, such a the E-Sign Act that gov-

erns the use of electronic notification in some jurisdictions.

Of most interest was a single response to the reconcili-

ation of a relationally dissimilar requirement in Scenario 2,

in which a respondent indicated that notifying a data subject

of the information that was affected by a breach could

actually be construed as a further breach itself. In this

scenario, Wisconsin requires that organizations, which are

contacted by individuals affected by a data breach, must

notify the individual of the information that had been

acquired during the breach (WI-28, WI-29). The other

jurisdiction in question, Mississippi, has no corresponding

requirement, and the union method prescribes following

this requirement in both jurisdictions. While 93 % of

respondents claimed that this practice would not put an

organization in violation of the law, one of the respondents

who recommended the requirements be kept disjoint

claimed that subsequent disclosures (i.e. notifying the

individual of the breached information) could ‘‘constitute a

further breach,’’ because the jurisdiction that does not have

this practice may have other rules in place in order to protect

against the possibility of fraudulent requests for information

in cases like this. This ‘‘extreme’’ interpretation of the term

‘‘breach’’ was taken only by a single respondent, but indi-

cates the wide variety of perspectives and factors some

experts consider in making their recommendation.

9.2.3 Experiences in practice

With regards to experiences encountered in practice, par-

ticipants’ responses reflected those of our original legal

experts, who provided a variety of motivations for an

organization choosing one approach over another, occa-

sionally being in direct conflict. While some claimed that

‘‘organizations… seek a common approach for all states

whenever possible’’, which would indicate that an organi-

zation would notify all affected individuals even if certain

jurisdictions did not have this obligation, others claimed

the direct opposite, that ‘‘most organizations do not want to

over notify’’ or that they will ‘‘do the minimum required’’.

Given the discrepancy in these responses, we suspect that

while organizations may be willing to take a higher stan-

dard of care in order to have uniform practices, they are

less willing take on an obligation that they are otherwise

not obligated to perform in any capacity. For example, if

one regulation states an organization must notify affected

individuals in a shorter time period than another regulation,

the organization will notify all individuals in the shorter

time period—but if one regulation specifies that an orga-

nization must notify affected individuals and the other

organization specifies that notification is not necessary if

the organization reasonably believes there is no risk of

harm, the organization may not notify the latter group.

A common stance for most respondents, though, was that

the organizations lacked familiarly with these laws: that

they didn’t ‘‘understand the nuances’’ or ‘‘minor require-

ments’’, or that they would ‘‘just send out basic breach

notifications…’’. Although few respondents indicated that

organizations would take the minimum approach, they

indicated that it happened with greater frequency than our

legal experts, with one response indicating that ‘‘most

breaches are not reported to law enforcement at all.’’

10 Discussion and summary

In this paper, we present a new method that combines pre-

viously validated techniques for extracting legal require-

ments from regulations and measuring differences between

two requirements sets, with new techniques for inferring

legal water marks for high and low standards of care across

multiple corresponding jurisdictions. We applied the com-

bined method to eight U.S. data breach notification (DBN)

laws and four medical record retention (MRR) laws. We

found that performing the union across the DBN domain

yields a reduction from 338 total requirements down to 80

requirements, which is a 76 % reduction. In the MRR

domain, we observed a much smaller reduction from 168

total requirements to 142 requirements, which is a 15 %

reduction. This stark difference reflects the dissimilarity

170 Requirements Eng (2013) 18:147–173

123

within each domain: in the DBN domain, laws were passed

relatively close together in time to address a common con-

cern and the laws in some cases appear to copy text from each

other. In the MRR domain, the laws were passed across a

longer period of time and address a much broader range of

issues in medical care. Furthermore, some states in the U.S.

play a more prominent role in the healthcare industry, which

may explain the increased refinement, level of detail and use

of domain-specific jargon that we observe in those jurisdic-

tions’ requirements. A contribution of the water marking

process is the method’s capability to distinguish these dif-

ferences with high fidelity to more accurately characterize

what an organization must do to comply with those laws.

We discovered the water mark process is commutative,

which means varying the order in which the analyst con-

structs the water mark does not produce a different outcome.

This discovery was done by generating specifications for a

subset of our jurisdictions (CT, MS, and NY) in which the

jurisdictions were reconciled in different orders: CT-MS-NY

and NY-MS-CT. The resulting water mark requirement

counts from the two orders were identical at 48 requirements

and the outcomes prescribed the same standard of care

measured using the metrics in this paper. Differences

between water marks were purely asthetic, such as the order

of reconciled phrases (e.g., ‘‘owns, licenses, or maintains’’

from one order appears as ‘‘owns, maintains, or licenses’’ in

the other order) or the identifier assigned to the requirement

is different (CT-MS-NY-14 versus NY-MS-CT-13). Inter-

mediate specifications (CT-MS, NY-MS) vary; however,

this is expected because they cover different jurisdictions.

Based on our interviews with legal experts, we believe

most companies appear somewhere between the union and

disjoint water marks in practice, and some companies may

appear below the minimum standard when faced with

resource constraints or when initially setting up their

internal compliance regime for a new domain. Based on the

feedback given during these initial interviews, we created

an additional step for post-water mark deconfliction, called

quality attribute validation, to address conflicts introduced

by the union water mark. Additionally, we used the inter-

view results to develop and administer a survey to a

broader sample of legal experts. From these survey results,

we observe how legal experts respond to the complexity of

negatively framed and non-modal requirements and we

collected information that illuminates how legal expert

recommendations guide clients in choosing a water mark

for their organization. This information not only served to

validate the existence of the water marks in practice, but

also provides critical context as to why one organization

might choose a higher or lower standard of care in industry.

During this analysis, we identified several opportunities

for improving the method. For example, by grouping

requirements into named categories (e.g., notification,

access, encryption, disposal) based on their action verbs, we

may be able to reduce the number of pairwise comparisons

required with a small loss in precision and recall (i.e., it is

possible to have equivalencies that cut across different cat-

egories). In addition, our expert reviewers noted how ratio-

nale can be used to resolve trade-offs by appealing to tacit or

undocumented regulatory and industry goals. For example, a

trade-off in which any decision (union or disjoint) would

yield a non-compliant outcome, the expert feedback may be

used to justify that a particular decision was at least a ‘‘best

effort’’ to an otherwise impossible legal landscape.

Finally, our method is primarily manual with tool sup-

port to encode the extracted requirements, produce visu-

alizations and record the comparison measures reported by

the analyst. To explore automation, we applied the ‘‘ideal’’

best IR-based technique reported by Falessi et al. [10] to

trace equivalent requirements pairs with the aim to improve

performance in Step 2 of our method in Fig. 1. This

technique is based on vector-space models with a Cosine

similarity measure, linear-incidence term weighting and

Stanford part-of-speech noun and verb extractor. With

respect to their dataset, this technique exhibited 0.935

precision and 0.936 recall with a 0.75 Lag, which measures

the number of true positives within a proportion of the

highest ranked results. Using our manually acquired results

from the DBN case study as the gold standard, the NLP

technique performed very poorly, with a 0.077 precision

and 0.300 recall. The reason for this discrepancy may be

the small size of the legal requirements (typically 10–20

words per requirement), whereas, NLP-based techniques

were originally developed by analyzing large corpus of

thousands of words. To our knowledge, automated trace-

ability methods have not yet advanced to a level where

they could automate application of our phrase-level mea-

sures. We see improvements in NLP-based analysis as a

welcome improvement in our research.

Acknowledgments We thank the CMU Requirements Engineering

Lab for participating in reviews of our research protocol and early

drafts on this manuscript, and we thank the International Association

of Privacy Professionals (IAPP) for allowing us to recruit survey

participants through their Global Privacy Summit. This research was

supported by the U.S. Department of Homeland Security (Grant

Award #2006-CS-001-000001) and Hewlett-Packard Labs Innovation

Research Program (Award #CW267287).

Appendix

The context-free grammar for an early version of the LRSL

is expressed here in the Extended Backus–Naur Form

(EBNF) described in ISO/IEC 14977 (1996E). The term

‘‘string’’ consists of any combination of letters and digits,

the term ‘‘regex’’ is a regular expression, and the term ref is

a string.

Requirements Eng (2013) 18:147–173 171

123

References

1. American Health Information Management Association (1999)

Practice Brief. Retention of Health Information (updated)

2. Bobkowska A, Kowalska M (2010) On efficient collaboration

between lawyers and software engineers when transforming legal

regulations to law-related requirements. In: 2nd International

Conference Information Technology, pp 105–109

3. Bogner A, Littig B, Menz W (2009) Interviewing experts. Pal-

grave Macmillan, UK

4. Breaux TB, Anton AI, Boucher K, Dorfman M (2008) Legal

requirements, compliance and practice: an industry case study in

accessibility. In: IEEE 16th International Req’ts Engr. Conf.,

pp 43–52

5. Breaux TD, Gordon DG (2011) Regulatory requirements as open

systems: structures, patterns and metrics for the design of formal

requirements specifications. Carnegie Mellon University Tech-

nical Report CMU-ISR-11-100

6. Breaux TD (2009) Legal requirements acquisition for the speci-

fication of legally compliant information systems. Ph.D. Thesis,

North Carolina State University

7. Bryan Cave LLP (2006) Wisconsin data-security law imparts

obligation to issue consumer notification in case of security

breach. Data Security Bulletin. http://www.bryancave.com

8. Corbin J, Strauss A (2007) Basics of qualitative research: tech-

niques and procedures for developing grounded theory, Sage

Publications, California, USA

9. Dekhtyar A, Dekhtyar O, Holden J, Hayes JH, Cuddeback D,

Kong W-K (2011) On human performance in assisted require-

ments tracing: statistical analysis. In: 19th IEEE International

Req’ts Engineering Conference, pp 111–120

10. Falessi D, Cantone G, Canfora G (2010) Comprehensive char-

acterization of NLP techniques for identifying equivalent

requirements. In: ACM-IEEE International symposium empirical

software engineering and measurement, vol 18, pp 1–10

11. Flick U (2009) An introduction to qualitative research, 4th edn.

Sage Publications Ltd, California, USA

12. Gacitua R Sawyer P Gervasi V (2010) On the effectiveness of

abstraction identification in requirements engineering. In: 18th

IEEE International Conference Req’ts. Engineering, pp 5–14

13. Gervasi V Zhowghi D (2011) Mining requirements links.In:

Req’ts Engneering: Fnd. Software Qual., LNCS, vol 6606, 96–201

14. Ghanavati S, Amyot D Peyton L (2009) Compliance analysis

based on a goal-oriented requirement language evaluation

methodology. In: IEEE 17th international requirements engi-

neering conference pp 133–142

15. Gordon DG, Breaux TD Managing Multi-jurisdictional require-

ments in the cloud: toward a computational legal landscape. In:

3rd ACM cloud computing security workshop (CCSW’11)

pp 83–94

16. Gordon DG, Breaux TD (2012) Reconciling multi-jurisdictional

requirements: a case study in requirements water marking. In:

20th IEEE international requirements engineering conference

17. Greenspan S (1993) Panel on recording requirements assump-

tions and rationale. In: IEEE international symposium req’ts

engineering, pp 282–285

18. Cleland-Huang J, Czauderna A, Gibiec M, Emenecker J (2010) A

machine-learning approach for tracing regulatory codes to prod-

uct specific requirements. In: IEEE international software engi-

neering conference, pp 155–164

19. Kroes N (2011) The clear role of public authorities in cloud

computing. Digital Agenda Comissioner—Neelie Kroes

20. Maxwell JC, Anton AI, Swire P (2011) A legal cross-references

taxonomy for identifying conflicting software requirements. In:

19th IEEE international req’ts engineering conference

pp 197–206

21. National Conference of State Legislatures (2012) State security

breach notification laws. Available https://www.ncsl.org/issues-

research/telecom/security-breach-notification-laws.aspx

22. Otto PN, Anton AI (2007) Addressing legal requirements in

requirements engineering. In: 15th IEEE International req’ts

engineering conference pp 5–14

23. Schlag PJ (1985) Rules and standards. 33 UCLA L. Rev., p 379

24. Randolph J (2005) Free-marginal multirater kappa (multirater

K[free]): an alternative to fleiss’ fixed-marginal multirater kappa.

Joensuu learning and instruction symposium

25. Rifaut A, Ghanavati S (2012) Measurement-oriented comparison

of multiple regulations with GRL. In: IEEE 5th workshop on

requirements engineering and law pp 7–16

26. Sabetzadeh M, Nejati S, Liaskos S, Easterbrook S, Chechik M

(2007) Consistency checking of conceptual models via model

merging. In:15th IEEE international req’ts. engineering confer-

ence pp 221–230

27. Siegel S, Castellan N (1988) Nonparametric statistics for the

social sciences. 2nd edn, McGraw-Hill, New York, USA

28. Siena A, Mylopoulos J, Perinir A, Susi A (2008) From laws to

requirements. In: 1st international work. req’ts engineering and

law, pp 6–10

29. Taber CW, Thomas CL (2009) Taber’s cyclopedic medical dic-

tionary. 21st edn, F.A. Davis Publications, Philadelphia, USA

30. United States Office of the Actuary (2009) State health expen-

diture accounts: state of provider 1980–2009. Available: http://

www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-

Trends-and-Reports/NationalHealthExpendData/NationalHealth

AccountsStateHealthAccountsProvider.html

start = header, body;header = "DOCUMENT" string, schema, "TITLE", ref, string;schema = "{", string, ":", regex, "}",([.-]?"{", string, ":",

regex, "}")*;body = (instruct | definition | rule)*;instruct = "SECTION", ref | "PAR", ref | "INCLUDE", ref, refdefinition = string, tab, ("=" | "~"), string, (tab, def_op,

string)*def_op = "&" | "|" "<"rule = actor_exp, rule_clause, rule_commandactor_exp = string, (tab+, act_op, string)+act_op = "&" | "|"rule_clause = tab, stringrule_command = tab, rule_command_word, ref, ("#", number)?rule_command_word = "REFINES" | "REFINED-BY" | "FOLLOWS"

| "PRECEDES" | "EXCEPT" | "EXCEPT-TO"

172 Requirements Eng (2013) 18:147–173

123

31. Urquhart J (2011) Regulation, automation, and cloud computing.

CNET. Available: http://news.cnet.com/8301-19413_3-200860

81-240/regulation-automation-and-cloud-computing

32. Warrens M (2010) Inequalities between multi-rater kappas.

Advances in data analysis and classification, pp 271–286

33. Weitzner D (2011) Privacy law scholars conference keynote

address, deputy chief technology officer in the white house office

of science and technology policy

34. Yin RK (2009) Case study research: design and methods. 4th edn,

Sage Publications, California, USA

35. Yu E (1993) Modeling organizations for information systems

requirements engineering. In: international symposium req’ts

engineering pp 34–41

36. Zou X, Settimi R, Cleland-Huang J (2010) Improving automated

requirements trace retrieval: a study of term-based enhancement

methods. Empir Soft Engr 15:119–146

Requirements Eng (2013) 18:147–173 173

123