Measuring Inconsistency over Sequences ofBusiness Rule Cases?

Carl Corea1, Matthias Thimm2, and Patrick Delfmann1

1 Institure for Information Systems Research, University of Koblenz-Landau2 Institute for Web-Science and Technologies, University of Koblenz-Landau

{ccorea,thimm,delfmann}@uni-koblenz.de

Abstract. In this report, we investigate (element-based) inconsistencymeasures for multisets of business rule bases. Currently, related worksallow to assess individual rule bases, however, as companies might en-counter thousands of such instances daily, studying not only individualrule bases separately, but rather also their interrelations becomes neces-sary, especially in regard to determining suitable re-modelling strategies.We therefore present an approach to induce multiset-measures from arbi-trary (traditional) inconsistency measures, propose new rationality pos-tulates for a multiset use-case, and investigate the complexity of variousaspects regarding multi-rule base inconsistency measurement.

Keywords: Inconsistency Measurement · Business Rule Bases · Culpa-bility Measurement

1 Introduction

In the context of Business Process Management, business rules are used as acentral artifact to govern the execution of company activities [6]. To this aim,business rules are modelled to capture (legal) regulations as a declarative busi-ness logic. Then, given a new process instance (denoted as a case), instance-dependent facts are evaluated against the set of business rules for reasoning atrun-time. For example, consider the following set of business rules in Figure 1(we will formalize syntax and semantics later) with the intuitive meaning thatwe have two rules stating that 1) platinum customers are credit worthy, and2) customers with a mental condition are not credit worthy. Then, given a newcustomer case, in the example a new loan application, the facts set is evaluatedagainst the rule set and the resulting rule base B1 can be used to reason aboutthe customer case.

? This research is part of the research project ”Handling Inconsistencies in BusinessProcess Modeling“, which is funded by the German Research Association (referencenumber: DE1983/9-1).

arX

iv:2

103.

0110

8v1

[cs

.AI]

1 M

ar 2

021

2 C. Corea et al.

Loan Application

platinumCustomer → creditWorthymentalCondition → ¬creditWorthy

Business Rules

Business Rule Base Instance

B1 = {platinumCustomer, mentalCondition,platinumCustomer → creditWorthymentalCondition → ¬creditWorthy}

platinumCustomer:True

mentalCondition:True

Fig. 1. Exemplary business rule base instance B1.

The observant reader might have noticed, that the shown example yields aninconsistency, i.e., the contradictory conclusions creditWorthy ,¬creditWorthy .In fact, this is a current problem for companies, which can result from modellingerrors in the business rules, or unexpected (case-dependent) facts. This problemhas widely been acknowledged and has been addressed by a series of recentworks, cf. e.g. [1, 3, 5].

While existing results allow to handle inconsistencies in a single business rulebase instance as shown above, in practice, companies often face thousands of suchinstances daily. For example, the retailer Zalando reported that 37 million caseswere executed in the first quarter of 2020 alone3. As we will show in this work,considering not only single rule bases individually, but rather the entirety of allcases and their interrelations, can yield valuable insights, especially in regardto inconsistency resolution. For example, consider the following rule set, andassume there were four customer cases (with respective case-dependent facts),yielding the set of business rule cases M1 shown in Figure 2:

Rule Base:

a → bc → ¬bb → xx → zy → ¬z

Set of Rule Base Instances (with case-dependent facts)

a, ca → bc → ¬bb → xx → zy → ¬z

a, ca → bc → ¬bb → xx → zy → ¬z

a, ya → bb → xx → zy → ¬z

c → ¬bcaa → bb → xx → zyy → ¬z

c → ¬b

Fig. 2. Exemplary rule base instances, constructed over a seq. of case-dependent facts.

3 https://zln.do/2SFRnfC

Measuring Inconsistency over Sequences of Business Rule Cases 3

When auditing such an overview of rule base instances, two main questionsare of interest from a business rules management perspective:

1. How inconsistent in general was the entirety of process executions?2. Which specific rules were responsible for these inconsistencies from a global

perspective?

Towards question 1, recent results studying inconsistency in sets of knowledgebases can easily be adapted to quantify the overall degree of inconsistency (cf.Section 2). While this is a beneficial step for companies, question 2 can howeverbe seen as of much higher importance in the scope of improving business rules.Pin-pointing the culprits of inconsistency is an essential challenge for deter-mining suitable resolution and re-modelling strategies. Here, new methods areneeded that support companies in assessing which individual rules are highlyproblematic from a global perspective. For instance, in Figure 2, the rule a→ bis part of all inconsistencies and can therefore be seen as highly problematic. Inthis work, we therefore introduce novel means for an element-based assessmentof inconsistency over a set of rule base instances by extending results from thefield of inconsistency measurement [14]. Here, our contribution is as follows:

We present a novel approach for inducing element-based quantitative mea-sures for multisets of business rule bases, allowing to pin-point problematic busi-ness rules from a global perspective (Section 3). Here, we also propose postulatesthat should be satisfied by respective measures for this use-case and analyze theproposed means w.r.t. these postulates. We implement our approach and performrun-time experiments with real-life data-sets, and also examine the complexity ofcentral aspects regarding inconsistency measurement in multisets of business rulebases (Section 4). We present preliminaries in Section 2 and conclude in Section5. Proofs for technical results are provided in a supplementary document4.

2 Preliminaries

Business Rule Bases. In this work, we consider a basic (monotonic) logicprogramming language to formalise business rule bases. A (business) rule baseis then constructed over a finite set A of atoms, with L being the correspondingset of literals, with a rule base B being a set of rules r of the form

r : l1, . . . , lm → l0. (1)

with every li ∈ L. Let B denote all such rule bases. Also, we denote head(r) = l0and body(r) = {l1, . . . , lm}. If body(r) = ∅, r is called a fact. For a rule base B,we denote F(B) ⊆ B as the facts in B and R(B) ⊆ B as the rules in B.

Example 1. We recall the business rule base B1. Then we have

F(B1) = {mentalCondition, platinumCustomer}R(B1) = {platinumCustomer → creditWorthy,

mentalCondition→ ¬creditWorthy}.4 https://bit.ly/2V2sIDw

4 C. Corea et al.

A set of literals M is called closed w.r.t. B if it holds that for every rule ofthe form 1: if l1, . . . , lm ∈ M then l0 ∈ M . The minimal model of a rule base Bis the smallest closed set of literals (w.r.t. set inclusion). A set M of literals iscalled consistent if it does not contain both a and ¬a for an atom a. We say arule base B is consistent if its minimal model is consistent. If B is not consistent,we say B is inconsistent, denoted as B |=⊥.

As discussed in the introduction, B1 is not consistent. To assess inconsistency,the field of inconsistency measurement [7] has evolved, which studies quantitativemeasures to assess the severity of inconsistency. An inconsistency measure [7,14]is a function I : B → R∞≥0, where the semantics of the value are defined suchthat a higher value reflects a higher degree, or severity, of inconsistency. A basicinconsistency measure is the IMI inconsistency measure, which counts the numberof minimal inconsistent subsets MI of a rule base B, defined via

MI(B) = {M ⊆ B |M |=⊥,∀M ′ ⊂M : M ′ 6|=⊥}.

Example 2. We recall B1. Then we have

MI(B1) = {M1}M1 = {platinumCustomer,

platinumCustomer → creditWorthy,

mentalCondition,

mentalCondition→ ¬creditWorthy},

consequently, IMI(B1) = 1.

As the concept of a ”severity“ of inconsistency is not easily characterisable,numerous inconsistency measures have been proposed, see [14] for an overview.To guide the development of inconsistency measures, various rationality postu-lates have been proposed, cf. [13] for an overview. For example, a widely agreedupon property is that of consistency, which states that an inconsistency measureshould return a value of 0 w.r.t. a rule base B iff B is consistent. As mentioned,various other postulates exist and we will revisit some of them later when intro-ducing culpability measures for multisets of rule bases.

Measuring Overall Inconsistency in Multisets of Business RuleBases. In this work, we are not only interested in measuring inconsistency insingle business rule bases, but rather in a series of corresponding business rulebase instances. As motivated in the introduction, companies currently apply aset of business rules in order to assess a stream of (case-dependent) fact sets.Therefore, given a stream of fact sets f = F1, ...,Fn, we consider multisets ofbusiness rule bases which are constructed by matching the individual fact setsin f to a shared rule set R. To clarify, a multiset of rule bases is an n-tupleM = ({F1 ∪R}, ..., {Fn ∪R}) = (B1, ...,Bn). Let M denote all such multisets.

An initial question for companies is to gain an overview of the overall in-consistency w.r.t. all business rule base instances in M. For that, we define aninconsistency measure for a multiset of rule bases as follows.

Measuring Inconsistency over Sequences of Business Rule Cases 5

Definition 1 (Multi-B Inconsistency Measure). An inconsistency measurefor a multiset of rule bases is a function m : M→ R∞≥0.

In other words, an inconsistency measure for a multiset of rule bases is afunction that assigns a non-negative numerical value to an n-tuple of rule bases.Similar to classical inconsistency measures, the intuition is that a higher valuereflects a higher degree of inconsistency of the multiset of rule bases. For sim-plicity, we refer to such measures as multi-rb measures where appropriate.

For the intended use-case of gaining insights about the severity of inconsis-tency regarding the entirety of process instances, existing inconsistency measurescan be adapted to induce multi-rb measures via a summation.

Definition 2 (Σ-induced multi-rb Measure). Given an inconsistency mea-sure I and a multiset of rule-bases M, the Σ-induced multi-rb measure mΣ

I isdefined as mΣ

I : M→ R∞≥0 with mΣI (M) =

∑B∈M I(B).

Example 3. We recall the introduced MI-inconsistency measure IMI. Correspond-ingly, given a multiset of rule basesM, IMI can be used to Σ-induce the multi-rbmeasure mΣ

IMI(M) =

∑B∈M IMI(B). Considering again the exemplary mul-

tiset M1 of business rules from Figure 1, with cases b1 − b4, we thus havemΣIMI

(M1) = IMI(b1) + ...+ IMI(b4) = 1 + 1 + 1 + 2 = 5.

Note that the approach in [12], who—roughly speaking—measures inconsis-tency in a multiset of knowledge bases by performing a multiset union on allsets and then measuring inconsistency on this union, is not applicable for ouruse-case, as we are not interested in the disagreement between the individualinstances, but rather want to gain an overview of inconsistencies in all instances.

While the above discussion showed how existing means can be used to mea-sure the overall degree of inconsistency for a multiset of rule bases, in the fol-lowing, we develop techniques for an element-based assessment of inconsistencyover a multiset of rule bases.

3 Culpability Measures for Multisets of Business RuleBases

In the field of inconsistency measurement, a culpability measure C [8] is a functionthat assigns a non-negative numerical value to elements of a rule base. Thisquantitative assessment is also referred to as an inconsistency value. Again, theintuition is that a higher inconsistency value reflects a higher blame, that thespecific element carries in the context of the overall inconsistency. In this section,we investigate culpability measures that can assess the blame that a rule carriesin the context of the overall multiset inconsistency.

3.1 Baseline measures and basic properties

Given a multiset of business rule basesM, let R(M) denote the shared rule setof the respective business rule bases in M. Furthermore, let RM denote the set

6 C. Corea et al.

of all possible rules that can appear in these shared rule sets. Then, a culpabilitymeasure for a multiset of rule bases is defined as follows.

Definition 3 (Multi-B Culpability Measure). A culpability measure for amultiset of rule bases is a function Cm : M× RM → R∞≥0.

Similar to Σ-induced inconsistency measures, existing culpability measurescan be exploited to entail Σ-induced culpability measures.

Definition 4 (Σ-induced multi-rb culpability measure). Given a culpa-bility measure C, a multiset of rule-bases M and a rule r ∈ R(M), a Σ-inducedmulti-rb culpability measure mΣ

C is defined as mΣC : M × RM → R∞≥0 with

mΣC (M, r) =

∑B∈M C(B, r).

Two baseline measures proposed in [9] are the CD and C# measures.

Definition 5. Let a rule base B and a rule r ∈ B, then

– CD(B, r) =

{1 if ∃M ∈ MI(B) : r ∈M0 otherwise

– C#(B, r) = |{M ∈ MI(B) | r ∈M}|

Using Σ-induction, we can use these baseline culpability measures to entailthe multi-rb culpability measures mΣ

CD and mΣC# .

Example 4. We recall the multiset of rule basesM1 from Figure 2. For the shownrule a→ b, we have that

mΣCD (M1, a→ b) = 4

mΣC#(M1, a→ b) = 5

Regarding multi-rb culpability measures, we propose the following rational-ity postulates based on an application of postulates for traditional culpabil-ity measures [8]. For that, we consider a multiset of business rules M anda rule r ∈ R(M). Also, we define a rule r ∈ R(M) as a free formula ifr /∈ M, ∀M ∈

⋃b∈MMI(b). We denote the set of all free formulas of R(M)

as Free(M). We then propose the following postulates.

Rule Symmetry (RS) Cm(M, r) = Cm((B1, ...Bn), r), for any permutation ofthe order of B1 to Bn.

Rule Minimality (RM) if r ∈ Free(M), then Cm(M, r) = 0.

The first postulate states that the order of rule bases in the multiset should notaffect the inconsistency value of an individual rule. The second postulate statesthat the inconsistency value of a rule is zero if this rule is a free formula w.r.t.the multiset of business rule bases.

Proposition 1. mΣCD and mΣ

C# satisfy RS and RM.

Measuring Inconsistency over Sequences of Business Rule Cases 7

The second postulate was adapted from a postulate for traditional culpabilitymeasures, namely

Minimality (MIN) Let a rule r ∈ B, if r 6∈ M,∀M ∈ MI(B), then the incon-sistency value of r is zero.

As this is a commonly satisfied postulate, this allows for a generalization of theprevious proposition.

Proposition 2. Any Σ-induced multi-rb culpability measure satisfies RS. Givena culpability measure C satisfying MIN, any multi-rb culpability measure Σ-induced via C satisfies RM.

In the following, given a multiset of business rules M and a multi-rb cul-pability measure Cm, we consider all rules of R(M) as a vector (r1, ...rn), anddenote V C

m

(M) as the vector of corresponding multi-rb culpability values of allrules in R(M) w.r.t. Cm, i.e., V C

m

(M) = (Cm(M, r1), ..., Cm(M, rn)). Next, letV̂ C

m

(M) = max r∈R(M)(Cm(M, r)) denote the largest multi-rb culpability valuew.r.t. Cm for all rules. Last, we denote adding a rule r to the shared rule setR(M) of a multiset M as M∪ {r} by a slight missuse of notation, i.e., givenM = (B1, ...,Bn),M∪{r} = (B1 ∪ {r}, ...,Bn ∪ {r}). This allows to adapt somefurther desirable properties.

Multiset Consistency (CO) V̂ Cm

(M) = 0 iff @B ∈M : B |=⊥.Multiset Monotony (MO) Let a multiset of business rule basesM and a rule

r, V̂ Cm

(M∪ {r}) ≥ V̂ Cm(M)Multiset Free formula independence (IN) If a rule r is a free formula of

(M∪ {r}), then V̂ Cm

(M∪ r) = V̂ Cm

(M)

The first property states that the largest multi-rb culpability value for a rule canonly be zero if all business rule bases of the multiset are consistent. The secondproperty demands that adding a rule to the shared rule set can only increasethe culpability values. Similar to this property, the third postulate demands thatadding a free formula to the shared rule set does not alter the culpability values.

Proposition 3. mΣCD and mΣ

C# satisfy CO, MO and IN.

Next to the introduced baseline culpability measures CD and C#, variousother culpability measures have been proposed (cf. [10]), which could also be usedto Σ-induce multi-rb culpability measures. While an analysis of such measuresw.r.t. the introduced postulates could be interesting, we refrain from such aspecific analysis and rather show a more generalized approach in the followingsection, namely how arbitrary inconsistency measures can be used to inducemulti-rb culpability measures using Shapley inconsistency values.

3.2 (Adjusted) Shapley Inconsistency Values for Multi-RBmeasures

Next to designing specific culpability measures, an important approach in element-based analysis is to decompose the assessment of inconsistency measures (in

8 C. Corea et al.

order to derive corresponding culpability measures) by means of Shapley incon-sistency values [8]. Given an inconsistency measure I and a rule base B, theintuition is that the overall blame mass I(B) is distributed amongst all elementsin B, by applying results from game theory. The advantage of this approach isthat arbitrary inconsistency measures can be applied to derive a correspondingelement-based assessment. The amount of blame that an individual element isassigned relative to I(B) is also referred to as the payoff.

Definition 6 (Shapley Inconsistency value [8]). Let I be an inconsistencymeasure, B be a rule base and α ∈ B. Then, the Shapley inconsistency value ofα w.r.t. I, denoted SIα is defined via

SIα(B) =∑B⊆B

(b− 1)!(n− b)!n!

(I(B)− I(B \ α))

where b is the cardinality of B, and n is the cardinality of B.

Example 5. Consider the rule base B2 = {a, a → b, a → ¬b}. Then, for theShapley inconsistency values w.r.t. IMI, for all elements e in B we have thatSIMIe (B2) = 1

12 + 14 = 1

3 .

As shown in Example 5, all elements were assigned an equal payoff. Thismakes sense w.r.t. IMI if all elements in the rule base are considered with equalimportance. However, in our setting, knowledge contained in rule bases is dis-tinguished into facts and rules. Here, facts have a different veracity than rules,as they are usually provided by a given (non-negotiable) case input and haveto be kept ”as-is” [6]. In the scope of inconsistency resolution, we are thereforeonly interested in identifying blamable rules, as these should be considered forre-modelling. Consequently, an element-based assessment for rule bases shouldonly assign a payoff to blamable rules, and not facts. Correspondingly, this has tobe considered when applying Shapley’s game theoretical approach to distributea blame mass over all elements. As recently discussed in [3], the Shapley in-consistency value can accordingly be adjusted as follows. For that, let Free(B)denote the free formula in a rule base B, i.e., all r ∈ B : r 6∈M,∀M ∈ MI(B).

Definition 7 (Adjusted Shapley Inconsistency Value [3]). Let I be a rule-based inconsistency measure, B be a rule base and α ∈ B. Then, the adjustedShapley inconsistency value of α w.r.t. I, denoted S∗Iα is defined via

S ∗Iα (B) =

0 if α ∈ F(B)∑B⊆B

(CoalitionPayoff Iα,B(B) + AdditionalPayoff Iα,B(B)) otherwise

with

CoalitionPayoff Iα,B(B) =(b− 1)!(n− b)!

n!(I(B)− I(B \ α))

Measuring Inconsistency over Sequences of Business Rule Cases 9

being the payoff for an element for any coalition B ⊆ B, and

AdditionalPayoff Ir,B(B) =

0 if r ∈ Free(B)∑f∈F(B) CoalitionPayoff If,B(B)

|r′∈R(B) s.t. r′ /∈Free(B)| otherwise

being the additional payoff that blamable rules receive, by shifting the blame massfrom (given) facts to blamable rules.

Example 6. Consider the rule bases B2 = {a, a→ b, a→ ¬b} and B3 = {a, a→b,¬b}. Then for the adjusted Shapley inconsistency values w.r.t. IMI, we havethat S∗IMI

a (B2) = 0, S∗IMI

a→b (B2) = 13 (+ 1

3/2) = 12 , and S∗IMI

a→¬b (B2) = 13 (+ 1

3/2) =12 . Also, we have that S ∗IMI

a (B3) = 0, S ∗IMI

¬b (B3) = 0, and S ∗IMI

a→¬b (B3) =13 (+ 2

3 ) = 1 . For the first rule base B2 (containing two rules), the blame is evenlydistributed amongst both rules. For the second rule base B3, the single rulereceives the entire blame. This assessment makes sense in a business rule setting,as the given fact input is evaluated against a set of humanly modelled rules, andany inconsistencies arise due to modelling errors in the set of business rules.The adjusted Shapley inconsistency values can thus be used for pin-pointingproblematic rules for re-modelling purposes.

This element-based measure can consequently also be used to identify prob-lematic rules over a multiset of cases, by inducing the corresponding multi-rbmeasure mΣ

S∗I (cf. Example 7). As mentioned, an advantage of using the adjustedShapley measure is that arbitrary inconsistency measures can be used to derivean element-based assessment over a set of cases, based on company needs. Fur-thermore, we can identify the following properties for an adjusted Shapley valuemΣS∗I . For this, we assume that any inconsistency measure I used to derive ad-

justed Shapley inconsistency values satisfies the basic properties of consistency’,monotony’ and free formula independence’ as defined in [8]5.

Proposition 4. The adjusted multi-rb shapley inconsistency value satisfies CO,MO and IN.

Also, regarding the relation of inconsisteny measures and the correspondingΣ−induced Shapley inconsistency values for multi-rb analysis, we propose thefollowing postulates.

Distribution (DIS)∑α∈R(M)m

ΣS∗I (M, α) = mΣ

I (M)

Upper Bound (UB) V̂ S∗I

(M) ≤ mΣI (M)

The first postulate states that the sum adjusted multi-rb Shapley inconsistencyvalues over all rules is equal to the overall blame mass of the original multi-rb inconsistency measure I (used as a parameter to derive the corresponding

5 Let a rule base B and an inconsistency measure I. Consistency’ states that I(B) = 0iff B is consistent. Monotony’ states that if B ⊆ B′ then I(B) ≤ I(B′). Free formulaindependence’ states that If α ∈ Free(B) then I(B) = I(B \ {α}).

10 C. Corea et al.

Shapley values). Also, the second property states that the adjusted multi-rbShapley inconsistency values for an individual element cannot be greater thanthe overall assessment of the original multi-rb inconsistency measure I.

As we are only interested in identifying problematic rules (e.g. for re-modelling),it would also be plausible to adapt the property of fact minimality as proposedin [3] for a multi-rb use-case.

Fact Minimality (FM) mΣSI (M, α) = 0∀α /∈ R(M).

This property states that (non-negotiable) facts should not be assigned anyblame value in an element-based multi-rb assessment.

Proposition 5. The adjusted multi-rb shapley inconsistency value satisfies DIS,UB and FM.

We conclude with an example illustrating the introduced multi-rb measures.

Example 7. We recall the set of business rule bases M1 from Figure 2 and itsshared rule setR(M1) = {r1, ..., r6}. A multi-rb assessment w.r.t. the introducedmeasures is then as follows.

r1 : a→ b mΣCD (M1, r1) = 4 mΣ

C#(M1, r1) = 5 mΣS∗IMI

(M1, r1) = 2

r2 : c→ ¬b mΣCD (M1, r2) = 3 mΣ

C#(M1, r2) = 3 mΣS∗IMI

(M1, r2) = 1.5

r3 : b→ x mΣCD (M1, r3) = 2 mΣ

C#(M1, r3) = 2 mΣS∗IMI

(M1, r3) = 0.5

r4 : x→ z mΣCD (M1, r4) = 2 mΣ

C#(M1, r4) = 2 mΣS∗IMI

(M1, r4) = 0.5

r5 : y → ¬z mΣCD (M1, r5) = 2 mΣ

C#(M1, r5) = 2 mΣS∗IMI

(M1, r5) = 0.5

mΣIMI

(M1) = 5

As can be seen, rule r1 is classified as most problematic my all measures. Thismakes sense, as this rule is a cause of inconsistency over all cases forM1. Hence,this rule should be prioritized in the scope of re-modelling and improving the setof business rules. Here, our proposed approach of multi-case inconsistency mea-surement can support modellers in identifying highly problematic rules from aglobal perspective, by recommending an order in which rules should be attendedto, e.g. < r1, r2, r3, r4, r5 > for the shown example. A further important aspectfor an application in practice is that the proposed measures can be combinedto obtain multivariate metrics for a more fine-grained analysis. For example,as the mΣ

CD (r) is equivalent to the number of distinct cases in which a rule ris part of an inconsistency, this measure can be used to normalize and explainother measures. For example, a normalization via mΣ

CD can be used to explain if

a high mΣC# -value originates from a few highly inconsistent cases (which might

be outliers) or if the corresponding rule contributes a smaller amount towardsinconsistency but in a vast majority of cases (in which case it might be sensibleto re-consider this rule).

Measuring Inconsistency over Sequences of Business Rule Cases 11

In this section, we have shown how arbitrary culpability measures can betransformed into multi-rb measures. Also, we have shown that the proposedmeasures satisfy desirable properties. Our results are summarized in Table 16.

Cm RS RM CO MO IN DIS UB FM

mΣCD 3 3 3 3 3 n/a n/a 7

mΣC# 3 3 3 3 3 n/a n/a 7

mΣSI 3 3c,i 3c,i 3m 3i 3 3 3

c: If I satisfies consistency’m: If I satisfies monotony’i: If I satisfies free formula independence’

Table 1. Compliance with rationality postulates of the investigated measures.

4 Tool Support and Evaluation

We implemented our approach to assess element-based inconsistency over se-quences of business rule cases7. Our implementation takes as input a sharedbusiness rule base and a sequence of fact sets, and can then computes the mostproblematic rules w.r.t. the multiset of rule bases. The mΣ

CD and mΣC# mea-

sures can be used out-of-the-box, however, arbitrary culpability measures canbe added based on company needs. To evaluate our tool, we then performedrun-time experiments and investigated the computational complexity regardingvarious aspects of measuring inconsistency over sequences of business rule cases.

4.1 Run-Time Experiments

In the following, we present the results of run-time experiments with real-lifeand synthetic data-sets.

Evaluation with real-life data sets. To evaluate the feasibility of applyingour approach in practice, we conducted run-time experiments with real-life datasets of the Business Process Intelligence (BPI) challenge8. This yearly scientificchallenge from the field of process management provides real-life process logsfor evaluating approaches in an industrial setting. In a nutshell, we mined arule set from each event log and then measured inconsistency over all cases ofthe respective log. Here, we analyzed the data-sets from the last four years,i.e., BPI’17 (log of a loan application process with 31,509 cases), BPI’18 (logof a fund distribution process with 43,809 cases), BPI’19 (log of an applicationprocess with 251,734 cases), and BPI’20 (log of a travel expense claim processwith 10,500 cases). From these event logs, declarative constraints of the generalbusiness rule form in (1) can be mined using the results from [5]. In this way, we

6 Proofs can be found in the supplementary document (https://bit.ly/2V2sIDw)7 https://gitlab.uni-koblenz.de/fg-bks/multi-rb-inconsistency-measurement/8 https://data.4tu.nl/search?q=bpi+challenge

12 C. Corea et al.

were able to mine a rule set from each of the provided data sets. The resultingnumber of rules for the respective rule sets is provided in Table 2. Also, the minedrule sets can be found online9. We refer the reader to [5] for further details onthe mining technique. Then, for each data set, we analyzed all cases as follows:

For each data set, a shared rule base R was mined as described above. Then,for all cases C1, ..., Cn, the individual case-dependent fact inputs F1, ..., Fn wereextracted from the log. We then constructed a multiset of rule bases B1, ..., Bn,where every Bi = (R,Fi). We then applied our implementation to analyze in-consistencies over B1, ..., Bn and measured the run-time. The results of our ex-periments are shown in Table 2. The experiments were run on a machine with 3GHz Intel Core i7 processor, 16 GB RAM (DDR3) under macOS.

Dataset # of Rules # of Cases Runtime # of inconsistent Cases

BPI’17 50 31.509 5657s 31.509 (100%)

BPI’18 84 43.809 3967s 0 (0%)

BPI’19 51 251.734 1610s 434 (0.17%)

BPI’20 330 10.500 1329s 323 (3.07%)

Table 2. Runtimes for analyzing all cases for the considered BPI data sets

As can be seen, an analysis of all cases was feasible for all data sets.A central assumption of our approach is that a global perspective over all

cases should be considered as opposed to viewing cases individually. Interestingly,this was also confirmed by our experiments with the above real-life data sets:

For every individual rule base instance, we computed the C# values for allrules and then ranked all rules by this value (rank 1 meaning that this rule isthe most problematic element, and so on). If n rules had the same C# value,they were assigned the sum of the occupied ranks divided by n (e.g. if two ruleshad the highest C# value, they were awarded the rank (1+2)/2 = 1.5, and thenext rule had the rank 3). Figure 3 shows the distribution of all assigned ranksfor the rules for the BPI’17 data set over all cases. For readability, rules that didnot participate in any inconsistencies are omitted.

r1 r2 r3 · ··

r26

1

10

20

30

Rank

Fig. 3. Rank distribution for the individual rules of the BPI’17 rule set over all cases.

9 https://bit.ly/365Vs4C

Measuring Inconsistency over Sequences of Business Rule Cases 13

While there were some rules that had the same rank in all cases (e.g. r2),there were many rules where the respective local rankings had a large variability(e.g. r3). This shows that the global perspective as proposed in this work shouldbe strongly considered in the scope of auditing.

In general, we see the above experiments as positive in regard to applyingour approach in practice. As the analyzed data-sets were unrelated, no furthercomparison of run-times can be made. Therefore, we further assess our approachwith synthetic data sets.

Evaluation with synthetic data sets. We created a generator for syntheticrule base instances. Our generator can produce set of business rule base instances,based on a shared rule set and a sequence of fact sets relative to this rule set. Asparameters, our generator takes the desired rule base size and a desired numberof cases. Then, the generator constructs a multiset of rule bases as follows:

The set of business rule instances is constructed over a (potentially infinite)alphabet A =< a, b, ... >. Then, for a desired number of rules nr, a rule set R ={r1, ..., rnr} is generated, where every ri is of the form Ai → ¬Ai+1. For example,for a parameter of 2 desired rules, the resulting rule set is R = {a→ ¬b, b→ ¬c}.To then generate a desired number of random cases nc, a multiset of fact setsF = {F1, ..., Fnc} is initialized. Then, each of the fact sets is populated by addingatoms of the rule base, based on a user-defined probability. For example, for theexemplary rule set R of size 2, a random fact set relative to R could be anyelement of {∅, a, b, c, ab, ac, bc, abc}. In this way, the generator can create a set ofrandom rule base instances B = {B1, ..., Bnc}, where every Bi = (R,Fi).

An advantage of our generator is that the structure of the contained rules issimilar in all rule bases, which thus allows for better comparability. We con-sequently used our generator to analyze multiple sets of business rule caseswith different parameters and measured the run-times. As parameter settings,based on the observed sizes from the real-life data-sets, we selected as pa-rameters the rule base size from 10,20,...,100, and the number of cases from10.000,20.000,...,100.000 and then tested every possible combination. Thus, a to-tal of 100 (10x10) different configurations were tested (see above for hardware).The results of our experiments are shown in Figure 4. The smallest setting (10rules and 10.000 cases) took around 90s, where the largest configuration (100rules and 100.000 cases) took around 50 minutes. As can be seen, the run-timescales proportionally with the size of the rule base and the number of cases.Thus, we could not identify any of these two factors to be a dominant limitingfactor to the run-times in our experiments.

To summarize, both the evaluation with real-life data sets and with syn-thetic data sets yielded feasible run-times. To extend this empirical analysis,we continue with an investigation of computational complexity in regard to ourproposed approach.

14 C. Corea et al.

2040

6080

100

50T

100T0

2,000

4,000

Rule base sizeNumber

of cases

Ru

n-t

ime

inse

con

ds

Fig. 4. Run-times for the analysis of 10x10 synthetic sets of rule base instances

4.2 Complexity Analysis10

We assume familiarity with basic concepts of computational complexity andbasic complexity classes such as P and NP, see [11] for an introduction. We firstobserve that the satisfiability problem for our formalism of business rules basesis tractable (note that similar observations have been made before on similarformalisms, see e. g., [4]).

Proposition 6. Let B be a rule base. The problem of deciding whether B isconsistent can be solved in polynomial time.

Then, the complexity of deciding whether a certain rule is contributing tothe overall inconsistency is as follows.

Proposition 7. Let M be a multiset of rule bases with M = (B1, ...,Bn) =({F1 ∪ R}, ..., {Fn ∪ R}) and let r ∈ B. The problem of deciding whether thereis a i ∈ {1, . . . , n} and M ∈ MI(Bi) s. t. r ∈M is NP-complete.

The following two results deal with the computational complexity of com-puting the baseline measure C#.

Proposition 8. Let B be a rule base and M ⊆ B. The problem of decidingwhether M ∈ MI(B) can be solved in polynomial time.

For our final result note that #P is the complexity class of counting problemswhere the problem of deciding whether a particular element has to be countedis in P, cf. [15].

Proposition 9. Let M be a multiset of rule bases with M = (B1, ...,Bn) =({F1 ∪ R}, ..., {Fn ∪ R}) and let r ∈ R. The problem of determining |{M ∈MI(Bi) | i ∈ {1, . . . , n}r ∈M}| is #P-complete.10 Proofs can be found in the supplementary document (https://bit.ly/2V2sIDw)

Measuring Inconsistency over Sequences of Business Rule Cases 15

5 Conclusion

In this work, we have shown how arbitrary culpability measures (for single rulebases) can be automatically transformed into multi-rb measures while maintain-ing desirable properties. This is highly needed in practice, as companies are oftenfaced with thousands of rule bases daily, and thus need means to assess incon-sistency from a global perspective. Here, our proposed measures can be used togain fine-grained insights into inconsistencies in sequences of business rule bases.For the analyzed (real-life) data-sets, the proposed multi-cases analysis could beperformed in a feasible run-time. Intuitively, the number of cases or the sizeof the rule base affect the run-time of our approach. Here, we plan to developmore efficient algorithms in future work. As a main takeaway, our results indi-cate that the interrelations of individual cases need to be considered for businessrules management, which should be addressed more in future works.

References

1. Corea, C., Deisen, M., Delfmann, P.: Resolving inconsistencies in declarative pro-cess models based on culpability measurement. In: 15. Internationale TagungWirtschaftsinformatik, WI 2019 (2019)

2. Corea, C., Thimm, M.: On quasi-inconsistency and its complexity. AI 284 (2020)3. Corea, C., Thimm, M.: Towards inconsistency measurement in business rule bases.

In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI2020), Santiago de Compostela, Spain, 2020 (2020)

4. Dantsin, E., Eiter, T., Gottlob, G., Voronkov, A.: Complexity and expressive powerof logic programming. In: Proceedings of the 12th Annual IEEE Conference onComputational Complexity (CCC’97). pp. 82–101 (1997)

5. Di Ciccio, C., Maggi, F.M., Montali, M., Mendling, J.: Resolving inconsistenciesand redundancies in declarative process models. Inf. Systems 64, 425–446 (2017)

6. Graham, I.: Business rules management and service oriented architecture: a patternlanguage. John wiley & sons (2007)

7. Grant, J., Martinez, M.V. (eds.): Measuring Inconsistency in Information. CollegePublications (2018)

8. Hunter, A., Konieczny, S.: On the measure of conflicts: Shapley inconsistency val-ues. Artificial Intelligence 174(14), 1007–1026 (2010)

9. Hunter, A., Konieczny, S., et al.: Measuring inconsistency through minimal incon-sistent sets. KR 8, 358–366 (2008)

10. McAreavey, K., Liu, W., Miller, P.: Computational approaches to finding and mea-suring inconsistency in arbitrary knowledge bases. International Journal of Approx-imate Reasoning 55(8), 1659–1693 (2014)

11. Papadimitriou, C.: Computational Complexity. Addison-Wesley (1994)12. Potyka, N.: Measuring disagreement among knowledge bases. In: International

Conference on Scalable Uncertainty Management. pp. 212–227. Springer (2018)13. Thimm, M.: On the compliance of rationality postulates for inconsistency mea-

sures: A more or less complete picture. KI 31(1), 31–39 (2017)14. Thimm, M.: Inconsistency measurement. In: Proceedings of the 13th International

Conference on Scalable Uncertainty Management (SUM’19) (2019)15. Valiant, L.: The complexity of computing the permanent. Theoretical Computer

Science 8, 189–201 (1979)

16 C. Corea et al.

Appendix A: Proofs of Technical Results

Proposition 1. mΣCD and mΣ

C# satisfy RS and RM.

Proof. We consider mΣCD and mΣ

C# in turn. For this, let M be a multiset of rulebases, B any rule base in M, and r any rule in a rule base B.

– We start with the measure mΣCD . To show rule symmetry, as mΣ

CD (M, r) =∑B∈M CD(B, r), we have thatmΣ

CD (M, r) = mΣCD ((B1, ...Bn), r), for any per-

mutation of the order of B1 to Bn due to commutativity via∑Bi∈M CD(Bi, r) =

CD(B1, r) + ...+ CD(Bn, r), with n = |{B ∈M}|. For rule minimality, recallthat a rule r is defined as a free formula in M if r /∈M, ∀M ∈

⋃b∈MMI(b).

Consequently, if r ∈ Free(M), then∑B∈M CD(B, r) = 0 per definition.

– The proofs for mΣC# are analogous, i.e., mΣ

C#(M, r) = mΣC#((B1, ...Bn), r),

for any permutation of the order of B1 to Bn due to commutativity, and∑B∈M C#(B, r) = 0 if r ∈ Free(M), as C#(B, r) = |{M ∈ MI(B) | r ∈ M}|

for any B ∈M.

Proposition 2. Any Σ-induced multi-rb culpability measure satisfies RS. Givena culpability measure C satisfying MIN, any multi-rb culpability measure Σ-induced via C satisfies RM.

Proof. LetM be a multiset of rule bases, B any rule base inM, and r any rulein a rule base B. To show rule symmetry, as mΣ

Cm(M, r) =∑B∈M Cm(B, r),

Cm(M, r) = Cm((B1, ...Bn), r) for any permutation of the order of B1 to Bndue to commutativity via

∑Bi∈M C

m(Bi, r) = Cm(B1, r) + ...+ Cm(Bn, r), withn = |{B ∈ M}|. To show rule minimality, given a culpability measure C, ifwe have for a rule r ∈ B: r 6∈ M,∀M ∈ MI(B) and C(B, r) = 0 , then Cm =∑B∈M C(B, r) = 0 per assumption.

Proposition 3. mΣCD and mΣ

C# satisfy CO, MO and IN.

Proof. We consider mΣCD and mΣ

C# in turn. For this, let M be a multiset of rulebases, B any rule base in M, and r any rule in a rule base B.

– We start with the measure mΣCD . To show multiset consistency, observe

that CD(X, r) for a consistent rule base X is 0 for any rule r per defi-nition, as MI(X) = ∅. Thus, if @B ∈ M : B |=⊥, then mΣ

CD (M) = 0.

In turn, V̂ mΣCD (M) = max r∈R(M)(CD(M, r)) = 0. For the other direc-

tion, assume that V̂ mΣCD (M) > 0 for a case where @B ∈ M : B |=⊥.

This would mean, that there must exist a rule r in a consistent rule baseX of M, s.t. CD(X, r) 6= 0. This contradicts CD by definition. To showmultiset monotony, observe that if a rule r′ is added, for any rule baseBi, we have that CD(Bi, r

′) = 0 or 1. Thus, mΣCD (M∪ {r′}) ≥ mΣ

CD (M).

Hence, V̂ mΣCD (M∪ {r′}) = max r∈R(M)(CD(M∪ {r′}, r)) ≥ V̂ m

ΣCD (M). To

Measuring Inconsistency over Sequences of Business Rule Cases 17

show multiset free formula independence, recall that a rule r′ is definedas a free formula in M if r′ /∈ M,∀M ∈

⋃b∈MMI(b). Consequently, if

r′ ∈ Free(M∪{r′}), then |MI(Bi ∪{r′})| = |MI(Bi)| for all B ∈M. In turn,for any other rule r in a rule base Bi ∈M, CD(Bi, r) = CD(Bi ∪ {r′}, r). In

result, if r′ ∈ Free(M∪ {r′}), then V̂ mΣCD (M∪ {r′}) = V̂ m

ΣCD (M).

– The proofs for mΣC# are analogous, i.e., if @B ∈M : B |=⊥ then mΣ

C#(M) = 0

as |MI(Bi)| = 0 for any Bi ∈ M, |MI(B ∪ {r})| ≥ |MI(B)| for any rule r,resp. |MI(B ∪ {r})| = |MI(B)| if r is a free formula in (B ∪ {r}).

Proposition 4. The adjusted multi-rb shapley inconsistency value satisfies CO,MO, IN(, RS and RM).

Proof. LetM be a multiset of rule bases, B any rule base inM, and r any rulein a rule base B. Also, recall that we assume any inconsistency measure I that isused to derived an adjusted multi-rb shapley inconsistency value mΣ

S∗I satisfiesconsistency’, monotony’ and free formula independence’. We then consider theindividual properties in turn. To show multiset consistency, observe that for aconsistent rule base B, we have that I(B) = 0 per assumption of consistency’.It follows that for a consistent rule base B, S ∗Iα (B) = 0 for any α ∈ B, thus

V̂ mΣS∗I (M) = max r∈R(M)(m

ΣS∗I (B, r)) = 0 if @B ∈ M : B |=⊥. For the only if

direction, assume we would have a rule α s.t. S ∗Iα (B) 6= 0 for a consistent rulebase B. This would mean I(B)−I(B\α) > 0 for a consistent rule base B, whichcontradicts the assumption of consistency’. To show multiset monotony, observethat I(B∪{r}) ≥ I(B) for any rule r if I satisfies monotony’. Therefore, S∗Iα(B∪{r}) ≥ S ∗Iα (B) for any α ∈ B. It follows that mΣ

S∗I (M∪{r}, α) ≥ mΣS∗I (M, α)

for any rule α, and thus V̂ mΣS∗I (M∪{r}) ≥ V̂ m

ΣS∗I (M). The proof for multiset

free formula independence is analogous, i.e., I(MI(B ∪ {r})) = I(MI(B)) forany rule r ∈ Free(B ∪ {r}) if I satisfies free formula independence’. Next, rulesymmetry follows from Proposition 2. Last, to show rule minimality, it sufficesto show that S∗I satisfies minimality, which has been shown in [3], i.e., for anyfact f , S∗If = 0 per definition, and for any free rule α, I(B) − I(B \ α) in thelast part of the summand of the coalition payoff will always equate to 0 due tothe consistency’ and free formula independence’ assumption of the underlyingmeasure I, thus, for any ri ∈ R(B), if ri 6∈M,∀M ∈ MI(B), then S∗Iri = 0.

Proposition 5. The adjusted multi-rb shapley inconsistency value satisfies DIS,UB and FM.

Proof. Let M be a multiset of rule bases, B any rule base in M, and r anyrule in a rule base B. We then consider the individual properties in turn. Theproof for distribution follows the proof in [3]: We recall the adjusted Shapleyinconsistency value

S ∗Iα (B) =

{0 if α ∈ F(B)∑B⊆B

CoalitionPayoff Iα,B(B) +∑B⊆B

AdditionalPayoff Iα,B(B) otherwise

18 C. Corea et al.

In the following, we abbreviate CoalitionPayoff as CP and AdditionalPayoff asAP for readability. Then, for a set of rule bases M, we consider the sum of alladjusted Shapley values (for all elements in M over all rule bases B ∈M).∑α∈B

∑B∈M

S ∗Iα (B)

=∑α∈B

∑B∈M

0 if α ∈ F(B)∑B⊆B

CPIα,B(B) +∑B⊆B

APIα(B) otherwise

=∑α∈B

∑B∈M

∑B⊆B

CPIα,B(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

α∈R(B)

∑B∈M

∑B⊆B

APIα,B(B)

Following [8], the first summand can be rewritten.

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

α∈R(B)

∑B∈M

∑B⊆B

APIα,B(B)

Then

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

α∈R(B)

∑B∈M

∑B⊆B

0 if r ∈ Free(B)∑f∈F(B) CPIf,B(B)

|r∈R(B) s.t. r/∈Free(B)| otherwise

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

r∈Free(R(B))

∑B∈M

∑B⊆B

0 +∑

r 6∈Free(R(B))

∑B∈M

∑B⊆B

∑f∈F(B) CP

If,B(B)

|r ∈ R(B) s.t. r /∈ Free(B)|

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

r 6∈Free(R(B))

∑B∈M

∑B⊆B

∑f∈F(B) CP

If,B(B)

|r ∈ R(B) s.t. r /∈ Free(B)|

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑B⊆B

∑B∈M

∑f∈F(B)

CPIf,B(B)

=∑B∈M

I(B)−∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B) +∑

f∈F(B)

∑B∈M

∑B⊆B

CPIf,B(B)

=∑B∈M

I(B)

To show upper bound, observe that due to∑α∈B

∑B∈M S∗Iα(B) =

∑B∈M I(B)

via distribution, we have that V̂ mΣS∗I (M) = max r∈R(M)(mS∗I (M, r)) ≤ mΣ

I (M).Last, to show fact minimality, observe that S ∗If (B) = 0 for any fact f ∈ B per

definition, thus mΣSI (M, α) = 0∀α /∈ R(M).

Proposition 6. Let B be a rule base. The problem of deciding whether B isconsistent can be solved in polynomial time.

Proof. The minimal model M of B can be determined as follows:

1. M = F(B)2. Let r ∈ R(B) be s. t. body(r) ⊆M

Measuring Inconsistency over Sequences of Business Rule Cases 19

3. If there is no such rule, then return M4. Otherwise, M := M ∪ {head(r)} and continue with 2.

I can be seen that M is both closed and minimal and therefore the minimalmodel of B. Both, the algorithm above and checking whether M is inconsistentare polynomial, therefore deciding whether B is consistent is polynomial.

Proposition 7. Let M be a multiset of rule bases with M = (B1, ...,Bn) =({F1 ∪ R}, ..., {Fn ∪ R}) and let r ∈ B. The problem of deciding whether thereis a i ∈ {1, . . . , n} and M ∈ MI(Bi) s. t. r ∈M is NP-complete.

Proof. For NP-membership consider the following non-deterministic algorithm:

1. Guess i ∈ {1, . . . , n}2. Guess a set M ⊆ Bi with r ∈M3. If M is consistent, return False4. For each x ∈M , if M \ {r} is inconsistent return False5. Return True

Observe that the above algorithm runs in polynomial non-deterministic time(due to consistency checks being polynomial, cf. Proposition 6) and returns Trueiff r is contained in a minimal inconsistent subset of at least one of B1, ...,Bn.

In order to show NP-hardness, we reduce the problem 3Sat to the aboveproblem. For that, let I = {c1, . . . , cn} be a set of clauses ci = {li,1, li,2, li,2}where each li,j is a literal of the form a or ¬a (with an atom a). 3Sat then askswhether there is an assignment i : A→ {True,False} that satisfies all clausesof I, where A is the set of all atoms appearing in I. We introduce new atomsk1, . . . , kn for each of the clauses and a new atom s (indicating satisfiability) anddefine BI through

F(BI) = {a,¬a | a ∈ A} ∪ {¬s}R(BI) = {li,j → ki | j = 1, 2, 3, i = 1, . . . , n} ∪ {r = k1, . . . , kn → s}

We now claim that I is satisfiable iff r is in a minimal inconsistent subset ofBI (which is a special case of our problem with M = (BI)). So assume I issatisfiable and let i be a satisfying assignment. Observe that M defined via

M = {a | i(a) = True} ∪ {¬a | i(a) = False} ∪ {¬s} ∪ R(BI)

is inconsistent: as i is a satisfying assignment, each ki (i = 1, . . . , n) can bederived in M ; then s can also be derived, producing a conflict with ¬s. On theother hand, note that M ⊆ {r} is consistent. It follows that there is a minimalinconsistent set M ′ ⊆M with r ∈M ′.

Now assume that there is a minimal inconsistent set M ⊆ BI with r ∈M . First observe that there is no atom a s. t., a,¬a ∈ M (otherwise M \ {r}would still be inconsistent). Let i : A → {True,False} be any assignmentwith i(a) = True if a ∈ M and i(a) = False if ¬a ∈ M . It follows that eachclause c1, . . . , cn is satisfied by i (as each ki could be derived in M) and so i isa satisfying assignment for I.

20 C. Corea et al.

Proposition 8. Let B be a rule base and M ⊆ B. The problem of decidingwhether M ∈ MI(B) can be solved in polynomial time.

Proof. Deciding whether M is inconsistent and M \ {x} for each x ∈ B is con-sistent can each be solved in polynomial time due to Proposition 6. It followsthat deciding M ∈ MI(B) can be solved in polynomial time.

Proposition 9. Let M be a multiset of rule bases with M = (B1, ...,Bn) =({F1 ∪ R}, ..., {Fn ∪ R}) and let r ∈ R. The problem of determining |{M ∈MI(Bi) | i ∈ {1, . . . , n}r ∈M}| is #P-complete.

Proof. Membership follows from Proposition 8 as deciding for a given M whetherM ∈ {M ∈ MI(Bi) | i ∈ {1, . . . , n}, r ∈M} is in P.

The proof of #P-hardness is analogous to the proof of Proposition 5 in [2].Observe that in the reduction the notion of “issue” coincides with notion of aminimal inconsistent subset containing the rule π ← α1, . . . , αn, δ1, . . . , δm if weadd facts a,¬a for each atom a occurring the in input instance.