Association Rules Dm[2]

8/9/2019 Association Rules Dm[2]

1/6

An Efficient Data Mining Technique forDiscovering Interesting Association Rules*

Show-Jane Yen an d Arbee L.P. C h e nDepartment of Computer Science

National Tsing Hua UniversityHsinchu, Taiwan 300, R.O.C.

Email: [email protected]

AbstractMining association rules is an important task. Past

transaction data can be analyzed to discover customerpurchasing behaviors such that the quality of busi-ness decision can be improved. The association rules

describe the associations among items in the largedatabase of customer transactions. However, the sizeof the database can be very large. It is very time con-suming to find all the association rules from a largedatabase] and users may be only interested in the as-sociations among some items. Moreover, the criteriaof the discovered rules for the user requirements maynot be the sam e. Many uninteresting association rulesfor the user requirements can be generated when tra-ditional mining methods ar e applie d. Hence, a datamining language needs to be provided such that userscan query only interesting knowledge to them from alarge data bas e of customer transact ions. In this pa-per, a da ta mining language is presented. From theda ta mining langua ge, users ca.n specify t he interesteditems and the criteria of the rules to be discovered.Also, an efficient data mining technique is proposedto extract the association rules according to the usersrequests.

1 I n t r o d u c t i o nData mining has high applicability in retail indus-

try . Th e effective managemen t of business is signifi-cantly dependent on the quality of its decision makin g.It is therefore important to analyze past transactiondata to discover customer purchasing behaviors andimprove the quality of business decision. Because theamount of these transaction data is very large, an ef-ficient algorithm needs to be devised for discoveringuseful information embedded in the transaction data.

An association rule describes the association among

items in which when some items are purchased in atrans actio n, others are purchased too. Th e follow-ing definitions are adopted from [ l ] . A transactions u p p o r t s an itemset Z , f Z is contained in the trans-action. The s u p p o r t fo r a n z t e m s e t is defined as the

*This work was partia lly supp orte d by the Republic of ChinaNational Science Council under Contract No. N SC 86-2213-E-007-009.

0-8186-8147-0197 $10.00 0 997 IEEE

ratio of the total number of transactions which sup-port this itemset to the total number of transactionsin the database. To make t he discussion easier, occa-sionally, we also let the total number of transactionswhich support the itemset denote the support for theite mse t. T he maj or work of mining association rules isto find all itemsets th at satisfy a certain user-specifiedminimum s u p p o r t . Each such itcmsct is rcferred to aslarge itemset.

In order to find association rules, all large itemsetsneed to be generated from the dat abas e. However,the size of the database can be very large. It is verytime consuming to find all association rules from thelarge database, and users may be only interested inthe associations among cert,ain items . Moreover, thecriteria (such as minimum support) to discover rulesfor the users may not be the same . Many uninter-esting association rules to the users can be generatedwhen traditional methods of mining association rulesare applied [ l , 2 , 4 , 51. Hence, a data mining languageis needed such that users can query knowledge from alarge database of customer transactions.

Meo, Psaila and Ceri [3] proposed a SQL-like op-erato r for extrac ting association rules. Th e SQL-likeoperator is capable of expressing the problem of min-ing association rules. However, the expressive powerof the SQL-like operator is still limitary. For exam-ple, users may want t o query th e associations betweencertain item s and all the other ite ms. Th e SQL-likeoper ato r cannot express this kind of query. Further-more, the SQL-like query language is inconvenient fornaive users, which is suita.ble to SQL programmersand experts, and the SQL-like operator performs set-oriented operations (i.e., join operations), which arevery inefficient operations.

For designing a data mining language, two impor-

ta nt issues need to be considered: the easy-to-use userinterface and the efficient data mining language pro-cessing. T his paper is concerned with t he two issues.We present a d at a mining langu age] from which usersonly need to specify the criteri a for discovering therules, and the it ems in the antecedent and the conse-quent in the rules. We also propose a n efficient dat amining technique to process users request. Accord-ing to the users request, the discovered large item-

664

Authorized licensed use limited to: Institute of Tech & Management. Downloaded on December 24, 2008 at 01:58 from IEEE Xplore. Restrictions apply.

2/6

sets are called the anterestang large atemsets, and thediscovered association rules the anterestang assocaataonrules.

For the efficient data mining technique, the inter-esting large itemsets are discovered firstly. After dis-covering all interesting la rge itemsets , the interes tingassociation rules can be extracted according to theantecedent and the consequent specified in the usersrequest. For an interesting large itemset Z , f itemsetsX and Y match the user-specified antecedent and theconsequent, respectively, and X U Y = 2 , hen therule X Y can be generated. The confidence ofX Y n database D is the probability that whenitemset X occurs in a transa ction in D , itemset Y alsooccurs in the same transact ion. Tha t is, the ratio ofthe support for itemset Z to the support for itemsetX . This rule is an interesting association rule if itsconfidence achieves the manLmum conjidencespecifiedin the users request. An example of such an associ-ation rule is 95% of the transactions in which coffeeand sugar are purchased, milk is purchased too. Theform of this rule is coffee, sugar =+ milk 95%. Theantecedent of this rule consists of coffee and sugar andthe consequent consists of milk alone. The percentage

95% is the confidence of the rule.The rest of the paper is organized as follows: Sec-tion 2 presents t he da ta mining language. Section 3proposes the efficient data mining algorithm for thepresented da ta mining language. The performanceanalysis for the dat a mining algorithm is presented inSection 4 . Finally, we conclude this paper and presentdirections for future research in Section 5.

Mining Association RulesFrom < Database >With

Antecedent (*))rConsequent (*))Support sConfidence c

 ::= item[, temIkWhere 0 6 s , c 5 1 and k > 0. The parameter

< Database > is used to specify the database nameto which users query the association rules.

In the With clause, users can specify items in theantecedent a nd th e consequent of the rules to be dis-covered after the keywords Antecedent and Con-sequent, respectively. If the items are specified in

 after the keyword Antecedent (Conse-quent), hen the antecedent (consequent) of each dis-covered rule will contain these i tems . Besides, usersneed t o specify the two criteria: minimum supportand minimum confidence by the keywords Supportan d Confidence, respectively.

Notice that the phrase within the parentheses isoptional. If the user does not specify the keyword

Antecedent (Consequent), then any item can ap-pear in the antecedent (consequent) of the discoveredrules. If the user does not specify the two keywordsAntecedent and Consequent, all association ruleswhich achieve the user-specified cr iter ia will be dis-covered. The notation *represents all it ems exceptthe items specified in . If the notation*is specified after the keyword Antecedent (Con-sequent), then in addition to the items specified in, other items can also be contained in theantecedent (consequent) of each discovered rule.

3 Efficient Data Mining AlgorithmIn this section, we describe how to process a users

request. We develop an efficient dat a mining (EDM)algorithm to generate th e interesting association rulesaccording to the users request.

For a users request, if both the two keywords An-tecedent and Consequent are specified in t he Withclause and there is no notation *specified, then theantecedent and the consequent of the discovered rulewill contain only the items specified in safter the keywords Antecedent and Consequent,respectively. We call thi s type of users requests the

Ty p e I request. If the user likes to extract associa-tion rules whose antecedent or consequent can containother items except the items specified in ,then the noataion * has to be specified in the Withclause. We call this type of users requests the Type IIrequest. Th e request in which only one of th e two key-words Antecedent and Consequent is specified alsobelongs to the Type I1 request. If both keywords An-

is the large atem generataon phase . In this phase, EDMalgorithm scans the database to record related infor-mation for each anterested atem and find large items.The interested i tems for the Type I request are theitems specified in the With clause. The interesteditems for the Type I1 and T ype I11 requests are allitems in the database.

The second phase is the assocaataon graph construc-tion phase which constructs an association graph to in-dicate the associations between every two large itemsgenerated in the first phase. The third phase is theznterestang large atemset generataon phase which gen-erates all interesting large itemsets by traversing theconstructed association graph according to the usersrequest. The final phase is the anterestang assocaa-ihon rule generataon phase which generates all inter-esting association rules according to the discovered in-

teresting large itemsets, the items specified after thetwo keywords Antecedent and Consequent, andthe user-specified minimum confidence in the usersrequest.

3.1 Large i tem generationIn the first phase, algorithm E DM scans the

database and builds a bit vector for each interested

66 5


3/6

4/6

Figure 1: The association graph for Request 2 andRequest 3

3.3 Interest ing large i te mset generationIn the third phase, the algorithm EDM generates

all interesting large itemsets according to the usersrequest. In this phase, we develop two algorithms:LGTE (Large itemse t Generation by Tree Expansion)and LGDE (large itemset Generation by Direct Ex-tension) to process the Typ e I1 and Typ e I11 requests,respectively.

3.3.1 interesting large itemset generation forType I1 requests

For Type I1 requests, in addi tion to t he ite ms specifiedin < Items > I s , other items can also be contained inthe antecedent and th e consequent of each rule to bediscovered. Th e Algorithm LGTE is applied to gen-erate interesting large itemsets for Type I1 requests.LGTE constructs an expunszon tree based on the as-sociation graph. Each node in the expansion tree con-tains an large itemset. During the expansion tre e con-struction, LGTE expands each node which containsa large k-itemset (k >_ 1) to generate its child nodeswhich contain large k + 1-itemsets.

First, LGT E checks if the set of the it ems specifiedin I s is a large itemset. If this itemset is a

large itemset, then the itemset is designated the rootnode of the expansion tree, an d the it em numbers ofthe items in the root node is in the increasing order.

Subsequently, LGTE marks *between every twoitems, before the first item and after the last itemin the root node. For example, consider Request 2 ,which is a Typ e I1 request. Th e itemset specified inRequest 2 is (3 , 5) , which is a large itemset sinceB G o B & is 5 (2 3) . After mark ing * in the itemset,the root node of the expansion tree becomes *3*5*.

After creating the root node of the expansion tree,the itemset in t he root node will be extended t o gener-ate extended ite msets. We have the following Lemmasto check if an itemset in a node can be extended.Lemma 2: If an itemset is not a large itemset, thenany itemset which contains the items in this itemsetcannot be a large itemset.Rationale: Because t he itemse t is not, a large itemset,the support for the itemset is less than the minimumsuppor t. Hence, the support for an itemset which con-tains the items in this itemset must be also less thanthe minimum support.Lemma 3: For a large itemset ( i l , 2 , ..., o ) , f thereis no directed edge from any item i h ( i k - 1 1) to i tem i k (1 5 k 5 n ) , then item-set ( i l l .., k - 1 , i h , k , ... , n ) (or ( i h , i l , 2 , . , n) whenk = 1) cannot be a large itemset.Rationale: Because there is no directed edge fromany item i h ( i k - 1 1) to i temi k , the itemset ( i h , i k ) (or ( i h , l ) when k = 1) isnot a large 2-itemset.. Hence, by Lemma 2, item-set ( i l l ..., k - 1 , I , + , l k , ..., n) (or ( i h , i l l 2 , ,.., i n ) whenk = 1) is not a large itemset.Lemma 4: For a large itemset ( i l l a , ... in), if thereis no directed edge from item ik (1 5 k 5 n) toany item ip ( ip < i k + l when k < n ) , then item-set ( i l l .., zk, p , 2 k + 1 , ... i n ) or ( i l l 2 , ... i o , p ) whenk = n ) cannot be a large itemset.Rationale: Because there is no directed edge fromitem i k to any item i, ( i p < i k + l when k < n) ,itemset ( i k , i p ) s not a large 2-itemset. Hence, item-set ( i l , .., k , ip , k + 1 , ..., n) (or ( i l l 2 , ..., n, i p ) whenk = n ) is not a large itemset according to Lemma 2.

If there is an n-itemset in a node with a * I marked,then this n-itemset can be extended into n +1-itemsetfrom the position of the mark *except one of thecases stated in Lemma 3 and Lemma 4 holds. Hence,for each *marked in a node, LGTE checks if theitemset I = ( i l , i 2 , .., n ) n the node can be extended.Suppose node N which contains itemset I is a rootnode, and is marked as *il*i2*...*in*. In the following,we describe how to construct an expansion tree.

If the position of the mark *is before the firstitem i l of the itemset I in node N I hen LGTE checksif there are directed edges from some items to th e itemi l in the association graph. If there is no directededge from any item i h to the item i l l then th e itemsetcannot be extended for this mark *, because for anyitem i h , itemset ( i h , i l l 2 , ... i n ) s not a large itemsetaccording to Lemma3. Th e mark *is then removedfrom node N . Hence, node N becomes i l * i 2 * ...* n*.If there is a directed edge from an item i, to the itemi l , then the itemset I is extended into the itemset(i,, i l , 2 , ... in ) , because this extended itemset can bea large itemset . If the extended itemset is a largeitemset, the node *iei l* i2*. . .* in* is created, and thisnode becomes a child node of node N .

If the position of the mark * is between itemsi k and ik+1 of the itemset I in node N I hen LGTEchecks if the re a re directed edges from th e item i kto the other items whose item nui k + l . If there is no directed edge from the item ik tothe other item i, ( ip < i k + l ) , then the itemset can-not be extended for this mark *,because for anyitem i,, itemset ( i l , ..., k , i,, i k + 1 , ..., n ) s not a largeitemset according to Lemma 4. The mark * be-tween items i k and i k + l is then removed from nodeN and its child nodes created so far, and node N be-comes * i l* . . . * i k ik+ l* . . . * i n* . However, if there is a di-rected edge from item ik to an item i , ( i , < i k + l )and item i, to item i k + l , then the itemset I is ex-tended into the itemset ( i l l .., k , i, , i k + l , ..., n), be-cause this extended itemset ca n be a large itemset . I fthe extended itemset is a large itemset, then the node* i l* . . . * i~ i , * ik+ l* . . . * in* is created, and this node alsobecomes a child node of node N .

. . .

667


5/6

If the position of the mark *is after the last itemi n of the itemset 1 in node N , then LGTE checksif there are directed edges from item in to the otheritems in the association graph. If there is no directededge from ite m in to the other item i,, then the itemsetcannot be extended for this mark *, because for anyitem i,, itemset ( i l , .., in , ,) is not a large itemsetaccording t o Lemm a4. The mark *is then removedfrom node N and its child nodes created so far, andnode N becomes *il*i2*...*in. However, if there isa directed edge from item in to an item i t , then theitemset I is extended into the itemset ( i l , .., in , t) ,because this extended itemset can be a large itemset.If the extended itemset is a large itemset, the node*il*i2*...*init* is created, and this node becomes achild node of node N .

For each created node, if there exist * s in thenode, LG TE expands all children of this node for eachmark *,and removes the mar k *from the node af-ter t he expansion. For a node, if all extended itemsetsfor a mark *are not large itemsets, then the mark*is removed from the node and its child node cre-ated so far. After constructing the expansion tree, theitemset in each node is an interesting large itemset.Finally, LGTE generates all interesting large itemsetsfrom each node of the expansion tree. Th e algorithmLGTE is described as follows:

For example, consider Request 2. First, LGTEcreates the root node *3*5*. For the first mark *in the node, because there exist directed edges fromitems 1 and 2 to item 3 in the association graph shownin Figure 1, the extended itemsets are (1 , 3 , 5) and (2,3 , 5). Because the itemsets (1 , 3 , 5) and ( 2 , 3 , 5) bothare large itemsets, *13*5* and *23*5* are created aschild nodes of the original node *3*5*. After expand-ing all child nodes for this mark *,the original node*3*5* becomes 3*5*.

For the mark *between item 3 and item 5 in theroot node, there is only one directed edge from item3 to item 4 whose item number is less than item 5.However, there is no directed edge from item 4 to item5. Hence, the itemset (3, 5 in the root node cannot

item 3 and item 5 is removed from the root node andits child nodes created so far . Hence, the root nodebecomes 35* and the two child nodes become 135and *235*, respectively.

For the mark *after the item 5 in th root node,there is only one directed edge from item 5 to item 7 inFigure 1 , and the extended itemset ( 3 , 5, 7) is a largeitemset. Hence, the node 357* is created as a child

node of the root node. After the expansion, this mark*is removed from the root node and the root nodebecomes 35. Similarly, LGTE continues to expand allcreated child nodes. Th e expansion tree for Request2 is shown in Figure 2 .

Finally, LGTE generates all interesting large item-sets from each node of the expansion tree in Figure 2.Because there are eight nodes in the expansion tree,there are eight interesting large itemsets generated.

be extended for this mark 2 . The mark *between

I 3 5 7 1235 z357

12357

Figure 2: The expansion tree for Request 2

3.3.2 interesting large itemset generation forType I11 requests

For Type I11 requests, there are no keywords An-tecedent and Consequent specified in the users re-quests. Hence, all association rules achieve the user-specified cri ter ia will be discovered. For example, if welike t o extra ct all association rules whose support andconfidence achieve 20 % and SO%, respectively, thenthe request is written as Request 3.

Request 3:Mining Association RulesFrom TDBWi t h

Suppor t 20%Confidence 80%

The LGDE algorithm is proposed to generate allinteres ting large itemsets for Type I11 requests. Sup-pose the set of large k-itemsets is L k ( k > 1) . Th e2-itemsets Lz is found in the association graph con-struc tion phase. In the interesting large itemset gen-eration phase, the LGDE algorithm generates largek-itemsets L I , ( k > 2 ) . For each large k-itemset inL k ( k 2 2), the last item of the k-itemset is used toextend the itemset into k + 1-itemsets. Suppose ( i l i ~..., k ) is a large k-itemset. If there is a directed edgefrom item ik to item U in the association graph, thenthe itemset ( i l , z , ..., k ) is extended into k+l- item set( i l , i 2 , ..., k , ~ ) , ecause this extended itemset can bea large itemset. The extended itemset ( i l , i 2 , ..., k , ~ )is a large k + 1-itemset if B E : 0 L?x2 ... 0 SE, 0 SV,is no less than the user-specified min imum suppo rt.If no large k-itemsets can be generated, the LGDEalgorithm terminates.

For example, consider Request 3 . In the associa-tion graph construction phase, 15 large 2-itemsets aregenerated and the association graph is shown in Fig-ure 1. For large 2-itemset (1 , a ) , there are five directededges from item 2 of the itemset (1 , 2) to items 3, 4,5 , 6 and 7 , respectively. Hence, th e 2-itemse t (1 , 2)can be extended into 3-itemsets (1 , 2, 3) , (1 , 2, 4) , (1 ,2, 5), (1 , 2 , 6) and (1, 2 , 7). Because SV , 0 BVz BVqand BV1 0 BV2 0 BVc are 1 and 0 , which are less thanthe user-specified mini mum support (i.e. , 3 ) , th e 3-itemsets (1, 2, 4) and (1, 2, 6) are not large itemsets.The other 3-itemsets (1, 2, 3), (1, 2, 5) and (1 , 2, 7)are large 3-itemsets. The LGDE algorithm terminateswhen no large 6-itemsets can be further generated.

668



6/6

3.4 Association rule generationAfter discovering all interesting large itemsets,

EDM generates interesting association rules accordingto the interesting large itemsets, the items specifiedafter the keywords Antecedent and Consequent,and the minimum confidence specified by the keywordConfidence in the user's request. If the keywordsAntecedent and Consequent are not specified in

the user's request, i.e., it is a Type I11 request, thenfor any interesting large itemset A P , all rules thatreference items in the interesting large itemset can begenerated. The antecedent of each of these rules is aproper subset SAP of A P , and the consequent is A P -S A P. For each SAP AP - S A P, EDM checksif the confidence achieves the user-specified minimumconfidence. If the confidence achieves the mini mumconfidence, then the rule SAP --I A P - SAP is aninteresting association rule.

If both keywords Antecedent and Consequentare specified in the user's request a nd the re are nonotation "*"'s specified in the user's request, i.e., itis a Type I request, then EDM checks if the rulewhose antecedent and consequent contain only theitems specified after the keywords Antecedent andConsequent respectively, is an interesting associa-tion rule. If there is only one of the keywords An-tecedent and Consequent specified or there is a no-tation "*'"s specified in the user's request, i.e., it is aType I1 request, then EDM generates rules whose an-tecedents (consequents) need t o match t he i tems spec-ified after the keywords Antecedent (Consequent).

For exam ple, consider Request 2 . For the interest-ing large itemset {A, C , E}, two combinations of theantecedent and consequent match the items specifiedin Request 2: AC E and C AE. The confi-

less than the minimum confidence 80%. Hence, C ==+AE is not an interesting association rule. However,

the confidence for AC j E is s ~ ~ ~ ~ : ~ ~ f " : ~ f ~ ~ $ l1which is greater than 80%. Hence, AC E is gen-era ted, which is an interesting association rule.

dence for C 3 AE is s u p p o r t f o r { A , C ~ E ) = S which iss u p p o r t f o r { C } 10

4 Performance AnalysisIn this section, we analyze the performance for

the efficient data mining algorithm EDM. The effi-cient data mining technique is implemented in SunSPARC/ 10 workst at on,

In the large item generation phase, EDM scans thedata base t o find large items from the interested it emsand build the bit vector for each generated large item.The cost for the first phase is one database scan. Forthe Type I request, suppose there are k items speci-fied in the With clause. After the first phase, EDMperforms ( k - 1) inner products on the bit vectors.For the Type I1 and Type I11 requests, EDM gen-erates interesting large itemsets through the followingtwo phases. For the graph construction phase, supposethere are 1 large items generated in the first phase.ED M performs inner products on bit vectorsto construct association graph

For the interesting large itemset generation phase,EDM develops two algorithms LGTE and LGDE toprocess the Type I1 request and th e Type I11 request,respectively. In the kth ( k > 2) iteration, LGDEextends each large k - -itemset into k-itemsets ac-cording to the association graph. Suppose the averageout-degree of each node is q in the association graph.LGDE performs ( k - 1) x I L k - 1 1 x q inner productst o find all large k-itemsets, which has been demon-str ate d to have a better performance [5] than the otherapproaches.

For LGTE algorithm, suppose there are n nodes inthe constructed expansion tree, and on the average,there are m extended itemsets on each node and thelength of each extended itemset is k . LGTE algorithmperforms n x m x ( k - 1) inner products to constructexpansion tree. Hence, EDM is an efficient algorithmfor generating association rules according to the user'srequest.

5 Conclusion and Future WorkWe introduce a da ta mining language. From the

da ta mi ning language, users can specify the i tems inthe antecedent and the consequent, and the two cri-

teria: minimum support and minimum confidence ofthe association rules to be discovered.We propose an efficient data mining algorithm

(EDM ) t o process a user's request. The algorit hmEDM needs only one database scan and some innerproducts to generate all interesting association rulesaccording to the user's request, which is very efficient.

In the future, we shall extend the data mining lan-guage to allow more flexible query specifications, anddevelop a n interactive da ta mining technique t o dis-cover other kinds of association rules according to theuser's request, such as generalized association rulesand multiple-level association rules.

ReferencesR . Agrawal and et al. Mining Association Rules

Between Sets o Items in Large Databas es. InProceedzngs of the ACM SIGMOD, pages 207-216, 1993.

R. Agrawal and R. Srikant. Fast Algorithmfor Mining Association Rules. In Proceedzngs ofthe Internataonal C onference on Very Large DataBases, pages 487-499, 1994.

R. Meo, G . Psaila, and S. Ceri. A New SQL-likeOpe ra tor for Mining Association Rules. In Pro-ceedzngs of the Internatzonal Conference on VeryLarge Data Bases, pages 122-133, 1996.

J.S. Park, M.S. Chen , and P.S. Yu. An Ef-fective Hash-Based Algorithm fo r Mining Asso-ciation Rules. Proceedangs of A C M SIGMOD,24(2):175-186, 1995.

S.J. Yen and A.L.P. Chen. An Efficient Approachto Discovery Knowledge from Large Databases.In Proceedzngs of the Internatzonal Conferenceon P a r a l l e l a n d Dzstrzbuted Inforniatzon Systems,pages 8-18, 1996.

669

Documents

Association Rules Dm[2]