MELJUN CORTES Automata Lecture Non-context-free Languages 1

8/21/2019 MELJUN CORTES Automata Lecture Non-context-free Languages 1

1/18

Theory of Computation (With Automata Theory)

Non-Context-Free Languages *Property o f STIPage 1 of 18

TOPIC TITLE: Non-Context-Free Languages

Specific Objectives:

At the end of the topic session, the students are expected to:

Cognit ive:

1. Discuss the details of the pumping lemma for context-freelanguages.

2. Use the pumping lemma to prove if a language is non-context-free.

Affective:

1. Listen to others with respect.2. Participate in class discussions actively.

MATERIALS/EQUIPMENT:

o topic slides

o OHP

TOPIC PREPARATION:

o Have the students review related topics that discussed inprevious courses.

o Prepare the slides to be presented in class.o It is imperative for the instructor to incorporate various kinds of

teaching strategies while discussing the suggested topics. o Prepare additional examples on the topic to be presented.


2/18



Non-Context-Free LanguagesPage 1 of 34

Non-Context-Free Languages


Introduction

Recall that a regular language has either a nondeterministic automaton thatrecognizes it or a regular expression that can generate its strings. In other words,to prove that a language is regular, it must be shown that this language has eitheran NFA or a regular expression that describes it.

Similarly, a context-free language has either a pushdown automaton thatrecognizes it or a context-free grammar that can generate its strings. To provethen that a language is context-free, it must be shown that this language has eithera PDA or context-free grammar that describes it.

Additional Information: An NFA can be viewed as a PDA that simply ignores ordoes not use its stack. Hence, technically speaking, every NFA is a PDA. Every

regular language then has a PDA that recognizes it. It can therefore be said thatall regular languages are context-free languages. However, not all context-freelanguages are regular. For example, the language L = {0

n1

n n ≥ 0} is context-free

but is not regular.


In the discussions on regular languages, it was established that there arelanguages that are not regular.

The question now is:

Are there non-context-free languages?

If the answer is yes, the next question would be:

How can a language be proven to be non-context-free?

Consider the language L= {anb

nc

n n ≥ 1}. This language is composed of strings

that start off with a block of consecutive a’s, followed by a block of consecutive b’s,and then followed by a block of consecutive c ’s. The number of a’s, b’s, and c ’smust be equal. It is quite simple for a PDA to keep track if the number of a’s isequal to the number of b’s, or if the number of a’s is equal to the number of c ’s, or ifthe number of b’s is equal to the number of c ’s (just look at the PDA for the

language {0n1

n n ≥ 0} in the lecture on pushdown automata). However, to keep

track if the number of a’s, b’s, and c ’s are all equal, it would require a PDA with twostacks. Since there is no such PDA, then the language is not context-free.

To prove that a language is non-context-free, it must then be shown that there is noPDA or CFG for that language. This is very difficult to do. It is difficult to ascertainif the language does not really have a PDA or CFG, or if it is due to the ability (orinability) of the person doing it.


3/18




Recall that to prove that a language is not regular, the technique is not to show thatthe language has no NFA or regular expression that describes it. Instead, thepumping lemma was used. The pumping lemma for regular languages exploits thefact that the strings w of regular languages have this special property that if theyare long enough (longer that the pumping length), then they can be divided into

three parts:

w = xyz

If certain conditions are satisfied, the middle substring y can then be repeated orpumped an arbitrary number of times and the resulting string will still be in thelanguage. So, if a certain language has at least one string that is long enough thatdoes not have this property (cannot be pumped), then the language is not a regularlanguage.


To prove then that a language is not context-free, it is therefore important toestablish a pumping lemma for context-free languages that is similar to thepumping lemma for regular languages.

As in regular languages, this pumping lemma should also exploit a certain propertyof context-free languages. Languages whose strings do not have this property areconsidered non-context-free.

The next step now is to find this special property of context-free languages.


Page 5 of 34

Context-Free Grammars Revisited

Assume the following context-free grammar G for a certain language L:

S → A1

A → 0A1 ε

The leftmost derivation for the string 0001111:

S → A1 (using the rule S → A1)→ 0A11 (using the rule A → 0A1)→ 00A111 (using the rule A → 0A1)→ 000A1111 (using the rule A → 0A1)

→ 0001111 (using the rule A → )

Take note that the language of grammar G here is L(G) = {0x1

x+1 x ≥ 0}. The

language L is therefore composed of strings that start with a block of consecutive0s followed immediately by a block of consecutive 1s where the total number of 1sis one more than the total number of 0s.


4/18




The parse tree for the leftmost derivation of the string 0001111 is

Recall that in a parse tree, the leaf nodes (nodes with no children) represent theterminals or the symbols of the string being derived. In this example, the leaf

nodes are (starting from left to right) 000 1111. The derived string is then0001111.


Because the string being derived is long, some variables were used more thanonce. In the given parse tree, the variable A appeared 4 times (the first time A appeared was when the rule S → A1 was used and the next 3 times it appearedwere when the rule A → 0 A1 was used 3 times). If the rule A → 0 A1 is used one

more time before applying the rule A → as the last step, the derivation will end upwith the string 000011111 which is still a member of language L. And, if the rule is

used several more times, the string derived will still belong to language L.

There is now a similarity between regular languages and context-free languages.Long strings in regular languages have a substring that can be repeated orpumped and the resulting strings will still be part of the regular language. Incontext-free languages, the derivation of long strings will have at least one variablethat will be used several times, and the strings derived will still belong to thelanguage.

In the pumping lemma for regular languages, strings whose length is greater thanor equal to the pumping length tend to have a substring that can be repeated anarbitrary number of times, and the resulting strings will still be in language. Recallthat the pumping length of a regular language is equal to the number of states ofthe DFA that recognizes it.

The next step now is to determine how long a string should be before a variable isrepeated in the derivation process. In other words, for the pumping lemma forcontext-free languages, what is the pumping length?


5/18




Deriving the Pumping Length

Assume that for a certain parse tree, the non-leaf nodes (nodes with children) havea maximum number of children equal to c . This implies that the length of the stringat the right-hand side of any rule of the grammar will not exceed c . For example, in

the grammar G:

S → A1

A → 0A1 ε

The rule with the longest string at its right-hand side is the rule A → 0 A1. Since thelength of its right-hand side string is 3, using this rule will produce a parse treewhere the maximum number of children of any non-leaf node is 3. This is seenbelow:

A 10

A

Using the other rules of grammar G will result in nodes with less than 3 children.

In the succeeding discussions, it will be assumed that c ≥ 2.

Assume further that the height of the parse tree for the derivation of a certain stringw is h. Remember that the height of a tree is the length of the longest path fromthe root node to a leaf node. For the given parse tree, the longest path from theroot node to a leaf node is the one that involves the nodesS → A → A → A → A → . The length of this path is 5, so the height of the parsetree h = 5. Take note that this path involves 6 nodes. If the length of a path is x ,

then the number of nodes within that path is x + 1.

A tree with height h can have a maximum number of leaf nodes equal to c h. This

maximum is reached if each non-leaf node of the tree has exactly c children each.For example, if the height of a tree is 2 (h = 2) and the maximum number ofchildren of a non-leaf node is 3 (c = 3), then the maximum number of leaf nodeswill be c

h = 3

2 = 9. This is shown below:

h = 2

No. of leaf nodes = 9


6/18




As mentioned earlier, the leaf nodes of the parse tree represent the terminals orsymbols of the string being derived. If the maximum number of leaf nodes is c

h,

then the length of the string w being derived will not exceed c h. Mathematically,

w ≤ c h

Let v be the number of variables in a grammar G.

If the length of a certain string w being derived is w ≥ c v , then the height of the

parse tree representing this derivation will be

h ≥ v

Assume now that there is a string w whose length is

w ≥ c v +1

It follows that the height of the parse tree will be

h ≥ v + 1


If the parse tree has a height h ≥ v + 1, then there is a path from the root node to aleaf node that is greater than or equal to v + 1 (by definition of the height of a tree).This implies that this path will have at least v + 2 nodes. Since the last node in thispath is a leaf node (which represents a terminal), then the remaining nodes ( v + 1)are variables.

If the derivation used more than v + 1 variables, then at least one variable wasrepeated (by the pigeonhole principle) since there are only v variables.

This answers the previous question on how long a string should be before a

variable is repeated in the derivation process. If the length of the string w beingderived w ≥ c

v +1, at least one variable will be used more than once in the

derivation process.


7/18




This is illustrated in the parse tree below:

S

A. . . . . .

u z

A. . . . . .

v y

x

In the derivation of a certain string w , the derivation begins with the start variable S. After using the different rules of the grammar and performing several substitutions,the string that was derived is a string uAz , where u and z are substrings composedof one or more terminals and A is one of the variables of the grammar. This isrepresented as

S uAz

which means that the string uAz is derived from variable S after making severalsubstitutions. Do not confuse this with a rule of the form S → A1 which means thatS is substituted by A1 in one step. The symbol means several substitutions orseveral steps.

Then, starting from variable A, the derivation continues by again making severalsubstitutions until the string uvAyz is reached, where v and y are also substringscomposed of terminals. This is represented by

A vAy

which means that the string vAy is reached from variable A after making severalsubstitutions.

The entire derivation so far is represented by

S uAz uvAyz

Then, starting from variable A again, the derivation continues once more by makingseveral substitutions until the string uvxyz is reached, where x is also a substringcomposed of terminals. This is represented by

A x

which means that the string x is reached from variable A after making severalsubstitutions.

The entire derivation is then represented by

S uAz uvAyz uvxyz


8/18



By the way, it is assumed that this parse tree represents the shortest derivation forthe string S. In other words, there are no useless derivations.


If the derivation A vAy is repeated before using the derivation A x , then theparse tree would look like

S

A. . . . . .

u z

A. . . . . .

v y

A. . . . . .

v y

x

This derivation is represented by

S uAz uvAyz uvvAyyz uvvxyyz

So, if the derivation A vAy is used two times, then the string derived will beuvvxyyz or uv

2 xy

2 z . Similarly, if the derivation A vAy is used three times, then

the string derived will be uvvvxyyyz or uv 3 xy

3z .


To generalize this, if the derivation A vAy is used i times, then the string that willbe derived will look like uv

i xy

i z . And, this string will still belong to the language

since it was derived using the grammar of the language.

Notice the similarity with regular languages. In the pumping lemma for regularlanguages, the input string w which is longer than the pumping length can bewritten as

w = xy i z

where the middle substring y can be pumped several times and the resulting stringwill still belong to the language concerned.


9/18




In the pumping lemma for context-free languages, the input string w that is longerthan the pumping length can then be written as

w = uv i xy

i z

where the 2nd and 4th substrings v and y can be pumped several times and theresulting string will still belong to the language concerned.

It is important to emphasize that the substrings v and y must be pumped the samenumber of times.


In the pumping lemma for regular languages, the pumping length p refers to thenumber of states of the DFA that recognizes the language.

Hence, the length of the string w should be w ≥ p in order for the pumping lemmato be used.

In the pumping lemma for context-free languages, the pumping length p is equal toc

v +1 where c is the length of the longest right-hand side string of any rule in the

grammar and v is the number of variables of the grammar.

Similarly, the length of the string w being analyzed must be w ≥ c v +1

.

Proof of Induction

Page 16 of 34

The Pumping Lemma for Context-Free Languages

The following is the formal statement of the Pumping Lemma for Context-FreeLanguages:

If L is a context-free language, then there is a number p (the pumping length )where, if w is any string in L whose length is greater than or equal to p, then w maybe divided into five parts, w = uvxyz , satisfying the following conditions:

1. for each i ≥ 0, uv i xy

iz L

2. vy ≥ 1, and

3. vxy p.

The pumping length p is equal to c v +1

where c is the length of the longest right-hand side string of any rule in the grammar and v is the number of variables of the

grammar. The length of string w should be greater than or equal to p ( w ≥ p) inorder for the string to be long enough for at least one variable to be repeated in the

derivation of the string.


10/18




The pumping lemma for context-free languages also has three major conditions:

1. Condition 1 states that the string w = uv i xy

iz L for each i ≥ 0. This

condition simply states that strings belonging to language L that are

longer than the pumping length can be divided into 5 parts u, v , x , y , andz .

The 2nd

and 4th parts or substrings (v and y ) can be pumped or repeated

several times and the resulting string will still be a member of language L.


2. Condition 2 states that vy ≥ 1. This refers to the fact that thesubstrings v and y cannot both be empty strings. However, take note thateither v or y can be the empty string.

As discussed earlier, the shortest derivation for the string S is given by:

S uAz uvAyz uvxyz

If v = y = , then the derivation A vAy becomes A A. This is a trivial(useless) derivation and therefore can be removed. The derivation for S is now

S uAz uxz

This now violates the assumption that the derivation S uAz uvAyz uvxyz is the shortest derivation for string S. Therefore, the substringsv and y cannot both be empty strings.


11/18




3. Condition 3 states that vxy ≤ p. Consider again the following parsetree:

S

A. . . . . .

u z

A. . . . . .

v y

x

h = v + 1

The length of the string w = uvxyz is assumed to be greater than or equalto the pumping length p. Since the pumping length p = c

v +1, then

w ≥ c v +1

As stated earlier, if w ≥ c v +1

, then the height of the parse tree for thederivation of this string will be

h ≥ v + 1

This is shown in the figure above.


Consider now the following subtree which is rooted at the noderepresenting the upper variable A of the given parse tree:

A

A. . . . . .

v y

x

h = v + 1

root of

the

subtree


12/18




Take note that no other variable is repeated below the root node of thissubtree. If this is the case, then the height of this subtree would notexceed v + 1.

If the height exceeds v + 1, then some other variable will be repeated.

Therefore,

h ≤ v + 1

From earlier discussions, if the height of the tree is h ≤ v + 1, then thelength of the string it derives will be less than or equal to c

v +1 (the

pumping length). Since the string derived by the given subtree is vxy ,then

vxy c v +1

vxy p

In other words, condition 3 simply states that the substring vxy is not toolong.


As in the pumping lemma for regular languages, proving a language is non-context-free will use proof by contradiction:

1. First, assume that the language L is context-free.

2. Since L is context-free, the pumping lemma states that all strings of L thatare long enough can be pumped.

3. Find a string w that is long enough but cannot be pumped.

4. If such a string exists, then the pumping lemma was contradicted.Hence, language L is non-context-free.


Page 23 of 34Example 1: Prove that the language L1 = {a

nb

nc

n n ≥ 1} is not context-free.

Assume first that L1 is context-free. This language is composed of strings thatstart with a block of consecutive a’s, followed by a block of consecutive b’s,and finally followed by a block of consecutive c ’s. The number of a’s, b’s, and

c ’s must be equal. These include strings such as abc , aabbcc , aaabbbccc ,and aaaabbbbcccc .

Let p be the pumping length of L1. Now choose a string w (that is a memberof L1 and whose length is greater than or equal to p) that will be pumped.

Let the string to be pumped be w = a p

b p

c p. Obviously, the length of w is

greater than p since it has p a’s, p b’s, and p c ’s—its length is actually 3 p.Hence, w is long enough to be pumped.

Now consider various ways in which w can be divided into the u, v , x , y , and z substrings.


13/18




Option 1: Let vxy be all a’s.

If vxy is composed of all a’s, then the division of the input string can be viewed as

aaa...aaa...aaa...aaabbb...bbb...bbb...bbbccc...ccc...ccc...ccc

v x y

p a ’s p b ’s p c ’s

Take note that vxy p since there are p a’s. Hence, this satisfies condition 2.

Pumping v and y would create a string where there are more a’s than b’s and c ’s.Hence, the resulting string will not be in L1. This violates condition 1 of thepumping lemma.

Option 2: Let vxy be some a’s followed by some b’s.

If vxy is composed of some a’s followed by some b’s, then the division of the inputstring can be viewed as


v x y


Pumping v and y would create a string where there are more a’s and b’s than c ’s.Hence, the resulting string will not be in L1. This violates condition 1 of the

pumping lemma.

It is possible for either substring v or y to be composed of some a’s and b’s.Pumping v or y would result in a string where the ordering of the symbols will notbe preserved (some a’s appearing after the b’s). Either way, the resulting stringwill not be in L1 thereby violating condition 1 of the pumping lemma.


Option 3: Let vxy be all b’s.

If vxy is composed of all b’s, then the division of the input string can be viewed as


v x y


Pumping v and y would create a string where there are more b’s than a’s and c ’s.Hence, the resulting string will not be in L1. This violates condition 1 of thepumping lemma.

Option 4: Let vxy be some b’s followed by some c ’s.

If vxy is composed of some b’s followed by some c ’s, then the division of the input


14/18



string can be viewed as


v x y


Pumping v and y would create a string where there are more b’s and c ’s than a’s.Hence, the resulting string will not be in L1. This violates condition 1 of thepumping lemma.

It is possible for either substring v or y to be composed of some b’s and c ’s.Pumping v or y would result in a string where the ordering of the symbols will notbe preserved (some b’s appearing after the c ’s). Either way, the resulting string willnot be in L1 thereby violating condition 1 of the pumping lemma.


Option 5: Let vxy be all c ’s.

If vxy is composed of all c ’s, then the division of the input string can be viewed as


v x y


Pumping v and y would create a string where there are more c ’s than a’s and b’s.Hence, the resulting string will not be in L1. This violates condition 1 of thepumping lemma.

The substring vxy cannot be composed of some a’s followed by b’s and thenfollowed by some c ’s as shown below:


v x y


Since there are already p b’s, the length of vxy will be greater than p, whichviolates condition 2 of the pumping lemma.

Since L1 has a string that cannot be pumped, then this contradicts the originalassumption that L1 is context-free. Therefore, L1 is non-context-free.


15/18




Example 2: Prove that the language L2 = {ww w {0, 1}*} is not context-free.

Assume first that L2 is context-free. Take note that L2 is a language that iscomposed of strings that are concatenations of the same string. In otherwords, the first half of the string is equal to its second half.


Let the string to be pumped be 0 p1 p0 p

1 p. Obviously, the length of w is greater

than p since it has 2 p 0s and 2 p 1s—its length is actually 4 p. Hence, w islong enough to be pumped.



Option 1: Let vxy be all 0s from the first or second 0 p of the string.

If vxy is composed of all 0s from either the first or second batch of 0s, then thedivision of the input string can be viewed as

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y

or

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y

Pumping v and y would create a string where the first half is not equal to thesecond half. Hence, the resulting string will not be in L2. This violates condition 1of the pumping lemma.

Option 2: Let vxy be all 1s from the first or second 1 p of the string.

If vxy is composed of all 1s from either the first or second batch of 1s, then thedivision of the input string can be viewed as

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y

or


16/18



000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y

Pumping v and y would create a string where the first half is not equal to thesecond half. Hence, the resulting string will not be in L2.


Option 3: Let vxy be some 0s followed by some 1s from the first or second half ofthe string.

If vxy is composed of some 0s followed by some 1s from the first or second half ofthe string, then the division of the input string can be viewed as

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y

or

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y



Option 4: Let vxy be some 1s followed by some 0s somewhere in the middleportion of the string.

If vxy is composed of some 1s followed by some 0s somewhere in the middleportion of the string, then the division of the input string can be viewed as

000...000...000...000111...111...111...111000...000...000...000111...111...111...111

v

p 1s p 0s p 1sp 0s

x y




17/18




Example 3: Prove that the language L3 = {ab c k ≥ j ≥ i ≥ 0} is not context-free.

Assume first that L3 is context-free. This language is composed of strings thatstart with a block of consecutive a’s, followed by a block of consecutive b’s,and then followed by a block of consecutive c ’s. The number of c ’s is greater

than or equal to the number of b’s, which in turn is greater than or equal to thenumber of a’s.


Let the string to be pumped be w = a p

b p

c p. Obviously, the length of w is

greater than p since it has p a’s, p b’s, and p c ’s—its length is actually 3 p.Hence, w is long enough to be pumped.



Option 1: Let vxy be all a’s.

If vxy is composed of all a’s, then the division of the input string can be viewed as


v x y


Pumping v and y would create a string where there are more a’s than b’s and c ’s.

Hence, the resulting string will not be in L3. This violates condition 1 of thepumping lemma.

Option 2: Let vxy be some a’s followed by some b’s.

If vxy is composed of some a’s followed by some b’s, then the division of the inputstring can be viewed as


v x y


Pumping v and y would create a string where there are more a’s and b’s than c ’s.Hence, the resulting string will not be in L3. This violates condition 1 of the pumpinglemma.


18/18


Non-Context-Free Languages *Property o f STI18 f 18


Option 3: Let vxy be all b’s.

If vxy is composed of all b’s, then the division of the input string can be viewed as


v x y


Pumping v and y would create a string where there are more b’s than c ’s. Hence,the resulting string will not be in L3. This violates condition 1 of the pumpinglemma.

Option 4: Let vxy be some b’s followed by some c ’s.

If vxy is composed of some b’s followed by some c ’s, then the division of the inputstring can be viewed as


v x y


Pumping v and y seems to create a string that will still belong to L3 because theresulting string may still have more c ’s than b’s and more b’s than a’s.

However, recall that the pumping lemma allows the pumping down of strings.

Pumping down v and y (making v and y disappear) would create a string wherethere are more a’s than b’s and c ’s. Hence, the resulting string will not be in L3.This violates condition 1 of the pumping lemma.


Option 5: Let vxy be all c ’s.

If vxy is composed of all c ’s, then the division of the input string can be viewed as


v x y


Like option 4, pumping v and y will create a string that will still belong to L3 becausethe resulting string will still have more c ’s than b’s, and more b’s than a’s.

However, pumping down v and y (making v and y disappear) would create a stringwhere there are more a’s and b’s than c ’s. Hence, the resulting string will not be inL3. This violates condition 1 of the pumping lemma.


[Non-Context-Free Languages, Pages 1 –34 of 34]

Documents

MELJUN CORTES Automata Lecture Non-context-free Languages 1