November 2004csa3050: Sentence Parsing II1 CSA350: NLP Algorithms Sentence Parsing 2 Top Down Bottom-Up Left Corner BUP Implementation in Prolog

November 2004 csa3050: Sentence Parsing II 1

CSA350: NLP Algorithms

Sentence Parsing 2 • Top Down• Bottom-Up• Left Corner• BUP• Implementation in Prolog


Sources

• Jurafsky & Martin Chapter 10

• Covington Chapter 6

November 2004 3

Derivation top down,

left-to-right,

depth first


Bottom Up Filtering

• We know the current input word must serve as the first word in the derivation of the unexpanded node the parser is currently processing.

• Therefore the parser should not consider grammar rule for which the current word cannot serve as the "left corner"

• The left corner is the first preterminal node along the left edge of a derivation.


Left Corner

The node marked Verb is a left corner of VP

fl fl


Left Corner

• B is a left corner of A iffA * Bαfor non-terminal A, pre-terminal B and symbol string α.

• Possible left corners of all non-terminal categories can be determined in advance and placed in a table.


Left Corner (Operational Definition)

• A nonterminal B is a left-corner of another nonterminal A if: – A=B (reflexive case); – there exists a rule A → Bα for non-terminal A,

pre-terminal B and symbol string α; (immediate case)

– there exists a rule C such that A → Cα and B is a left-corner of C (transitive case).


DCG-style Grammar/Lexicon

s --> np, vp.s --> aux, np, vp.

s --> vp.np --> det nom.nom --> noun.nom --> noun, nom.nom --> nom, pppp --> prep, np.np --> pn.vp --> v.vp --> v np

What are the left corners of S?


Example of Left Corner Table

Category Left Corners

S

NP

Nominal

VP

Det, Proper-Noun, Aux, Verb

Det, Proper-Noun

Noun

Verb


How to use the Left Corner Table

• If attempting to parse category A, only consider rules A → Bα for which category(current input) LeftCorners(B)

S → NP VPS → Aux NP VPS → VP


Prolog Implementations

• Top Down: depth first recursive descent

• Bottom Up: shift/reduce

• Left Corner

• BUP


Top Down Implementationin Prolog

• Parser takes form of predicate parse(C,S1,S) : parse a constitutent C starting with input string S1 and ending with input string S.?- parse(s,[the,dog,barked],[]).

• If C is a pre-terminal category, check, use lexicon to determine that current input word has that category.

• Otherwise expand C using grammar rules and parse rhs constitutents.


Recoding the Grammar/Lexicon

% Grammar

rule(s,[np,vp]).

rule(np,[d,n]).

rule(vp,[v]).

rule(vp,[v,np]).

% Lexicon

word(d,the).

word(n,dog).

word(n,cat).

word(n,dogs).

word(n,cats).

word(v,chase).

word(v,chases).


Top Down Parser

parse(C,[Word|S],S) :- word(C,Word).

parse(C,S1,S) :- rule(C,Cs), parse_list(Cs,S1,S).

parse_list([],S,S).parse_list([C|Cs],S1,S) :- parse(C,S1,S2),parse_list(Cs,S2,S).


Shift/Reduce Algorithm

• Two data structures– input string– stack

• Repeat until input is exhausted– Shift word to stack– Reduce stack using grammar and lexicon until no

further reductions

• Unlike top down, algorithm does not require category to be specified in advance. It simply finds all possible trees.


Shift/Reduce Operation

→|Step Action Stack Input0 (start) the dog barked1 shift the dog barked2 reduce d dog barked3 shift dog d barked4 reduce n d barked5 reduce np barked6 shift barked np7 reduce v np8 reduce vp np9 reduce s


Shift/Reduce Implementationin Prolog

parse(S,Res) :- sr(S,[],Res).

sr(S,Stk,Res) :- shift(Stk,S,NewStk,S1), reduce(NewStk,RedStk), sr(S1,RedStk,Res).sr([],Res,Res).

shift(X,[H|Y],[H|X],Y).

reduce(Stk,RedStk) :- brule(Stk,Stk2), reduce(Stk2,RedStk).reduce(Stk,Stk).

%grammarbrule([vp,np|X],[s|X]).brule([n,d|X],[np|X]).brule([np,v|X],[vp|X]).brule([np,v|X],[vp|X]).

%interface to lexiconbrule([Word|X],[C|X]) :- word(C,Word).


Shift/Reduce Operation

• Words are shifted to the beginning of the stack, which ends up in reverse order.

• The reduce step is simplified if we also store the rules backward, so that the rule s → np vp is stored as the fact

brule([vp,np|X],[s|X]).

• The term [a,b|X] matches any list whose first and second elements are a and b respectively.

• The first argument directly matches the stack to which this rule applies

• The second argument is what the stack becomes after reduction.


Left Corner Parsing

• Key Idea: accept a word, identify the constituent it marks the beginning of, and parse the rest of the constituent top down.

• Main Advantages:– Like a bottom-up parser, can handle left recursion

without looping, since it starts each constituent by accepting a word from the input string.

– Like a top-down parser, is always expecting a particular category for which only a few of the grammar rules are relevant. It is therefore more efficient than a plain shift-reduce algorithm.


Left Corner Algorithm

To parse a constituent of type C:1. Accept a word W from input and determine K,

its category.2. Complete C:

– If K=C, exit with success; otherwise– Find a constituent whose expansion begins with K.

Call that CC. For instance, if K=d (determiner), CC could be Np, since we have rule(np,[d,n])

– Recursively left-corner parse all the remaining elements of the expansion of CC (in this case, [n]).

– Put CC in place of K, and return to step 2


Left Corner Implementationparse(C,[W|Rest],P) :- word(K,W), complete(K,C,Rest,P).

parse_list([],P,P).parse_list(([C|Cs],P1,P) :- parse(C,P1,P2), parse_list(Cs,P2,P).

complete(C,C,P,P). % if C=W, do nothingcomplete(K,C,P1,P) :- rule(CC,[K|Rest]), parse_list(Rest,P1,P2), complete(CC,C,P2,P).


Trace of Left CornerCall: ( 7) parse(np, [the, cat], []) ? creepCall: ( 8) word(_L128, the) ? creepExit: ( 8) word(d, the) ? creepCall: ( 8) complete(d, np, [cat], []) ? creepCall: ( 9) rule(_L153, [d|_G306]) ? creepExit: ( 9) rule(np, [d, n]) ? creepCall: ( 9) parse_list([n], [cat], _L155) ? creepCall: ( 10) parse(n, [cat], _L181) ? creepCall: ( 11) word(_L196, cat) ? creepExit: ( 11) word(n, cat) ? creepCall: ( 11) complete(n, n, [], _L181) ? creepExit: ( 11) complete(n, n, [], []) ? creepExit: ( 10) parse(n, [cat], []) ? creepCall: ( 10) parse_list([], [], _L155) ? creepExit: ( 10) parse_list([], [], []) ? creepExit: ( 9) parse_list([n], [cat], []) ? creepCall: ( 9) complete(np, np, [], []) ? creepExit: ( 9) complete(np, np, [], []) ? creepExit: ( 8) complete(d, np, [cat], []) ? creepExit: ( 7) parse(np, [the, cat], []) ? creep


BUP: Bottom Up Parser(Matsumoto et. al. 1983)

• Each PS rule goes into Prolog as a clause whose head is not the mother node but the leftmost daughter. The rule np → d n pp is translated as:d(C,S1,S) :- parse(n,s1,s2), parse(pp,S2,S3), np(C,S3,S).

• i.e. if you have just completed a d, parse an n, then a pp, then call the procedure for a completed np.

• In addition to a clause for each PS rule, BUP needs a terminating clause for every kind of constitutent, e.g.np(np,S,S).

• i.e. if you have just accepted an np and np is what you are looking for, you are done.


BUP - Remarks

• BUP is efficient because the hard part of the search – what to do with a newly completed leftmost daughter – is handled by Prolog’s fastest search mechanism – finding a clause given the predicate.


BUP Implementation - Parser% parse(+C,+S1,-S)% Parse a constituent of category C% starting with input string S1 and% ending up with input string S.

parse(C,S1,S) :- word(W,S1,S2), P =.. [W,C,S2,S], call(P).


BUP Implementation - Rules% PS-rules and terminating clauses

np(C,S1,S) :- parse(vp,S1,S2), s(C,S2,S). % S --> NP VPnp(C,S1,S) :- parse(conj,S1,S2), parse(np,S2,S3), np(C,S3,S). % NP --> NP Conj NPnp(np,X,X).

d(C,S1,S) :- parse(n,S1,S2), np(C,S2,S). % NP --> D Nd(d,X,X).

v(C,S1,S) :- parse(np,S1,S2), vp(C,S2,S). % VP --> V NPv(C,S1,S) :- parse(np,S1,S2), parse(pp,S2,S3), vp(C,S3,S). % VP --> V NP PPv(v,X,X).

p(C,S1,S) :- parse(np,S1,S2), pp(C,S2,S). % PP --> P NPp(p,X,X).

% Terminating clauses for all other categoriess(s,X,X). vp(vp,X,X). pp(pp,X,X). n(n,X,X). conj(conj,X,X).


BUP Implementation - Lexicon% Lexicon

word(conj,[and|X],X).word(p,[near|X],X).word(d,[the|X],X).

word(n,[dog|X],X). word(n,[dogs|X],X).word(n,[cat|X],X). word(n,[cats|X],X).word(n,[elephant|X],X). word(n,[elephants|X],X).

word(v,[chase|X],X). word(v,[chases|X],X).word(v,[see|X],X). word(v,[sees|X],X).word(v,[amuse|X],X). word(v,[amuses|X],X).


Principles for success

• Left recursive structures must be found, not predicted

• Empty categories must be predicted, not found• An alternative way to fix things is to tranform the

grammar into an equivalent grammar.– Grammar transformations can fix both left-recursion

and epsilon productions– But then you parse the same language but with

different trees

Documents

November 2004csa3050: Sentence Parsing II1 CSA350: NLP Algorithms Sentence Parsing 2 Top Down Bottom-Up Left Corner BUP Implementation in Prolog