A Domain-Specific Language and Rule Engine for Python
Michael Joseph Walshmjwalsh{nospam}mitre.org
Engineer
The MITRE Corporation
Approved for Public Release: 10-3919. Distribution Unlimited. Copy Right (c) -- The MITRE Corporation. All rights reserved.
1Monday, October 31, 11
mailto:[email protected]:[email protected]
the MITRE corporationabout
2Monday, October 31, 11
meabout
3Monday, October 31, 11
the backstory
4Monday, October 31, 11
Design and development
philosophy5Monday, October 31, 11
AntiPatterns |ant patrns|a pattern that tells how to go from a problem to a bad solution
6Monday, October 31, 11
http://www.amazon.com/gp/product/0471197130/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0471197130http://www.amazon.com/gp/product/0471197130/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0471197130http://www.amazon.com/gp/product/0471197130/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0471197130
Selling a Product You Can't Realize
Already being on the hook to deliver a marvelous rules engine that would solve all the projects problems, I didnt want to find out later I wasn't capable of delivering what I had promised.
promising youll be able to deliver a marvelous product that you know in advance you will be incapable of developing.
7Monday, October 31, 11
http://c2.com/cgi/wiki?SellingaProductYouCantRealizehttp://c2.com/cgi/wiki?SellingaProductYouCantRealize
Reinventing The Wheel
I'm not spoiling anything by saying I rolled my own rules engine, but in doing so I didn't want to unnecessarily create one when others already existed.
Go ahead. Grab that chisel and start pounding. You know you want to.
8Monday, October 31, 11
http://c2.com/cgi/wiki?ReinventingTheWheelhttp://c2.com/cgi/wiki?ReinventingTheWheel
Design for Sake of Design
I simply couldnt get caught caught up in a lengthy design with a finite amount of time and user base to be concerned with.
where you get caught up making a beautiful design, and forget your end result must be useful and doable in a finite time.
9Monday, October 31, 11
http://c2.com/cgi/wiki?DesignForTheSakeOfDesignhttp://c2.com/cgi/wiki?DesignForTheSakeOfDesign
Voodoo Chicken Programming
Not having programmed in either Python or ANTLR3, I had a significant learning curved to surmount. I wanted to avoid cobbling something together with the risk of having little understanding of how it worked.
Just wave a dead chicken at the code.
10Monday, October 31, 11
http://c2.com/cgi/wiki?VoodooChickenCodinghttp://c2.com/cgi/wiki?VoodooChickenCoding
false laziness, false impatience, and false hubris
I want to avoid
11Monday, October 31, 11
http://www.amazon.com/gp/product/0596000278/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0596000278http://www.amazon.com/gp/product/0596000278/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0596000278http://www.amazon.com/gp/product/0596000278/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0596000278
absorbing the cost of doing it right up front, so those coming behind you can be saved time
laziness
12Monday, October 31, 11
impatiencethe anger you have for when things aren't done right
13Monday, October 31, 11
hubrisExcessive pride, the sort of thing Zeus zaps you for. The quality that makes you author what other people won't want to say bad things about.
14Monday, October 31, 11
you should build new software,reuse,
or just get on with the damn job, and write something easy
The false, inverts of these, can lead you into making poor choices as to whether
15Monday, October 31, 11
a rules engine?whats
16Monday, October 31, 11
intellect |intlekt|the faculty of reasoning and understanding
it provides a form of artificial intelligence, an
17Monday, October 31, 11
knowledge |nlij|relevant facts and information
consisting of a working memory retaining
18Monday, October 31, 11
rules |rools|explicit or understood principles governing conduct within a particular activity or sphere
and a set of
19Monday, October 31, 11
condition |kndiSHn|boolean evaluations on the state of something
each rule has an optional
20Monday, October 31, 11
action |akSHn|process of doing something, typically to achieve an aim
and a suite of one or more actions
21Monday, October 31, 11
these actions eitherfurther direct the behavior of
the system, and/orfurther inform the system
22Monday, October 31, 11
facts |fakts|truths known about past or present circumstances
the engine starts with some
23Monday, October 31, 11
infer |infr|deduce or conclude information from evidence and reasoning rather than from explicit statements
uses rules to
more facts24Monday, October 31, 11
facts |fakts|truths known about past or present circumstances
these
25Monday, October 31, 11
fire more rules, that
infer |infr|deduce or conclude information from evidence and reasoning rather than from explicit statements
26Monday, October 31, 11
facts |fakts|truths known about past or present circumstances
infer yet more
and so on.27Monday, October 31, 11
Setting out
28Monday, October 31, 11
Facts should be Python objects
29Monday, October 31, 11
object-oriented programming |bjkt rnt prgramiNG |a paradigm of using objects, data structures consisting of data fields and methods together with their interactions -- to design applications.
30Monday, October 31, 11
Decorator pattern |dekrtr patrn|is a design pattern in object-oriented programming that allows behavior to be added to an existing object dynamically.
31Monday, October 31, 11
http://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29http://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29http://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Object_%28computer_science%29http://en.wikipedia.org/wiki/Object_%28computer_science%29
"Python is not Java."
32Monday, October 31, 11
Rules should be written in a manner that can be easily
grokked |grked|understood intuitively
33Monday, October 31, 11
I set out looking for a Python-based Open Source Rules Engine
to reuse34Monday, October 31, 11
PyKE
35Monday, October 31, 11
http://pyke.sourceforge.net/http://pyke.sourceforge.net/
Pychinko
36Monday, October 31, 11
http://www.mindswap.org/~katz/pychinko/http://www.mindswap.org/~katz/pychinko/
Fuxi |foo-shee|
37Monday, October 31, 11
http://code.google.com/p/fuxi/http://code.google.com/p/fuxi/
Could we use Drools?
38Monday, October 31, 11
Service
Drools
AMQP Broker
Python Client
reply queue
service rpc queuerequest
reply
39Monday, October 31, 11
Could we author our own DSL and Rules Engine in
Python?40Monday, October 31, 11
target to Python be actively maintained have an active user base be open sourced under a
favorable license literature supporting, tutorials for, and a Tool chain the would fit with my workflow, and be multi-platform
My selection
criteria came down
to these musts
41Monday, October 31, 11
ANother Tool for Language Recognition, version 3
42Monday, October 31, 11
grammar |gramr|a set of rules which describes how a stream of text defines the syntax of a language
ANTL3 allows you to author a
43Monday, October 31, 11
and generate a recognizer
44Monday, October 31, 11
lexer |lekr|breaks up an input stream into tokens
both a
45Monday, October 31, 11
and a
parser |prsr|feeding off the lexers token stream attempting to recognize structure either immediately emitting output or constructs an internal data structure
46Monday, October 31, 11
47Monday, October 31, 11
http://www.amazon.com/gp/product/0978739256/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0978739256http://www.amazon.com/gp/product/0978739256/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0978739256http://www.amazon.com/gp/product/0978739256/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeASIN=0978739256
48Monday, October 31, 11
http://www.amazon.com/gp/product/0123745144/ref=as_li_tf_il?ie=UTF8&tag=nemonikcom-20&linkCode=as2&camp=217145&creative=399377&creativeAS
