Upload
sivajag
View
158
Download
0
Embed Size (px)
DESCRIPTION
Simple introduction to Datomic
Citation preview
Introduction to Datomic
Siva Jagadeesan
“there’s an another database!”
• what do you think of first?
The job of a database
•Coordination •Consistency •Indexing •Storage •Queries
The traditional world
• Place-oriented-programming (PLOP)
• designed decades ago
• tiny RAM, tiny disks
• Collocated components
• Difficult to scale
• Necessitates application-level sharding
The dreaded update-in-place
sharding
• lose
• consistency
• transactions
• queries
key-value stores
• lose
• querying
• really just storage systems
• not “databases”
Deconstruction
• relocate subsystems
• separate services each do one thing
• simplification
Datomic architecture
• Peers
• Transactors
• Storage services
The new world
Benefits
• Separating reads and writes
• applications only read from storage
• transactor
• handles transactions
• provides consistency
• reflects changes to peers
Benefits
• Integrated data distribution
• built-in in-memory caches
• self-tunes to working set
• automatic
Benefits
• Peers each have a query engine
• Datalog
• simple rules and data patterns
• declarative: implicit joins
• declarative: meaning is evident
• locality: datomic db and app data
Benefits
• Elasticity
• as elastic as peers
Benefits
• Cloud ready
• commodity hardware
• resilient to failures
Data model
• Immutable data
• remember everything
• things don’t actually “change”
• audit everything
• automatic
Data model
• Facts-oriented (vs. PLOP)
• Atomic Data
• datoms
• entity, attribute, value, transaction
• Not embedded into strucures
Data model
• Minimal Schema
• at the datom level
• directly supports cardinality
• avoid rigidity
• even other “schema-free” documents impose their structure into apps
• Hickey: “nothing pivots like a datom”
Programming model
• Peer embedded in your application
• Pulls indexes or data segments as needed
• Caches locally, gets updates from transactor
Programming model
• No strings-based query language
• Data-structure-driven
• Lists and maps
• Easy to generate and test
Programming model
• Transactional
• Totally ordered
• First class: peers get a queue of all transactions
• Can use them in queries
• Facilitates event-driven triggers without polling
• Annotated transactions
Programming model
• Datalog
• deductive query system
• facts = datomic db
• can also include other app data
• extensible through custom functions that queries can use
Programming model
• Consistent
• without impeding other threads or peers
• reads/queries without transactions
• through immutability
• db as a value
Programming model
• Time
• as-of queries
• windowed queries
The so-what
• Simplicity
• consistency
• built-in caching
• no manual sharding
• configuration-free
• cloud-scale
The so-what
• Evolvability
• datom-level schema
• application free to change “structural” thinking
The so-what
• Local query power
• each peer has full query engine
• the db is effectively local
• isolation from others:
• e.g. - analytics usage won’t bog down transaction processing
The so-what
• Multiple storage choices
• in-memory for developing, unit-testing
• RDBMS backend possible (behind firewall, etc)
• Distributed storage services for redundancy and unlimited read scaling
• backup/restore tools available
The so-what
• Integrated mem-cache support
• local caches can be backed by OTS mem-cache cluster
• datomic is a “good citizen” user
The so-what
• Leverage datalog against multiple sources
• datomic db
• application data
The so-what
• Travel through time
• audits
• bug-resolution
Schema def
{:db/id {:part :db.part/db, :idx -1000001}, :db/ident :user/login, :db/valueType :db.type/string, :db/cardinality :db.cardinality/one, :db/fulltext true, :db/doc "the login handle of a user", :db.install/_attribute :db.part/db}
Schema def
{:db/id {:part :db.part/db, :idx -1000003}, :db/ident :user/friends, :db/valueType :db.type/ref, :db/cardinality :db.cardinality/many, :db/fulltext false, :db/doc "the friends of a user", :db.install/_attribute :db.part/db}
transactions
[:db/add entity-id attribute value]
[:db/retract entity-id attribute value]
{:db/id entity-id! attribute value! attribute value! ... }
[:db.fn/retractEntity entity-id]
identities
{:part :db.part/user, :idx -1000026}
{:db/id 17592186045420}
queries
[:find ?e :in $ ?email !:where ![?e :person/email ?email]]
[:find variables :where clauses]
[:find ?e ?x !:where ![?e :age 42] [?e :likes ?x]]
queries
• lots more
• pattern matching
• logic programming
• multiple data-sources
demonic
• datomic helper
• work at the data-structure level
• supports graphs: nested maps
• only writes dirty data
• “demarcations” as “batched datomic transactions”
• unit-testing support
Questions