Seeking Life Beyond Relational: RavenDB

Sasha GoldshteinCTO, Sela Group

Level: Intermediate

NoSQL

• The Zen-like answer: No one can tell you what NoSQL is, they can only tell you what it isn’t

• It doesn’t use SQL• It usually is less consistent than RDBMS• It doesn’t put an emphasis on relations• It emphasizes size and scale over

structure

Classes of NoSQL Databases

Document DB

Key-Value

DB

Graph DB

Column DB

RavenDB

• Transactional document database• Open source

https://github.com/ravendb/ravendb with licensing for commercial projects

• Schema-less documents, JSON storage• RESTful endpoints• LINQ-style .NET API• Implicit (usage-based) or explicit indexing• Powerful text search based on Lucene• Replication and sharding support

Oren Eini(Ayende Rahien)

Hosting RavenDB

• Raven.Server.exe• Windows Service• Integrated in IIS• Embedded client for stand-alone apps• Cloud-hosted (e.g. RavenHQ)

Management Studio

DEMORavenDB Management Studio

Opening a Session

• DocumentStore is the session factory; one per application is enough

• Supports .NET connection strings or direct initialization:

var ds = new DocumentStore{ Url = "http://localhost:8888"};ds.Initialize();

CRUD Operations on Documents

• Unit of work pattern (ORM-style)

using (var session = documentStore.OpenSession()){ session.Store(new Speaker(“Sasha”, “Tel-Aviv”)); session.SaveChanges();}using (var session = documentStore.OpenSession()){ Speaker sasha = session.Query<Speaker>() .Where(e => e.City == “Tel-Aviv”).First(); sasha.City = “Orlando”; session.SaveChanges();}

Collections and IDs

• Documents are stored in JSON format• Documents have metadata that includes

the entity type• A collection is a set of documents with the

same entity type• Documents have unique ids, often a

combination of collection name + id– speakers/1– conferences/7

DEMOBasic Operations

Modeling Data as Documents

• Don’t be tempted to use a document store like a relational database– Documents should be aggregate roots– References to other documents are OK but (some)

data duplication (denormalization) is also OK

“conference/11” : { tracks: [ { title: “Web”, days: { 1, 2 }, sessions: [ ... ] }, ... ]}

Should the tracks be references?

…But Don’t Go Too Far

• Is this a reasonable document?

“blogs/1” : { tags : [ “Windows”, “Visual Studio”, “VSLive” ], posts : [ { title: “Migrating to RavenDB”, content: “When planning a migration to Raven…”, author: “Sasha Goldshtein”, comments: [ ... ] ... }, ... ]}

My blog has 500 posts

One More Example

“orders/1783”: { customer: { name: “James Bond”, id: “customers/007” }, items: [ { product: “Disintegrator”, cost: 78.3, qty: 1 }, { product: “Laser shark”, cost: 99.0, qty: 3 } ]}

What if we always need the customer’s address?

What if the customer’s address changes often?

What if we always need to know whether the product is in stock?

Include

• Load the referenced document when the referencing document is retrieved– Also supports arrays of referenced documents

Order order = session.Include<Order>(o => o.Customer.Id) .Load(“orders/1783”);Customer customer = session.Load<Customer>( order.Customer.Id);

Order[] orders = session.Query<Order>() .Customize(q => q.Include<Order>(o => o.Customer.Id)) .Where(o => o.Items.Length > 5) .ToArray();

DEMOInclude and Load

Indexes

• RavenDB automatically creates indexes for you as you run your queries– The indexing happens in the background after

changes are made to the database– Indexes can become stale– Can wait for non-stale results (if necessary)

RavenQueryStatistics stats;var results = session.Query<Speaker>() .Statistics(out stats) .Where(s => s.Experience > 3) .ToArray();if (stats.IsStale) ...

ACID?

• If indexes can become stale, does it mean RavenDB is not ACID?

• The document store is ACID• The index store is not

• You can insert lots of data very quickly and load it quickly, but indexes take a while to catch up

Indexing Fundamentals

• A document has fields that are indexed individually

• An index points from sorted field values to matching documents

"orders/1" : { customer: "Dave", price: 200, items: 3}

"orders/2" : { customer: "Mike", price: 95, items: 1}

"orders/3" : { customer: "Dave", price: 150, items: 2}

Customer Document IDs

Dave orders/1, orders/3

Mike orders/2

Price Document IDs

95 orders/2

150 orders/3

200 orders/1

Static (Manual) Indexes

• Static indexes can provide map and reduce functions to specify what to index

• The simplest form specifies a map function with the fields to index:

ds.DatabaseCommands.PutIndex(“Speaker/ByCity”, new IndexDefinitionBuilder<Speaker> { Map = speakers => from speaker in speakers select new { speaker.City } });

Hierarchical Data

• How to index the following hierarchy of comments by author and text?

public class Post{ public string Title { get; set; } public Comment[] Comments { get; set; }}public class Comment{ public string Author { get; set; } public string Text { get; set; } public Comment[] Comments { get; set; }}

Hierarchical Index with Recurse

public class CommentsIndex : AbstractIndexCreationTask<Post>{ public CommentsIndex() { Map = posts => from post in posts from comment in Recurse(post, c=>c.Comments) select new { Author = comment.Author, Text = comment.Text } }}

This is an index over Post objects but the output produces Comment objects!

Map/Reduce Index

• We often need the speaker count for each of our conferences:

ds.DatabaseCommands.PutIndex(“Conferences/SpeakerCount”, new IndexDefinitionBuilder<Conference, SpeakerCount> { Map = conferences => from conf in conferences from speaker in conf.Speakers select new { Item1 = speaker.Name, Item2 = 1 },

Reduce = results => from result in results group result by result.Item1 into g select new { Item1 = g.Key, Item2 = g.Sum(x => x.Item2) }

});

class SpeakerCount : Tuple<string, int> {}

Using Indexes

• In most cases you simply run a query and it will implicitly use or create an index

• Or, instruct the query to use your index:

var d = session.Query<SpeakerCount>(“Conferences/SpeakerCount”) .FirstOrDefault(s => s.Item1 == “Dave”);Console.WriteLine(“Dave spoke at {0} conferences”, d.Item2);

var posts = session.Query<Comment>(“CommentsIndex”) .Where(c => c.Author == “Mike”) .OfType<Post>();

Indexing Related Documents

• Use the LoadDocument method

public class OrderCustomerCityIndex : AbstractIndexCreationTask<Order, OrderCustomerCityIndex.Result>{ public class Result { public string City; } public OrderCustomerCityIndex() { Map = orders => from order in orders select new { City = LoadDocument(order.Customer.Id).City } }}

session.Query<OrderCustomerCityIndex.Result, OrderCustomerCityIndex>() .Where(c => c.City == “Orlando”) .OfType<Order>() .ToList();

DEMOUsing Indexes

Full-Text Search Indexes

• Made possible by the underlying Lucene.NET engine

public class SpeakerIndex : AbstractIndexCreationTask<Speaker>{ public SpeakerIndex() { Map = speakers => from speaker in speakers select new { speaker.Name }; Index("Name", FieldIndexing.Analyzed); }}

Using Full-Text Search and Query Suggestions

var query = session.Query<Speaker, SpeakerIndex>() .Where(s => s.Name == name);var speaker = query.FirstOrDefault();

if (speaker == null){ string[] suggestions = query.Suggest().Suggestions;}

Will find “Dave Smith” when searching for “dave” or “smith”

Will suggest “dave” when searching for “david”

Using Lucene Directly

• You can also query Lucene directly on any analyzed fields

• E.g., fuzzy search for sessions:

string query = String.Format("Title:{0}*", term);

session.Advanced.LuceneQuery<Session>("SessionIndex") .Where(query) .ToList();

DEMOFull-Text Search and Suggestions

Advanced Features

• Batch operations by index• Async API (OpenAsyncSession, await)• Attachments• Patching (partial document updates)• Change notifications (IDatabaseChanges)• Transactions (TransactionScope)• …and many others

http://ravendb.net/docs

Upcoming Features in RavenDB 3.0

• Management Studio rewrite in HTML5• Web API-based infrastructure• First-class Java client SDK• Custom storage engine (Voron)

Thank You!

Sasha Goldshtein

@goldshtn

blog.sashag.net