Advanced Couchbase LiteJens Alfke — Mobile Architect, Couchbase

• Advanced Querying• Prefix matching• The power of compound keys• Grouping• Pseudo-joins

• Conflict Resolution• What goes on during a conflict?• Detecting documents in conflict• Resolving conflicts

Topics

Fancy Queries

Getting the most out of map/reduce

1. String Matching

• Doing string prefix matching?• Normalize (lowercase) the string• Set the startKey to the prefix• Set the endKey to the prefix plus a high Unicode char

Prefix Matching

…jedjemimajenniferjensjenzieljen⊱℥⧝jerry…

"jen"

"jen\uFFFE"

• Searching for a value in multiple properties?• Emit each property as a key

Prefix Matching

emit(doc.firstName, value) emit(doc.lastName, value)

…

Example: Name Lookup

normalize(str) { return str.toLowercase.stripDiacriticals}

map(doc) { value = [doc.firstName, doc.lastName] emit(normalize(doc.firstName), value) emit(normalize(doc.lastName), value)}

// Search for name starting with "jen":query.startKey = normalize("jen")query.endKey = query.startKey + "\uFFFE"

2. Compound Keys

emit([k1, k2, k3], value)

• The view index compares arrays item-by-item

• This creates a hierarchical sort

The Joy Of Compound Keys

primary = doc.yearsecondary = doc.titleemit([primary, secondary], value)

[2001, "Crows, The"][2001, "Vizier Of One"][2005, "Melon Growing For Dummies"][2005, "Zamboni Simulator"][2010, "Aardvark Trees"]

• Use key ranges to pick specific primary key values

Search Ranges With Compound Keys

query.startKey = [2005]query.endKey = [2005, {}]

sorts after anything else

• Results are always sorted by key

• But you may want a different display order

• If so, you need to sort manually

Search Order vs. Display Order

rows = query.run().allObjectsrows.sort({a, b -> a.value.rating < b.value.rating})

3. Grouping

query.groupLevel = 2

• Query.groupLevel coalesces adjacent rowswhose key prefixes match• Key is shortened to the common prefix• Value is computed by view’s reduce function

Compound Keys For Grouping

[2001, "Crows, The"][2001, "Vizier Of One"][2005, "Melon Growing"][2005, "Zamboni Simulator"][2010, "Aardvark Trees"]

[2001] ➞ 2

[2005] ➞ 2

[2010] ➞ 1

groupLevel=1

Example: Column View Of Music Library

{ "_id": "EA735A02", "Album": "Atomizer", "Artist": "Big Black", "Name": "Kerosene", "TotalTime": 365, "TrackNumber": 4}

Example: Column View Of Music Library

map(doc) { emit([doc.Artist, doc.Album, doc.TrackNumber, doc.Name], doc.TotalTime)}

reduce(keys, values, rereduce) { return sum(values)}

Example: Column View Of Music Library

// Column 1: List all artists:query.groupLevel = 1for row in query.run() { print row.key[0], row.value }

// Column 3: List tracks of “Atomizer”:query.groupLevel = 0 // no grouping!query.startKey = ["Big Black", "Atomizer"]query.endKey = ["Big Black", "Atomizer", {}]for row in query.run() { print row.key[3], row.value }

// Column 2: List albums by Big Black:query.groupLevel = 2query.startKey = ["Big Black"]query.endKey = ["Big Black", {}]for row in query.run() { print row.key[1], row.value }

4. Pseudo-Joins

by grouping document types

• Usually a view will index only one type of document

• …but you can combine types to do “pseudo-joins”

Joining by mixing document types

Example: Blog Posts With Comment Counts

{"_id": "4312DFC8",

"type": "post", "title": "lunch today", "date": "2014-09-24", "body": "…"}

{"_id": "79111745",

"type": "comment", "post": "4312DFC8", "date": "2014-09-25", "body": "…"}

Blog Post Comment

Example: Blog Posts With Comment Counts

"4312DFC8" ➞ "Welcome""76AFB751" ➞ "Some news""76AFB751" ➞ null"82EB5C7F" ➞ "Question""82EB5C7F" ➞ null"82EB5C7F" ➞ null"82EB5C7F" ➞ null"82EB5C7F" ➞ null

"4312DFC8" ➞ ["Welcome", 0]"76AFB751" ➞ ["Some news", 1]

"82EB5C7F" ➞ ["Question", 4]

groupLevel=1

Example: Blog Posts With Comment Counts

map(doc) { switch doc.type { case "post": emit(doc._id, doc.title) case "comment": emit(doc.post, null) }}

reduce(keys, values) { //Note: ignoring rereduce title = values.first({x -> x != null}) return [title, count(values)-1]}

// List every post's title and comment count:query.groupLevel = 1for row in query.run() { print row.value[0], row.value[1] }

• Easy prefix matching

• Custom sorting

• In-memory filtering

• Fancy “query planner”

Upcoming Query Enhancements!

Don’t miss the next session:The Future Of Couchbase Lite

Conflict Resolution

• Different revisions with the same parent revision

What Are Conflicts?

Resolving A Conflict: Diagram

name: jillscore: 2

name: Jillscore: 2

name: jillscore: 3

99cf02e8

• Different revisions with the same parent revision

• Conflicts are prevented when working locally• Document.putProperties returns a “conflict” error

• Cannot be prevented in a distributed system• Multiple clients can update a document, then sync• Locking documents is infeasible

• A conflict is a branch in the revision tree

What Are Conflicts?

• How to detect conflicts?

• How to resolve conflicts?

Dealing With Conflicts

• Checking a document for conflicts• Document.getConflictingRevisions()

• Querying the database for conflicts• All-documents query with OnlyConflicts mode

Detecting Conflicts

query = database.createAllDocsQuery()query.allDocsMode = OnlyConflictsfor row in query.run() { conflicts = row.conflictingRevisions() myResolveConflict(row.document, conflicts)}

• Determine the resolved/merged properties

• Update one of the conflicting revisions

• Delete the other(s)

Resolving A Conflict

Resolving A Conflict: Diagram

name: jillscore: 2

name: Jillscore: 2

name: jillscore: 3

name: Jillscore: 3

_deleted: true

Resolving A Conflict: Pseudocode

myResolveConflict(doc, conflicts) {mergedProps = myMergeRevisions(conflicts)current = doc.currentRevisionfor rev in conflicts {

newRev = rev.createRevision()if rev == current {

newRev.properties = mergedProps} else {

newRev.isDeletion = true}newRev.save()

}}

app-specific merge strategy

• Any client can resolve conflicts• Resolution will be synced to everyone else• Resolution can be done on the server too

• If multiple clients resolve the same conflict• It might be fine (if the resolutions are identical)• Or it might create a new conflict to resolve

Interesting Facts

Questions?