Indexing & query optimization

Indexing & Query Optimization

Jared Rosoff (jsr@10gen.com)

Overview

•Indexing 101

•Profiling your queries

•Creating Indexes

•Managing Index

Indexing 101

Ack! My queries are slow!

http://michaelprescott.typepad.com/.a/6a00d83451574c69e20134858a87a2970c-800wi

Index FTW!

http://musformation.com/pics/excited.jpg

Why did that

happen?Magic

More Magic

Table scans

1 2 3 4 5 6 7

Looked at 7 objects

Find where x equals 7

Tree Lookup

7

6

5

4

3

2

1

Looked at 3 objects

Find where x equals 7

O(n) vs. O(log n)

Table scan

Index

Number of records

Num

ber

of

com

pari

sons

Analyzing Query Performance

Using the Profiler

db.setProfilingLevel( level )

0 == off

1 == slow operations (>100ms)

2 == all operations

Profiler Output

db.system.profile.find({millis:{$gt:5}});

{

"ts" : "Thu Jan 29 2009 15:21:27 GMT-0500 (EST)" ,

"info" : "query test.foo ntoreturn:0 exception bytes:53" ,

"millis" : 88

}

Use explainquery = db.coll.find({title:”My blog”})

query.explain();{ "cursor" : "BasicCursor", "indexBounds" : [ ], "nscanned" : 57594, "nscannedObjects" : 57594, "n" : 3, "millis" : 108}

Creating Indexes

Index a field

db.posts.ensureIndex( { ‘name’: 1 })

1 = ascending

-1 = descending

Compound indexes

db.posts.ensureIndex({name: 1, date: -1})

Unique Indexes

db.posts.ensureIndex({title: 1}, {unique: true})

Embedded documents

db.posts.save({

title: “My First blog”,

comments: [

{author: “James”, ts : new Date()} ]

});

db.posts.ensureIndex({“comments.author”: 1})

Multikeys{“tags”: [“mongodb”, “cool”], ...}

db.posts.ensureIndex({“tags”: 1})

Covered Indexes•New in 1.7.4

• Query can be resolved in index only

• Need to exclude _id from items projected

db.posts.ensureIndex({“title”: 1})

db.posts.find({“title”: “My blog post:}, {title: 1, _id:0}))

Sparse Indexes

Geospatial Indexes

Managing Indexes

Listing indexes

db.posts.getIndexes()

Dropping indexes

db.posts.dropIndex({“tags”: 1})

Background building

db.posts.ensureIndex(..., {background: true})

Query Planning

Query Planning

When isn’t an index used?

Picking an Index

scan

Index on x

Index on yRemember

Terminate

Review

•Understand your workload

•Profile your queries

•Use explain on the slow ones

•Create indexes for slow operations