113
More than SQL, but Less than ORM MoSQL (after v0.6)

MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Embed Size (px)

DESCRIPTION

It is the slides of the talk, "MoSQL: More than SQL, but Less than ORM", at PyCon APAC 2013. It introduces MoSQL after v0.6. About MoSQL: MoSQL is a Python library which lets you use common Python’s data structures to build SQLs. http://mosql.mosky.tw/

Citation preview

Page 1: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

More than SQL, but Less than ORMMoSQL (after v0.6)

Page 2: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Mosky

2

Page 3: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

MoskyI'm working at Pinkoi

2

Page 4: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

MoskyI'm working at Pinkoi

COSCUP staff

2

Page 5: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

MoskyI'm working at Pinkoi

COSCUP staff

Python trainer

2

Page 6: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

MoskyI'm working at Pinkoi

COSCUP staff

Python trainer

Speaker at COSCUP 2013, PyCon TW 2013, PyCon JP 2012, PyCon TW 2012 ...

2

Page 7: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

MoskyI'm working at Pinkoi

COSCUP staff

Python trainer

Speaker at COSCUP 2013, PyCon TW 2013, PyCon JP 2012, PyCon TW 2012 ...

http://mosky.tw/

2

Page 8: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Pinkoi.com  Builds  Design  Ecosystem

for  people  to  BUY  /  SELL  /  SHARE  designs  and  to  be  INSPIRED.

Page 9: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Pinkoi.com  Builds  Design  Ecosystem

Pinkoi  はアジアで最も大きいデザインショッピングウェブサイトです。優秀なデザイナー達がお客さんのためにいつもPinkoiで一番新しいデザインを提供しています。早めに

あなた達に会いたいですね。お楽しみ!

Page 10: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

5

Page 11: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

Why not SQL? But ...

5

Page 12: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

Why not SQL? But ...

Why ORM? But ...

5

Page 13: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

Why not SQL? But ...

Why ORM? But ...

MoSQL

5

Page 14: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

Why not SQL? But ...

Why ORM? But ...

MoSQL

The Usage, Performance, and Security

5

Page 15: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Outline

Why not SQL? But ...

Why ORM? But ...

MoSQL

The Usage, Performance, and Security

Demo

5

Page 16: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Doc: http://mosql.mosky.tw

Page 17: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Why not SQL?

Page 18: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

8

Page 19: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to UseSELECT * FROM article LIMIT 1;

8

Page 20: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to UseSELECT * FROM article LIMIT 1;

add ORDER BY created?

8

Page 21: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to UseSELECT * FROM article LIMIT 1;

add ORDER BY created?

add OFFSET 10?

8

Page 22: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to UseSELECT * FROM article LIMIT 1;

add ORDER BY created?

add OFFSET 10?

add GROUP BY author?

8

Page 23: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to UseSELECT * FROM article LIMIT 1;

add ORDER BY created?

add OFFSET 10?

add GROUP BY author?

UPDATE article WHERE title='SQL' SET title='ORM'?

8

Page 24: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

9

Page 25: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

Programming Error

9

Page 26: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

Programming Error

Programming Error

9

Page 27: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

Programming Error

Programming Error

Programming Error

9

Page 28: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Hard to Use

Programming Error

Programming Error

Programming Error!@#$

9

Page 29: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

May Be Injected

10

Page 30: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

May Be Injected

'WHERE ' + ' AND '.join( "%s = '%s'" for k, v in inputs)

10

Page 31: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

May Be Injected

'WHERE ' + ' AND '.join( "%s = '%s'" for k, v in inputs)

Cracker can inject from value

10

Page 32: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

May Be Injected

'WHERE ' + ' AND '.join( "%s = '%s'" for k, v in inputs)

Cracker can inject from value

or identifier, actually.

10

Page 33: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

May Be Injected

'WHERE ' + ' AND '.join( "%s = '%s'" for k, v in inputs)

Cracker can inject from value

or identifier, actually.

DON'T copy the code here!

10

Page 34: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

It seems bad! But ...

Page 35: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL ...

12

Page 36: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL ...

is fastest way to communicate with db,

12

Page 37: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL ...

is fastest way to communicate with db,

and everyone understands or learned it.

12

Page 38: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Why ORM?

Page 39: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

14

Page 40: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

class Person(Base): __tablename__ = 'person' person_id = Column(String, primary_key=True) name = Column(String) ...

14

Page 41: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

15

Page 42: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

mosky = Person('mosky', 'Mosky Liu')session.add(mosky)

15

Page 43: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

mosky = Person('mosky', 'Mosky Liu')session.add(mosky)

for person in session.query(Person).all(): print person.name, person.person_id

15

Page 44: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Easy to Use

mosky = Person('mosky', 'Mosky Liu')session.add(mosky)

for person in session.query(Person).all(): print person.name, person.person_id

Let you forget the ugly SQL so far.

15

Page 45: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL Injection Free

16

Page 46: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL Injection Free

Usually ORM guarantees it.

16

Page 47: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

It seems good! But ...

Page 48: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

ORM ...

18

Page 49: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

ORM ...

is slower,

18

Page 50: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

ORM ...

is slower,

and you need to learn it from scratch.

18

Page 51: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

ORM ...

is slower,

and you need to learn it from scratch.

Sometimes it is just a black box.

18

Page 52: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL vs. ORMSQL ORM

Easy-to-Use V

Secure V

Easy-to-Learn V

Fast V

Page 53: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

So ... MoSQL

Page 54: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The First Glance

21

Page 55: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The First Glance

from mosql.query import selectprint select('person')

21

Page 56: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The First Glance

from mosql.query import selectprint select('person')

-> SELECT * FROM "person"

21

Page 57: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is just condition

22

Page 58: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is just condition

select('person', { 'person_id': 'mosky'})

22

Page 59: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is just condition

select('person', { 'person_id': 'mosky'})

-> SELECT * FROM "person" WHERE "person_id" = 'mosky'

22

Page 60: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Sequence is just a list

23

Page 61: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Sequence is just a list

select('person', select=('name', ))

23

Page 62: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Sequence is just a list

select('person', select=('name', ))

-> SELECT "name" FROM "person"

23

Page 63: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is also a set-list

24

Page 64: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is also a set-list

insert('person', { 'person_id': 'mosky', 'name' : 'Mosky Liu'})

24

Page 65: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Map is also a set-list

insert('person', { 'person_id': 'mosky', 'name' : 'Mosky Liu'})

-> INSERT INTO "person" ("person_id", "name") VALUES ('mosky', 'Mosky Liu')

24

Page 66: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Order doesn't matter

25

Page 67: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Order doesn't matter

update('person', where={'person_id': 'mosky'}, set ={'name' : 'Mosky Liu'},})

25

Page 68: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Order doesn't matter

update('person', where={'person_id': 'mosky'}, set ={'name' : 'Mosky Liu'},})

-> UPDATE "person" SET "name" = 'Mosky Liu' WHERE "person_id" = 'mosky'

25

Page 69: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Operator also works!

26

Page 70: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Operator also works!

select('person', { 'age >=': 20})

26

Page 71: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Operator also works!

select('person', { 'age >=': 20})

-> SELECT * FROM "person" WHERE "age" >= 20

26

Page 72: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

All fromthe native data structures!

Page 73: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

28

Page 74: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

insert(table, set, ...)

28

Page 75: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

insert(table, set, ...)

select(table, where, ...)

28

Page 76: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

insert(table, set, ...)

select(table, where, ...)

update(table, where, set, ...)

28

Page 77: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

insert(table, set, ...)

select(table, where, ...)

update(table, where, set, ...)

delete(table, where, ...)

28

Page 78: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The Overview

insert(table, set, ...)

select(table, where, ...)

update(table, where, set, ...)

delete(table, where, ...)

...

28

Page 79: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

If you like it,

Page 80: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

sudo pip install mosql

Page 81: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Join is also available

31

Page 82: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Join is also available

select(    'person',    {'person_id': 'mosky'},    joins=left_join('detail',using=('person_id',)))

31

Page 83: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Join is also available

select(    'person',    {'person_id': 'mosky'},    joins=left_join('detail',using=('person_id',)))

-> SELECT * FROM "person" LEFT JOIN "detail" USING ("person_id") WHERE "person_id" = 'mosky'

31

Page 84: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

A Partial Query

32

Page 85: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

A Partial Query

fixed_args = {'table': 'person'}person_select = select.breed(fixed_args)person_select()

32

Page 86: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

A Partial Query

fixed_args = {'table': 'person'}person_select = select.breed(fixed_args)person_select()

-> SELECT * FROM "person"

32

Page 87: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

A Partial Query

fixed_args = {'table': 'person'}person_select = select.breed(fixed_args)person_select()

-> SELECT * FROM "person"

select('person')

32

Page 88: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

A Partial Query

fixed_args = {'table': 'person'}person_select = select.breed(fixed_args)person_select()

-> SELECT * FROM "person"

select('person')

-> SELECT * FROM "person"

32

Page 89: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Performance

33

Page 90: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Performance

About 4x faster than SQLAlchemy.

33

Page 91: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Performance

About 4x faster than SQLAlchemy.

Just a little bit slower than pure SQL.

33

Page 92: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Security

34

Page 93: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Security

Security by default.

34

Page 94: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Security

Security by default.

Use escaping technique.

34

Page 95: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Security

Security by default.

Use escaping technique.

Prevent SQL injection from both value and identifier.

34

Page 96: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Security

Security by default.

Use escaping technique.

Prevent SQL injection from both value and identifier.

Passed the tests from sqlmap at level=5 and risk=3.

34

Page 97: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL vs. ORMSQL ORM

Easy-to-Use V

Secure V

Easy-to-Learn V

Fast V

Page 98: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL < ______ < ORMSQL ORM

Easy-to-Use V

Secure V

Easy-to-Learn V

Fast V

Page 99: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

SQL < MoSQL < ORMSQL MoSQL ORM

Easy-to-Use V V

Secure V V

Easy-to-Learn V V

Fast V V

Page 100: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Demo

Page 101: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Demo

39

Page 102: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Demo

Arbitrary Query with Web

39

Page 103: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Demo

Arbitrary Query with Web

Serious Usage using Class

39

Page 104: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

Demo

Arbitrary Query with Web

Serious Usage using Class

All the code are in the Github!

39

Page 105: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The End

Page 106: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The End

41

Page 107: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

41

Page 108: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

41

Page 109: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

Easy-to-Learn

41

Page 110: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

Easy-to-Learn

Secure

41

Page 111: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

Easy-to-Learn

Secure

Fast

41

Page 112: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

Easy-to-Learn

Secure

Fast

sudo pip install mosql

41

Page 113: MoSQL: More than SQL, but Less than ORM @ PyCon APAC 2013

The EndMoSQL is ...

Easy-to-Use

Easy-to-Learn

Secure

Fast

sudo pip install mosql

http://mosql.mosky.tw/

41