NoSQL: Why, When, and How
Preview:
Citation preview
- 1. NoSQLWhy, When, and How starring CouchDB
- 2. akabuilt rebuilding onhack atcontribute towork at
- 3. One More
thing...CoorganizingRESTFest2011w/MikeAmundsenrestfest.org
- 4. What isWhy NoSQL ^
- 5. NoSQL is...Not OnlySQLorSQLIvoteforthisone
- 6. Origins of the
NameCarloStrozziNoSQLappstartedin1998.RDBMSsansSQLappliedtononrelationalDBsaround2008ishno:sql(east)selectfun,profitfromreal_worldwhererelational=false;
- 7. Origin of the
SpeciesnonrelationaldatabasespredaterelationalonesitssortalikeAJAXbeendoinitforawhilegetsanamenowitscool!!1!
- 8. Types of NoSQL
DBsgraph(RDF/SemanticWeb/triples)keyvalue(justwhatitsays)document(k/v+queriability)object(biginthe1980s)multivalue(oldtech...like1960s)NewSQL?
- 9.
NewSQL?Recently(April,2011)coinedtermbyThe451GroupmostlymeansSQLdbs+betterscallabilityorNoSQLdbs+SQLlayersyeah!morekeywords!!keepsmarketinghappy...
- 10. ...on to specics
- 11. Graph (RDF) prettyheadystuff Web3.0?Maybe...
simpleconcept,FlockDB (from Twitter) complicatedexecution
(often)AllegroGraph Queriedwith: Cytoscape SPARQL Java
- 12. Key Value Stores scalablecaches generallyno
querylanguageProject Voldemortgetkey(s)Hibari returnvalue(s)
BigConeed driven
- 13. Document Databaseskey value + querying Lotus NotesAmazon
SimpleDB
- 14. object
- 15. multivalueprettyantiquelotsoflegacyrollouts
- 16.
NewSQLanyofthepreviouslymentionedDBs+querylayers...maybeexpectanyandallSQLDBstojumponthistrainand/orslipinthenextfewyearsworthalookifyou*must*havenormalizedstoragebutwhoneedsnormalization?
- 17. Why
- 18. One Reason
- 19. options
- 20.
denormalizationschemalessgraph/objectschemaclosermatchtobusinesslogicgenerallyfaster/morescalablecertainlymoredistributable
- 21.
alt.queriesMap/Reducethanks,Google(paperfrom2004)XPathand/orXQuerySPARQLSQL...orsomethingquitesimilarLinq
- 22.
licensingApacheLicense2.0thefavoriteAGPLv3thecommercialfavoriteothersincludeLGPL(v2/3),GPL(v2/3),BSD,MIT,customcommercialoropensource...heretherebedragons...
- 23. (un)expected extras at least in
CouchDBBuiltinWebServerorAppServerGeospatialboundingboxqueriesnmasterreplicationbinaryfilestoragescalesupanddown
- 24. When
- 25.
ScenariosScalabilitycaching,shardingAnalyticsDataWarehousingUbiquitous/DistributedDatamobile,desktop,server
- 26.
ScalabilitycachestyleDBsdataservedfromRAM(mostly)Membase,ElasticCouchbase,memcached,MongoDB,Casandrahorizontalscalabilityaddmoreservers,notmoreserver
- 27.
AnalyticsHadoopHBaseCassandrathisbleedsoverintodatawarehousingquickly
- 28. Ubiquitous/Distributed Data CouchDB serverdesktopmobile
RiakMobile? gotoErlangFactoryinJune
- 29. FALE
Scenariospoweroutagesdatalossfailedpersistence,nopersistencenetworkunavailability
special thanks to @coats who runs fale.ca for the FALE stamp
- 30. Solution: CouchDB
- 31. Howwith CouchDB
- 32. Time to Relax
- 33. Time to Relax ThatsDamien.HebuiltCouchDB.
- 34. ThatstheCouchDB gangsign.Time to Relax
ThatsDamien.HebuiltCouchDB.
- 35. ThatstheCouchDB gangsign.Time to Relax Learnit!
ThatsDamien.HebuiltCouchDB.
- 36. and heres why
- 37. CouchDB has Super
Powers!SchemalessDocumentcentricReplication/SyncFailFastArchitecturestatelessAPIappendonlyfilestorage
- 38. Document
centricnaturaldatamodelstoredatalikeitexistseverywhereelse:asadocumentmap/reducevs.sqlsortingdocumentsoutofadrawervs.reassemblingthemfrombitsofdata
- 39.
Replication/SyncMVCCbasedtransactionsversioning...butonlymeantfortransactionssafelymergedatabasesdocumentsarentcompared,onlyUUIDs&revisionIDsconflictingdocumentsaremarkedandawinnerispicked
- 40. Fail
Fastappendonlydatabasefileeverythinggoesontheendofthefilequerysarecachedtheretoobouncebackfromerrorsratherthanspinwheelsindefinitely
- 41. C.O.U.C.HCollectionOfUnreliableCommodityHardware
- 42. CouchDB
ScalingscalesupanddownserverdesktopmobilethankstonmasterreplicationBigCouchpluginforshardingHTTPAPIcanbeloadbalancedreverseproxiesandcaching
- 43. Get
CouchDBhttp://www.couchbase.com/downloads/couchbaseserver/communityhttp://iriscouch.com/bothhaveGeoCouchbuiltin!https://cloudant.com/(BigCouch/sharded)yoursystemspackageinstaller...http://wiki.apache.org/couchdb/Installation
- 44. Time to Relax (command line)Apache CouchDB 1.0.2
(LogLevel=info) is starting.Apache CouchDB has started. Time to
relax.[info] [] Apache CouchDB has started on
http://127.0.0.1:5984/
- 45. PHP & CouchDB
- 46. HTTP
ClientsAllyoureallyneedtogetstartedDoesrequireabetterunderstandingofCouchDBsAPI...butthatsAGoodThing!WillimproveyourHTTPskillz...anotherGoodThing!requiresmorework...thepartyouwontlikebutitsworthittolearnHTTP&REST
- 47. CouchDB
ClientsCouchDBsAPIisjustHTTPbut...helperlibrariescan...help:auto(en|de)codeJSONhandlebase64inginlineattachmentsmanageauthentication,cookies,OAuthtokenexchangecaching!!!_changesfeedwatching
- 48. CouchDB
ClientsSagforCouchDB(ApacheLicense2.0)PHPOnCouch(GPLv2orv3)Beyondhere,therebegiants...PHPillow(LGPL3)PHPObject_Freezer(BSD)PHPCouchDBExtension(PHPLicense3.0)Doctrine2CouchDBODM
- 49. HTTP
Clientscurlugh...messypecl_httplovely(nexttocurl),buttakessomeinstalltime,lacksexamplesZend_HTTP&PEARHTTP_Request2Mostmajorframeworkshavetheirown
- 50. Client
SuggestionsHTTPpickonethatsflexible(canhandleCOPY)CouchDBIuseSagcurrently.Caching,CookieAuth,nicename.PHPonCouchseemsgreat,butwatchthelicense(GPL)
- 51. Todays
StackInPHP:Sagsaggingcouch.com/ForHTTPAPIDemoing/Testing:Restygithub.com/micha/restyPosterforFirefoxcode.google.com/p/posterextension/
- 52. Other Handy
HTTPClientsHTTPClientforMacOSXCharlesProxy($$)httptwiddleforEmacsFiddlerforWindowsSolexforEclipse
- 53. CouchDB HTTP APIwell be back to PHP in a bit
- 54. JSON
DocumentsallresponsesarevalidJSONJSONsupportisbuiltintoPHP5.2+pecl&purePHP(de|en)codeforolderversions
- 55. A JSON Document{ json: key/value pairs, _id : some uuid,
_rev: mvcc key, string keys: [1,2,3,four,null], schema free: {so
its:flexible}}
- 56. JSON to PHP
Object$json={"json":"document","with":["an","array"]};$j=json_decode($json);//$jstdClassObject([json]=>document[with]=>Array([0]=>an[1]=>array))echo$j>with[1];
- 57. JSON to PHP array $json = {"json":"document","with":["an",
"array"]}; $j = json_decode($json, true); // $j Array ( [json]
=> document [with] => Array ( [0] => an [1] => array )
) echo $j[with][1];
- 58. I got tired of having topick -> or [] $j=newArrayObject(
json_decod($json), ArrayObject::ARRAY_AS_PROPS );
print_r($j[with][0]); print_r($j>with[0]);
//ansameresult!noerrors!
- 59. HTTP / REST basics GETread PUT create or
updateDELETEdeletePOSTbulk operation
- 60. Restycommand line RESTful good times
- 61. Setup
Restyinstall(seerestypage)$resty.#GET,POST,PUT,DELETE&HEAD#arescriptsnow!$restyhttp://localhost:5984/#setRestytodefaulttoCouchDB
- 62. Create a
Database$PUT/pouch/{ok:true}$GET/pcouch/{"db_name":"pcouch","doc_count":0,"doc_del_count":0,"update_seq":0,"purge_seq":0,"compact_running":false,"disk_size":79,"instance_start_time":"1289923325819422","disk_format_version":5,"committed_update_seq":0}
- 63. DELETE a
Database$DELETE/pouch/{ok:true}#^^^becarefulwiththatone!#letsrecreateit$PUT/pouch/
- 64. Create
DocumentPUT/pouch/tek{php:tek}{"ok":true,"id":"test","rev":"189af21439a03933bc3fc8c14cbeb496e"}
- 65. GET the
DocumentGET/pouch/tek{"_id":"test","_rev":"189af21439a03933bc3fc8c14cbeb496e","php":"tek"}//wellneedthat_revvaluetoupdatethisdoc
- 66. PUT (failure)on
purpose...PUT/pouch/tek{php:tek}{"error":"conflict","reason":"Documentupdateconflict."}#weneedthat_revvaluenow
- 67. PUT things
rightPUT/pouch/tek{"_rev":"189af21439a03933bc3fc8c14cbeb496e","php":"tek"}#thatrepresentsanewline#doccouldalsocontaintheid{"ok":true,"id":"tek","rev":"2fff750985c2c2602e859fe38cd1d347e"}
- 68. PUT binary attachments PUT/pouch/tek/photo?rev=2
fff750985c2c2602e859fe38cd1d347e Qfilename.png
HContentType:image/png {"ok":true,"id":"tek",
"rev":"318d519e58b569e43a6fd5e87491f0c4c"} GET/pouch/tek
{"_id":"tek", "_rev":"318d519e58b569e43a6fd5e87491f0c4c",
"_attachments":{"photo":{"content_type":"image/ png",
"revpos":1,"length":18,"stub":true}}}
- 69. A bit
aboutattachmentsEachattachmenttoadochasitsownURL:/pouch/tek/photo/pouch/tek/schedule.pdfeachattachmentshasitsownmimetypeattachmentscanbeadded/updatedviatheirownURLsorinline
- 70. DELETE (failure) on
purpose...againDELETE/pouch/tek{"error":"conflict","reason":"Documentupdateconflict."}#youalwayshavetosendarevwhenchangingadocinanyway
- 71.
DELETEDELETE/pouch/tek?rev=318d519e58b569e43a6fd5e87491f0c4cQ#thatQtellsRestynottourlencode{"ok":true,"id":"tek","rev":"43e8c21e8e610c4ea7f8e247a45c6eb04"}#what?!?anotherrev?...butthe#documentshouldbedead?!
- 72. GETing
404sGET/pouch/tek{"error":"not_found","reason":"deleted"}GET/pouch/tek12{"error":"not_found","reason":"missing"}#RESTifariannote:wouldbea409#ifcacheswerebuiltbetter
- 73. That was just the very basics GET/_stats(serverstats)
GET/_all_dbs(listallDBs) GET/_all_docs(listalldocs)
GET/db/_changes(listrecent changes) superpowers!
- 74. Map/Reduce Queriessearching the le
draweressentiallystoredqueriesa.k.a.ViewswritteninJavaScript(orPythonorRubyorErlangorPHP?)similartoarray_map()/array_reduce(),butscalablenoadhocqueries
- 75. Pouchput your media in the Couch starring PHP!
- 76. i want to
coverschemalessJSONdocs_securityBinaryAttachmentsserversiderenderingstuffMap/Reduce
_show Javascript _list severalexamplesURLRewritingreplication
_rewritesecurity vhosts validate_doc_updates
- 77. Pouch
is...aDatabaseofJSONdocsoffilemetadatawiththefileattached!afilesystemimporteraWebbasedCouchAppforbrowsing
- 78. 0.1 Files into CouchDB gatherEXIFdata addattachment
PUTtoCouchDB letusknowhowitwent
- 79. code preamble#!/usr/bin/php