Upload
sencha
View
42
Download
0
Embed Size (px)
Citation preview
Using ExtJS with Elasticsearch
Sam Imberman & Karim Besbes
AGENDA
1/ Introduction
Building a faceted catalog of video game assets using ExtJS and Elasticsearch
2/ Walkthrough of our applications
3/ Sharing our experience
4/ Lessons learned
5/ Q & A
INTRODUCTION
• The Technology Group is the primary technology partner of game production teams.
• We develop tools, middleware and online solutions used in Ubisoft games.
• Two sister teams- the TG, which is dedicated to developing tools and middleware solutions; and
- the TGO, which develops solutions and manages operations for online gaming.
Ubisoft – Technology GroupWho are we?
• Flare is “an internal Youtube for Ubisoft”- A video review solution built for Ubisoft game teams to more efficiently collaborate during the
creation of game art assets.
• Portfolio is “an internal Pinterest for Ubisoft”- A search engine for Ubisoft art assets. It enables production to easily find and reuse high quality
assets produced by other Ubisoft studios.
Our productsFlare & Portfolio
• For artists and animators
• Easy to use
• Same “look and feel”
• Deliver new features fast
• Built on top of ExtJS
Our productsFlare & Portfolio
• Designer (Sencha Architect) - Rapid prototyping for R&D
• Browser compatibility
• Data stores / Services / REST
• Routing
• Theming
• Build system
A Javascript Framework Why ExtJS ?
• Document oriented search engine• Distributed, multi-tenant, shards
• RESTful
• Extensible
• Faster aggregations than SQL! Faceting!
• Runs inside many turnkey systems that have a search component
An internal search engineWhat is Elasticsearch?
An internal search engineQuery language in JSON
• ES output is not good ExtJS input
• The client does not issue searches- ES is used in the back-end
• ExtJS is good to display information which is derived from ES data- Large search results are easily represented by grids, treegrids, etc
An internal search engineUsing ES with ExtJS
DEMO APPLICATION & WALKTHROUGH
Flare and Portfolio Walkthrough
• Flare uses Ext JS 5.x
• Portfolio uses Ext JS 6.x
SHARING OUR EXPERIENCE
• The scheduler updates too soon to catch dirty bound values
• notify() to update dirty bound values
View models & storesScheduler overview
var panel = Ext.create('Ext.panel.Panel', { layout: 'fit', viewModel: vm, bind: { title: '{foo}' }, renderTo: Ext.getBody()});
var vm = new Ext.app.ViewModel({ data: { foo: 'cool' }});
panel.getTitle(); // null vm.notify(); panel.getTitle(); // cool vm.set('foo', 'very cool'); panel.getTitle(); // cool Ext.defer(function() { panel.getTitle(); // very cool }, 5);
• ViewModel Store that has autoload = false and a bind descriptor in the URL is not processed by the ViewModel mechanism
View models & storesScheduler overview
var vm = new Ext.app.ViewModel({ data: { username: 'sencha' }, stores: { Profile: { autoLoad: false, fields: ["id", "wtv"], proxy: { type: 'ajax', url: 'api/v1/{username}/profile', reader: { type: 'json' } } } }});
var view = Ext.create('Ext.view.View', { viewModel: vm, tpl: new Ext.XTemplate( '<tpl for=".">', '<div class="selector"></div>', '</tpl>'), itemSelector: 'div', bind : { store: '{Profile}' }});vm.getStore('Profile').load()// Uncaught TypeError: Cannot read property ‘load' of nullvm.notify();vm.getStore('Profile').getProxy().getUrl();// api/v1/sencha/profile
• What could be easier than writing HTML to build a website?
• Powerful concept for a single page application
• HTML is stored in a string variable
• Full expressiveness of any CSS layout
• Looping structures display DOM based on data, with no ExtJS overhead
XTemplatesUsing HTML in ExtJS layouts
I can write tables in HTML!
XTemplatesUsing HTML in ExtJS layouts
XTemplatesInline Editing extension
• What if the video finishes processing while I’m writing a description?
XTemplatesDon’t override my DOM
• What happens if the layout run takes place with one of these sections closed?
XTemplatesLayout run fun!
• The fish is a video turntable
• We want to drag on it to rotate it
• In principle, we’d want to bind to the mousemove and mousedown events of a “video” tag
XTemplatesHow to turn a fish?
XTemplatesHow to deal with DOM events?
An individual element
Xtemplate container
View
Capture
Bubble
Xtemplate container
View
An individual element
XTemplatesHow to deal with DOM events?
Bind events here?
Capture
Bubble
Xtemplate container
View
An individual element
XTemplatesHow to deal with DOM events?
Bind events here?
Capture
Bubble
Xtemplate container
View
An individual element
XTemplatesHow to deal with DOM events?
Bind the event with a delegation
// component refers to the container which contains// the template
component.el.dom.addEventListener("mousemove", function(e) { if(e.target.id == "player-video" && e.buttons === 1) {
// . . . }}, true); // true to use capture phase because of video tag
• ExtJS layout runs don’t know how to deal with templates that change size
• Interactions are not always obvious to develop- Use “onclick=” , or
- Rebind events when regenerating template , or
- Delegate events to parent DOM elements
• But you get absolute flexibility within your layout
XTemplatesCreating interactions?
• Hard to maintain large chunks of HTML in XTemplates
• Why not load them separately?
• Load templates asynchronously
XTemplatesDynamic loading extension
//resources/templates/xtemplate.tpl.html
// script.js
• A dynamic template loader library
• Templates are declared in an external HTML file
• Wrapped in a <script> with type=“text/template”
• Easy to read and to maintain
XTemplatesDynamic loading extension
TemplateLoader.TEMPLATES_PATH = "./resources/templates/";
TemplateLoader.require('xtemplate.tpl.html', function (templates) { Ext.create('Ext.panel.Panel', { data : { helloWorld: 'Hello sencha con' }, tpl: templates.get('my-x-template-1') });
Ext.create('Ext.panel.Panel', { data : { anotherHelloWorld: 'Hello sencha con' }, tpl: templates.get('my-x-template-2') });});
<script id=“my-x-template-1” type="text/template"> <p> {helloWorld} </p></script><script id=“my-x-template-2” type="text/template"> <p> {anotherHelloWorld} </p></script>
Ext JS routingRoute nomenclature
https://portfolio/#!/library?q=couch&weapon=54dab480-e9f8-429a-9bcd-a62800fc9d53• In Portfolio -- IDs in URLs
• In Flare -- values in URLs
facet=GUID query=keyword
https://flare/#!/search?q=couch&uploader=Foo
No normalization required but is not human readable
Human readable but requires normalizationfacet=name query=keyword
• By default, ExtJS treats query strings as part of routes
• We overrode the routing class’s regular expression, so now routes can accept query strings
Ext JS routingRouting with query strings
// /? -> allows url route to end with or without "/"// (\\?.*) -> allows query string in urlreturn new RegExp('^' + url + '/?(\\?.*)?$', modifiers);
Ext JS routingThe query data manager
• URL and query string values should be generated and processed by one entry point
• We developed the concept of a query data manager
• A QueryData is a class that processes a query string and provides an interface to interact with
var q1 = "uploader=kbesbes&tag=%23foo&tag=%23boo", q1 = QueryData.fromQueryString(queryString);console.log(q1.getData());
// outputs: Object {uploader: Array[1], tag: Array[2]}
var queryString = "name=sencha&name=%C3%89tudiants", q1 = QueryData.fromQueryString(queryString);q1.getData();// name:// Array[2] 0:"etudiants"// 1:"sencha"q1.addOrRemove('name', 'Étudiants');q1.getData();// name:// Array[1] 0:"sencha"
Example 1:
Example 2:
Example 3: var result, queryString = "con=sencha&year=2016", q1 = QueryData.fromQueryString(queryString), q2 = QueryData.fromQueryString(queryString);
q1.add('speaker', 'karim');q1.toQueryString();// Outputs "con=sencha&year=2016&speaker=karim"result = QueryData.difference(q1,q2);result.toQueryString() ;// Outputs "speaker=karim"
• Render pages based on a specified route
• Controls facets and filters view based on its query string values
Ext JS routingThe route drives the view
renderMainContent: function(cmp) { var mainContent = Ext.ComponentQuery.query("#main-content")[0]; mainContent.removeAll(true); mainContent.add(cmp); },
onHomepageRouteTrigger: function() { this.renderMainContent(Ext.widget("homeMainContainer"));},
routes: { '!': 'onHomepageRouteTrigger', '!/:tenant/home': 'onTenantHomeRouteTrigger'},
onGalleryRouteTrigger: function(tenant) { this.setTenant(tenant); this.renderMainContent(Ext.widget("galleryMainContainer"));},
Example: http://localhost/#!http://localhost/#!/search?name=foo&since=2015
• ExtJS assumes 1 endpoint = 1 request = 1 proxy = 1 store
• But in Portfolio, we have data that is often related, but not associated
Display search resultsData model presentation
Display search resultsOne to one down the line
ResultStore
Proxy
ResultModel
/results/results: { … }facets: { … }
ResultStore
Proxy
ResultModel
/results/results: { … }facets: { … }
FacetStore
FacetModel
Display search resultsBut let’s make it messier
// Trigger the load of the facet tree store.var facetTreeStore = this.getViewModel()
.getStore("FacetTreeStore"), response = Ext.JSON.decode(
operation.getResponse().responseText), dataForFacets = this
.generateFacetStoreData(response);
facetTreeStore.setRootNode(dataForFacets);
• Chain-load a second store from a first
Display search resultsChain-loading stores
The tree we have The tree ExtJS wants"facets":[ { "path":"assetTypes", "name":"Asset type", "label":"Asset Type", "type":"Facet", "sortIndex":1, "expanded":true, "hasChildren":true, "children":[ { "value":"21b3a093-b591-409e-a4db-a5990133136d", "name":"Wildlife", "label":"Wildlife", "count":6, "filter":"assetTypes/any(e: e/id eq guid'21b3a093-b591-409e-a4db-a5990133136d')", "type":"FacetValue", "selected":true, "expanded":true, "hasChildren":true, "children":[ … ] }, … ]}, … ]
[{ "label":"Asset Type", "value":null, "checked":null, "count":null, "expanded":true, "leaf":false, "children":[ { "label":"Wildlife", "value":"21b3a093-b591-409e-a4db-a5990133136d", "checked":true, "count":6, "expanded":true, "leaf":false, "children":[ … ] }, … ]}, … ]
Display search resultA tale of two trees
generateFacetStoreData: function(obj) { // Traverse the facet tree to generate a new tree that will
// work properly with the ExtJS tree store.
return { label: obj.label, value: obj.value || null,
checked: obj.selected, count : Ext.isNumber(obj.count) ? obj.count : null, expanded: obj.expanded,
// If we have no children, mark this node as a leaf. // At top level, "children" node is called "facets". leaf: obj.children ? !obj.children.length : obj.facets ? !obj.facets.length : false,
// Recurse over every child. children: Ext.Array.map( obj.children || obj.facets || [],
function(item, index, array) { return this.generateFacetStoreData(item); }, this) }; }
• API data only contains state of API
• Our API doesn’t look exactly like an ExtJS Tree Panel
Display search resultGenerate tree data locally
LESSONS LEARNED
• ExtJS is a relatively good fit for our artistic-yet-enterprise applications
• Don’t fight the tide – use ExtJS for what it does well- Usually it is worth it to massage data to fit into ExtJS views – ‘it just works!’
- Instead of templates, maybe we should start writing components?
Lessons LearnedExt JS framework
• We have made a lot of incredible extensions, including …
Lessons LearnedExtensions
Q & A