Custom Maps in Oracle Big Data Discovery with Oracle's ... · Oracle Spatial Summit at BIWA 2017 . Custom maps in Oracle Big Data Discovery (BDD) with Oracle’s Spatial and Graph

Oracle Spatial Summit at BIWA 2017

Custom maps in Oracle Big Data Discovery (BDD) with Oracle’s Spatial and Graph technologies David Lapp, Product Manager Oracle Spatial and Graph


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2


Agenda

• Cloud enabled data lab • Use cases for custom mapping in BDD • Technical overview • Demonstration • Resources


Oracle Big Data Discovery The visual face of Big Data, in the Oracle Cloud

find explore transform discover share


Oracle BDD built-in map features

“Thematic Map” • Standard administrative boundaries • Drill Country>State/Province>County • Use any metric

“Map” • Point locations using BDD’s geocode attribute type • Render as points, numbered markers, or heatmap • Radius search


BDCS Cluster

Cloud Enabled Data Lab

• Cloudera Enterprise • Oracle Big Data Connectors • Oracle Big Data Spatial and Graph • Oracle Data Integrator • Oracle Big Data SQL (add-on)

Oracle Big Data Cloud Service

Oracle Big Data Discovery Cloud Service

Had

oop

Nod

es

• Web Studio • In-memory data index • Data Processing • Sampling, Profiling, Enrichment,

Cataloging, Transforms

Had

oop

Nod

es


Oracle Big Data Spatial and Graph 12c (BDSG) Spatial features

• Vector and Raster analysis on Hadoop • Point in polygon, distance, buffer etc • Raster mosaic, subset etc

• Spark and MapReduce processing • Clusters, binning, categorization

• HTML5 map visualization API


Oracle BDD with BDSG Use cases

• Mapping with custom layers • Sales territories, flood zones, public safety regions • Transportation routes, utility network, waterway

• Integration and mapping of BDSG processing results • Spatial clusters • Spatial binning • Spatial catergorization


Custom Visualization Componant (CVC) BDD feature enabling custom mapping

• Framework for adding custom visualization to BDD

• Invoke any client-side JS library from CVC JS API

• Custom visualizations maintain state with the page

• CVC tutorial available on OTN


BDD custom mapping Using CVC with BDSG map visualization library

Oracle Maps JS API

CVC JS API

BDD dashboard

Custom map

Merge BDD results with custom layers

End user interactions

Custom layers

BDD results as JSON

GeoJSON

Standard componants


Demo • Using Big Data Lite VM bundled “Movie demo” content • Added “Division” attribute for mapping (based on US Census Divisions)



Reference individual BDSG map visualization libraries to avoid conflict with BDD’s libraries.

I used the name in the CVC tutorial. You can

use any name

Use any name for the CVC


1st part of code:

• Boilerplate from CVC tutorial

• Collect BDD results

• Create region on the canvas


2nd part of code:

• Merges BDD results with GeoJSON layer

• Renders map


Define EQL query, from CVC tutorial








Next steps for this demo

• Determine the map layer at runtime from the selected dimension

• Drive refinements from map to portal (supported by CVC API)

• Leverage BDSG spatial analytics such as clustering and binning


BDD custom mapping Using CVC with and Oracle Spatial and Graph

Oracle Maps JS API

CVC JS API

BDD dashboard

Custom map

Invoke map component

End user interactions

Automatically merge BDD results with custom layers

Spatial data

Predefined styles and themes

WebLogic

Oracle Database

Oracle Spatial and Graph

Map component

BDD results as JSON


Alternatives for custom BDD maps using CVC

Oracle Maps JS API only

• For Hadoop environment without Oracle Database (i.e. Oracle Big Data Cloud Service)

• Leverages mapping API of BDSG so no additional technology needed.

• All functionality is JS client

Oracle Spatial and Graph, and map component

• For environment that includes Hadoop and geospatial data in Oracle Database

• Requires deployment and administration of map component (servlet)

• Leverages server features, reducing need for JS client code


Resources Demo CVC JS code: Appendix to this presentation

Big Data Lite 4.21 VM: http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite421-2843803.html

Big Data Discovery CVC tutorial: http://www.oracle.com/technetwork/middleware/big-data-discovery/downloads/index.html

Big Data Spatial and Graph sessions: Workshop: Apply Location Intelligence and Spatial Analysis to Big Data with Java Tues 3:34-5:20, Room 202

Presentation: Bring Location Intelligence To Big Data Applications on Spark, Hadoop, and NoSQL Thus 2:30, Room 103

http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite421-2843803.html







http://www.oracle.com/technetwork/middleware/big-data-discovery/downloads/index.html







Appendix – CVC custom map example code. The following code sample is solely for demonstration purposes.


Oracle.BDD.Portlets.Visualization.Renderers.RendererTest = Oracle.BDD.Portlets.Visualization.Renderers.BaseRenderer.extend({ /** * Init function is straight from CVC tutorial */ init: function() { /** * Get the queryConfig for the initial query */ var queryConfig = this.getQueryConfig("eql"); /** * Set default value for num_recs data token if (!queryConfig.getToken("num_recs").getValue()) { queryConfig.getToken("num_recs").setValue("10"); } */ /** * Execute the initial query, indicating renderTest() * as the callback */ this.executeQuery(queryConfig, true, this.renderTest); } ,


/** * Callback function to create map */ renderTest: function(queryResults) { var self = this; /** * Get references to the data records and the queryConfig that * defined the EQL query. */ var records = queryResults.getRecords(); /** * Acquire reference to canvas, setting the root node as <div> */ var canvas = this.getCanvas({ tagName: 'div', margins: {top: 0, bottom: 0} }); canvas.clear(); var root = canvas.getRoot(); var divElement = document.createElement("div"); divElement.id = "mapDiv"; divElement.style.width= "800px"; divElement.style.height= "500px"; root.appendChild(divElement);


// oraclemaps code OM.gv.setResourcePath("http://localhost:7777/mapviewer/jslib/v2/"); var mpoint = new OM.geometry.Point(-100.45,25,8307); var map = new OM.Map(divElement) ; var tileLayer = new OM.layer.ElocationTileLayer("tilelayer"); map.addLayer(tileLayer); /** Next command adds map layer which requires url to json file. * In BDL VM, the following path maps to http://.../bdd/html * /u01/bdd/v1.1.1/user_projects/domains/bdd-1.1.1.13.11_domain * /servers/bigdatalite.localdomain/tmp/_WL_user/ * /oracle.endecastudio_1.1.1.0.0/b4iezl/war/html * so it’s convenent to put json map layer there */ layer = new OM.layer.VectorLayer("layer1", { def:{ type:OM.layer.VectorLayer.TYPE_DATAPACK, url: "http://localhost:9003/bdd/html/oraclemaps/datapacks/us_divisions.json" } });

// add layer listener to run after layer loads in order to // join the map layer to the BDD results layer.addListener(OM.event.LayerEvent.FEATURES_LOADED, this.afterFeaturesLoaded); // add layer listener to run when layer is clicked to // create pop up window layer.addListener(OM.event.MouseEvent.MOUSE_CLICK, this.featureClicked);


// transfer vars to listener fn by adding to layer // i.e. layer.testing = {"myRecords":records, "myDataObj": dataObj}; layer.bdd = {"bdd_records":records}; map.addLayer(layer) ; map.setMapCenter(mpoint); map.setMapZoomLevel(2) ; var legend = new OM.control.MapDecoration( "<table border=0 width=80> " +"<tr><td bgcolor='#0000ff'>&nbsp&nbsp&nbsp&nbsp</td><td>&nbsp Top Third </td></tr>" +"<tr><td bgcolor='#00ff00'>&nbsp&nbsp&nbsp&nbsp</td><td>&nbsp Middle Third </td></tr>" +"<tr><td bgcolor='#ff0000'>&nbsp&nbsp&nbsp&nbsp</td><td>&nbsp Bottom third </td></tr>" +"</table>", { offsetX:10, offsetY:375, draggable:true, contentStyle:{ "border-style":"none", "border-color":"#101418", "border-width":"1px", "font-size":"11px", "color":"#101418", "background-color":"#DAE1E3", "font-family":"Tahoma" } }); map.addMapDecoration(legend); map.init() ; } ,


afterFeaturesLoaded: function(event) { // layer is automatically transferred to listener fn as the event target var layer = event.target; var features = layer.getAllFeatures(); // retreive BDD records from layer.bdd var bddRecords = layer.bdd.bdd_records; console.log("---------------"); console.log("BDD data :"); // create hash of BDD results to use for matching to map layer var bddDataHash = {}; for (var n in bddRecords) { var tmp = JSON.stringify(bddRecords[n].groupby); if (tmp) { bddKey = tmp.replace(/\"/g, ""); //get rid of double quotes } // var bddVal = parseInt(JSON.stringify(bddRecords[n].metric)); var bddVal = parseFloat(JSON.stringify(bddRecords[n].metric)); if ( !isNaN(bddVal) ) { bddDataHash[bddKey]=bddVal; console.log(bddKey+":"+bddDataHash[bddKey]); } } console.log("---------------"); console.log("map layer -> BDD matching :");


// for each feature find matching BDD data and add its attribute for (var i = 0; i < features.length; i++) { feature = features[i]; mapKey=feature.getAttributeValue("NAME"); var featureAttr; if (bddDataHash[mapKey]){ console.log(mapKey+": match"); featureAttr = {"MAP_KEY": mapKey, "BDD_METRIC":bddDataHash[mapKey]}; } else { //console.log('no match for mapKey = '+mapKey); featureAttr = {"MAP_KEY": mapKey, "BDD_METRIC":null}; } layer.attrNames = ["MAP_KEY","BDD_METRIC"]; features[i].attributes=featureAttr; } var defaultColor = new OM.style.Color({fill:"#000000",strokeThickness:2,stroke:"#000000"}); var color01 = new OM.style.Color({fill:"#ff0000",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var color02 = new OM.style.Color({fill:"#00ff00",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var color03 = new OM.style.Color({fill:"#0000ff",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var equalBucket = new OM.style.BucketStyle({ styleName: 'Equal Bucket', numClasses : 3, classification: 'equal', gradient: "on", styles : [color01,color02,color03] }) ; layer.setRenderingStyle(equalBucket, ["BDD_METRIC"]); layer.redraw(); },


featureClicked: function(event) { var map = event.target.parentMap; var point = map.getCursorLocation(); var attributes = event.feature.attributes; var content = attributes.MAP_KEY+"<br>"+attributes.BDD_METRIC; map.displayInfoWindow( point,content); } });

// end of sample code


Documents

Custom Maps in Oracle Big Data Discovery with Oracle's ... · Oracle Spatial Summit at BIWA 2017 . Custom maps in Oracle Big Data Discovery (BDD) with Oracle’s Spatial and Graph