Managing the Proliferation Problem€¦ · Copyright © 2017,Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|

ManagingtheProliferationProblemOracleLabsperspectiveonlanguage-integratedquerying&dataanalytics

Confidential– OracleInternal/Restricted/HighlyRestricted

HassanChafiSr.Director,ResearchandAdvancedDevelopmentOracleLabs

MatthiasBrantnerLaurentDaynesSungpack Hong


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Note:ThespeakernotesforthisslideincludeinstructionsforwhentouseSafeHarborStatementslides.

Tip!Remembertoremovethistextbox.

Confidential– OracleInternal/Restricted/HighlyRestricted 3


TheTradeoffTriangleLeadstoProliferationofSolutions


Productive/Easeofuse

General/ExpressivePowerExecutionPerformance


• ProgrammingLanguages(andassociatedlibraries)– Java– Python– R– Javascript– Ruby

• Databases– RowandOLTP– ColumnarandOLAP– DomainSpecific(Spatial,Multimedia)– ScaleOut(NoSQL)

• AnalyticalRuntimes–MapReduce,PIG,Flume– Delite,Spark,Flink– SparkProcessing,Storm– Tensorflow,Caffe


ProliferationofSolutionsEverywhereYouLook


ZoominonOneDomain:GraphAnalysis• Twomajorapporaches– Computationalgraphanalytics• Iteratethegraphmultipletimesandcomputeitsmathematicalproperties

– Graphpatternmatching• Querythegraphtofindsub-graphsthatmatchtothegivenrelationshippattern

•Manyworkloadsrequireapplyingbothapproachesatonce

6

spouse

friendfriend

friend


GraphSystemLandscape

GraphDatabase GraphFrameworkFocus:datamodeling,management,andquerying

RDF

PropertyGraph

Focus:graphtraversalandcomputation

RDF

PropertyGraph

John NewYorkbornIn

hasCarfriendOf

Hyundai Mary

LiveIn

favor

poetry

Twodifferentdatamodels

name:JohnhasCar:HyundaibornIn:New York

name:MaryliveIn:NewYorkfavor:poetry

label:friendOf

igraph NetworkX

SingleMachine

ClusterEnvironment

Executionenvironments

PGX

PGX(dist)

Newin12.2c

Integrationtoprovidebestofbothworlds

Confidential– OracleInternal 7


• AboutPGQL– NostandardquerylanguageforPropertyGraphsexists

– CypherfromNeo4J(proprietary)hasseveralissuesbothinsyntaxandsemantics

– PGQLisourownproposal,implementedinPGX

• PublicationActivities– Publishedthespecification(pgql-lang.org)– WeprovideaparserforitonGitHubforwideradoption

– WorkingwithISO/SQLtoextendSQLwithGraph

PGQL:aPropertyGraphQueryLanguageGRADES2016

Confidential– OracleInternal 8


ProliferationofLanguages• Languageshavebecomeafashionfordevelopers– Differentlanguagesfordifferenttasks• Web:JavaScript,Ruby,Php;Statisticalcomputing:R,Python;Systemscomputing:C,C++• Alltiersarebecomingpolyglot

–Manylargecompaniesallowdeveloperstochoose

• LongTailProblem(TIOBEPLIndexfromAugust2016)

OracleConfidential– HighlyRestricted

0%

5%

10%

15%

20%

Java C

C++ C#

Python PHP

JavaScrip

t

VisualBasic…

Perl

Assembly…

Delphi/O

bject…

Ruby

VisualBasic

Swift

Objective-C

Groo

vy R

MATLAB

PL/SQL

Go

ProgrammingLanguageMarketShare


GraalVM


Truffle/Graal

JavaScript R Ruby Python Go LLVM(C, Fortran)

HotSpotJVM

SubstrateVM

Polyglot Embeddable Performance


GraalVM forDataProcessing

UDFandStoredProcedureLanguageinDatabaseSystems

(OracleDatabase,MySQL,OracleNoSQL)

Just-in-timecompilationforSQLoperatorsandexpressions



JavaScriptUser-DefinedFunction

OracleConfidential– Internal/Restricted/HighlyRestricted 12

var validator = require(‘validator’);

module.exports.isEmail = function(email) {return validator.isEmail(input);

};

--- InSQLSELECTisEmail(email)FROMADDRESSBOOK;

// Export entry point to the Database (CommonJS)

// Use off-the-shelf NPM module

// TypeScript declarationexport function isEmail(email : string) : boolean;

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| Confidential– OracleInternal/Restricted/HighlyRestricted 13

ExpressioninWHEREclauseImpactonPerformance

30.01

5.63

0

5

10

15

20

25

30

35

sql wse

SQLExpressionOnly

31.93

5.22

05

101520253035

sql+js wse+js

SQLExpression+JavaScriptFunction

select count(*) from wmbint2 where (n1 * ((((n2 * 2) - n3)+n4) –

JS_ADD(n5, n6))) > 0;

select count(*) from wmbint2 where (n1 * ((((n2 * 2) - n3)+n4) –

(n5 + n6))) > 0;

second

s


AverageofseveralrunsinsamesessionTPC-HQ1&Q6

0

2

4

6

8

10

12

Oracle Walnut Oracle Walnut

Q1 Q6

Fetch LNX Walnut Other JIT0

0.5

1

1.5

2

2.5

1 2 3 4 5

Times(s)

Run#

JIT

Walnut

LNX

Fetch


SomeotherKeyChallengesResultingfromSolutionProliferation• DataMovementisabigissueforcustomers– Savvycustomersareawareofthisanditisoneofthefirstquestionstheyaskwhenbeingpitchedanewcomputationalengine

• Customerswanttoreducetimetoinsightfromrealtimedata– Dubiousonanythingthatrequireshoursofpre-processingwithnoincrementalstory

• Typicalworkflowsspanvariousdatasourcesthattypicallyarequeryableandencompasssometypeofsecuritymodel,socopyingthedataisnotalwaysfeasible– UnstableGeopoliticalenvironmentaddsfurtherrelevancytothisissue



• TopPGXrequestwas/isvisualizationandhigherlevelinterfaces(targetingtheExceluser)

• Ledtoadoptionofnotebookinterfaces– FirstintroducedbyMathematica1.0followedby

IPython (Jupyter),RStudio,thenZeppelin– EveryBigDatavendorisintroducinganotebook

product

• Generalrequirementforbigdataera– SWEngineersrequireIDEs– DataScientistsrequireDataStudios

• WesetouttoonlydevelopagraphvisualizationplugininJettobeembeddedinZeppelin– HitincompatibilitiesbetweenJetandZeppelin– DecidedtowriteaJetfront-endtoZeppelin

• Ambitionsgrewasweencounteredsuccess

MashupsofSolutionsmadeevenSimplerwithNotebooks


EnoughProblems,WhataretheOpportunities?• Canweachievemoreinteroperabilitybetweenruntimes?– Notonlyinhidingdatamovementdetail– ButalsoinbreakinguptheoverallworkflowcomputationintoahighlevelIRthatcanallowforoptimalplacementofcomputationandreductionofdatamovement

• Howdowegetvendorstoadoptaproposedsolution?– Networkeffectatplayhere

• Standardsaretypicallythemainwaytoimprovethesituation• WebStandardsseemtobeagoodmodelintermsofbalancingcommonstandardsandinnovations



WhatShouldThisLookLike?• CoreIR/Algebrafordataanalytics–OthersystemscanextendthisIRwithvendorspecificextensions,butthoseshouldalwaysbeexpressibleintermsofthecoreIRthatarepartofthecorestandardIR

• SecurityModelshouldbebakedin,withcelllevelsecuritylabelmodel– ReturnNullifuserdoesn’thaveprivilegetoaccessthedata– EnsureNullsemanticsallowreasonableprocessinginthepresenceofnulls

• DataProvenancemightalsobeimportanttobakein– Govt Regulationandcrossborderdatamovementemergingregulationbothachallengeandanopportunity



PluggableRuntimesandOptimizers• RuntimesthatarecompatiblewiththisIRhavetoimplementatleastasubsettogetthe“certified”seal,withmultiplelevelsofcompatibilitysupported– Scan+Filterataminimum

• RuntimesshouldreporttoaglobaloptimizerthesetofIRoperatorstheysupport• DatamovementbetweenRuntimes(includingdatalayouttransformation)shouldbeexplicitlysurfacedinthecostmodeltomakedecisiononwhendatashouldnotbemovedevenwhensignificantperformancegainsaretobehad• Datacachingandincrementalupdatestothedatashouldalsobepartofthedesignofthesystem– Dataisrarelystatic



LookingforwardtotheGreatDiscussions



SafeHarborStatementTheprecedingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Note:ThespeakernotesforthisslideincludeinstructionsforwhentouseSafeHarborStatementslides.

Tip!Remembertoremovethistextbox.



Documents

Managing the Proliferation Problem€¦ · Copyright © 2017,Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general