Oracle PL/SQL: The Scripting Language Liberator · Model$ View Controller. ... Oracle PL/SQL: The Scripting Language Liberator Author: Oracle Subject: Oracle OpenWorld 2014 Presentation

Oracle PL/SQL The Scrip*ng Language Liberator

Steven Feuerstein Christopher Jones Oracle Corpora9on October 1, 2014 CON8265

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | -‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement The following is intended to outline our general product direc9on. It is intended for informa9on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func9onality, and should not be relied upon in making purchasing decisions. The development, release, and 9ming of any features or func9onality described for Oracle’s products remains at the sole discre9on of Oracle.

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 3


Program Agenda

Applica9on overview and problems

Chris’s client-‐side applica9on changes

Steven’s improvements

Adding new features

Conclusion

1

2

3

4

5

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 4


This session will… Focus on the data layer and database interac2on Demonstrate issues common to most client-‐server languages Leverage Oracle Database applica2on development features

<CODE> Client-‐Side Client-‐Side

<CODE>

SQL

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 5

PL/SQL


This session will not…

Dive into benchmarking or sta2s2cs Discuss GUI design

The specific client implementa9on language is not important

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 6


Presen9ng....DinoDate! Applica*on overview and problems

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 7


DinoDate: The Premier Da9ng Site for Dinosaurs

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 8

Model

View

Controller


Problems with DinoDate

High database server load

Slow user response *me

Monolithic architecture

<CODE>

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 9

Time for a demo!


Preliminary Fixes for DinoDate Chris’s client-‐side applica9on op9miza9ons

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 10


End-‐to-‐End Tracing Preliminary fixes for DinoDate

Allows DBA monitoring of the applica9on and statement execu9on

Set Client Iden2fier for each connec2on oci_set_client_identifier($this->conn, 'chris');

Add Client Informa2on, Module Name, Ac2on oci_set_client_info($this->conn, 'DinoDate');

oci_set_module_name($this->conn, $module);

oci_set_action($this->conn, $action);

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 11


Connec9on Pooling Preliminary fixes for DinoDate

Connec2on or session pooling reduces the number of Oracle server processes on the database host

DinoDate uses Database Resident Connec*on Pooling (DRCP)

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 12

SQL> execute dbms_connection_pool.start_pool()


Bind Variables Preliminary fixes for DinoDate

Hard Parse

Shared cursor in SGA

Cache latch conten9on

Shared pool conten9on

Scalability issues

SELECT dino_name FROM dd_members WHERE member_id = 123

Bind Variables

Reduce hard parses

Reduce risk of SQL injec9on

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 13

:mid


Statement Caching

Statement Caching – Reduces soe parses – Cuts repeated metadata processing – Consumes less network bandwidth – Reduces code path in the client 9er

Preliminary fixes for DinoDate

1. select …

Use #1

select … select …

Cache size was tuned for DinoDate

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 14

oci8.statement_cache_size = 40

Use #1


Some Language Drivers Auto-‐Commit by Default Preliminary fixes for DinoDate

Turned off auto-‐commit

oci_execute($this->statement, OCI_NO_AUTO_COMMIT);

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 15


Prefetching Preliminary fixes for DinoDate

Oracle fetches mul9ple rows in each “roundtrip” fetch call Buffering handled by Oracle Libraries No applica9on changes

Oracle Client Libraries

DB Driver


DB Driver

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 16

oci8.default_prefetch = 100


Client Query Result Caching

Caches query results in the applica9on Useful for small, sta9c lookup tables Oracle keeps cache in sync with server

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 17

Preliminary fixes for DinoDate

!


DB Driver

No applica9on changes needed ALTER SYSTEM SET client_result_cache_size=64M SCOPE=SPFILE; ALTER TABLE dd_dinosaurs RESULT_CACHE (MODE FORCE);


Preliminary Fix Summary

18

Chris has taken many of the basic steps toward op9mizing database usage and reducing communica9on between client and database…

-‐ End-‐to-‐end tracing -‐ Connec9on pooling -‐ Bind variables -‐ Statement Caching -‐ Not auto-‐commijng -‐ Pre-‐fetching -‐ Enabling the Client Query Result Cache

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 18


Steven, I S2ll Need Help! Deeper database op*miza*ons

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 19


Match up requirements with Oracle Database features Use built-‐in features instead of wri*ng code

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 20

Requirement Oracle Database Feature Improve speed and control of transac9ons Oracle PL/SQL stored procedures

Avoid processing bollenecks Oracle Advanced Queuing Reduce network traffic Oracle PL/SQL bulk processing Google-‐like search for matches Oracle Text Search for nearby dinosaurs Oracle Spa9al

Time for a demo!


User Registra9on

// Do some basic validation on the credentials if (is_null($dino_name) || is_null($password1) || is_null($password2) || ($password1 != $password2)) { return 0; } // Enter the new dinosaur details, setting subscription to 'P' since payment is pending $sql = "INSERT INTO dd_members (dinosaur_id, location_id, dino_name, about_yourself, subscription_status) VALUES (:did, :lid, :dn, :ay, 'P') RETURNING member_id into :mid”; . . . // Process the credit card payment $sql = "BEGIN dd_process_cc_payment(:mid, :dn, :card_num, :amount); END;”; . . . // Send a welcome message $sql = 'BEGIN dd_send_message(:fid, :tid, :s, :mc); END;’; . . . $db->commit();

Ini*al applica*on code

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 21




Remove blocking opera9ons Oracle Advanced Queuing Avoid row-‐by-‐row processing Oracle PL/SQL bulk processing Google-‐like search for matches Oracle Text Search for nearby dinosaurs Oracle Spa9al

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 22


User Registra9on

BEGIN IF LENGTH (password1_p) != LENGTH (password2_p) OR password1_p != password2_p THEN RETURN (0); END IF; INSERT INTO dd_members (dinosaur_id, location_id, dino_name, about_yourself, subscription_status)

VALUES (dinosaur_id_p, location_id_p, dino_name_p, about_yourself_p, 'P') RETURNING member_id INTO member_id_l; dd_process_cc_payment (member_id_l, dino_name_p, card_number_p, amount_p); dd_admin_pkg.send_message (dd_admin_pkg.c_admin_id, member_id_l, 'Welcome to DinoDate', 'When it comes to dinosaurs and love, DinoDate is the place to be!'); COMMIT; RETURN (member_id_l); END;

Move code into one PL/SQL block

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 23





-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 24


User Registra9on

BEGIN IF LENGTH (password1_p) != LENGTH (password2_p) OR password1_p != password2_p THEN RETURN (0); END IF; INSERT INTO dd_members (dinosaur_id, location_id, dino_name, about_yourself, subscription_status)

VALUES (dinosaur_id_p, location_id_p, dino_name_p, about_yourself_p, 'P') RETURNING member_id INTO member_id_l; dd_process_cc_payment (member_id_l, dino_name_p, card_number_p, amount_p); -- Send a message to AQ dd_payment_pkg.queue_payment(dd_user_payment_type(member_id_l, dino_name_p, card_number_p, amount_p)); dd_admin_pkg.send_message (dd_admin_pkg.c_admin_id, member_id_l, 'Welcome to DinoDate', 'When it comes to dinosaurs and love, DinoDate is the place to be!'); COMMIT; RETURN (member_id_l); END;

Use Oracle Advanced Queuing for slow opera*ons

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 25

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | -‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 26

DinoDate's Advanced Queuing Architecture User Registra9on

procedure auto_dequeue(..., msgid, ...) as begin DBMS_AQ.dequeue(..., msgid); dd_process_cc_payment(...); end;

Oracle Advanced Queue

function process_registration_aq(mem_id, dino_name,card_num, amount)

as begin ... dd_process_cc_payment(...); queue_payment(dd_user_payment_type(...)); ... end;

Time for a demo!


Administrator Broadcast Messages

$sql = 'SELECT member_id

FROM dd_members

WHERE member_id > 0’;

$res = $db->execFetchAll($sql, 'send_bc_1');

foreach ($res as $key => $to_member_id) {

$sql = 'BEGIN dd_admin_pkg.send_message(:fid, :tid, :s, :mc); END;';

$db->execute($sql, 'send_bc_2', array(

array(':fid', $from_member_id, -1),

array(':tid', $to_member_id['MEMBER_ID'], -1),

array(':s', $subject, -1),

array(':mc', $message_contents, -1)));

$db->commit();

}

Ini*al applica*on code

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 27





-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 28


Administrator Broadcast Messages

Can avoid excessive consump9on of PGA memory with the LIMIT clause.

Use PL/SQL procedure; u*lize bulk processing PROCEDURE broadcast_message (from_member_id_p IN INTEGER, subject_p IN VARCHAR2, message_in IN VARCHAR2) IS TYPE members_t IS TABLE OF dd_members%ROWTYPE; l_members members_t; BEGIN SELECT * BULK COLLECT INTO l_members FROM dd_members; FORALL indx IN 1 .. l_members.COUNT INSERT INTO dd_messages (from_member_id, to_member_id, subject, message_contents) VALUES (from_member_id_p, l_members (indx).member_id, subject_p, message_in); END;

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 29


Users always want more (job security!) Adding new features

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 30


DinoDate Search Results Improving the quality of matches

Implement “fuzzy” searches No dino will be happy with an exact-‐match search Messy typing shouldn’t be penalized (those keys are 9ny for dinos!)

Find nearby dinos These are the days before you could fly across Gondwana in a jet

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 31


DinoDate Search Results

Oracle Text is a SQL-‐level toolkit for developing text-‐centric applica9ons -‐ Provides indexes in Oracle Database for free-‐text (keyword) searching -‐ Much improved version of “like” operator -‐ Classical features (e.g. Fuzzy matching) and Advanced (e.g. Text Analy9cs)

Free with all edi9ons of Oracle Capable of managing many types of documents

Using Oracle Text

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 32


DinoDate Search Results

CREATE INDEX dd_ft_members ON dd_members (about_yourself) INDEXTYPE IS ctxsys.context; SELECT member_id, dino_name FROM dd_members

WHERE CONTAINS(about_yourself, 'love') > 0; MEMBER_ID DINO_NAME ---------- ---------- 1 Bob 10 Vicki

12 Alison

Crea*ng an Oracle Text index

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 33


DinoDate Search Results The CONTAINS keyword gives you access to powerful text analysis. – Use ? for fuzzy searching: bobb -‐> bob

– Use $ for stem-‐based searches: fly -‐> flying

Using an Oracle Text index

PROCEDURE text_only ( member_id_p IN INTEGER, search_string IN VARCHAR2, member_id_array OUT member_id_array_type, dino_name_array OUT dino_name_array_type, num_dinos IN INTEGER) IS BEGIN SELECT member_id, dino_name BULK COLLECT INTO member_id_array, dino_name_array FROM dd_members WHERE CONTAINS(about_yourself, search_string) > 0 AND member_id != member_id_p OFFSET 0 ROWS FETCH NEXT num_dinos ROWS ONLY; END;

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 34


DinoDate Members Love Australian Na9onal Parks

Big Country.

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐

FOUR HOURS FLYING TIM

E TODAY

35

Sure would be helpful to find nearby dinos . . .


Oracle Database “Spa9al” Tables Just like regular tables A column of type SDO_GEOMETRY stores the geometric shape of the objects

CREATE TABLE dd_locations ( id NUMBER PRIMARY KEY, location_name VARCHAR2(30), . . . geometry SDO_GEOMETRY );

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 36


SELECT geometry FROM dd_locations WHERE location_name like 'Kakadu%'; SDO_GEOMETRY( 2001, 4326, SDO_POINT_TYPE(132.673056, -12.271111, NULL), NULL, NULL)

Oracle Spa9al Data Contains a list of X and Y coordinates for points that describe the shape

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 37

Geometry type (here 2 indicates two dimensional, and 1 indicates a single point)

Coordinates of one point of the shape

Coordinate system of the shape (here WGS 84 Longitude/La9tude)


DinoDate PL/SQL query code Oracle Spa9al “Within Distance” Query

. . . AND sdo_within_distance ( dd_locations.geometry, geometry, 'distance=' || max_distance || ' unit=km') = 'TRUE'

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 38

Time for a demo!


Trees Lovers within 500 km and 1000 km DinoDate Members in Australian Na9onal Parks

Topsy @ Sturt NP Searching for 'trees'

Babylove @ Coorong NP

Tomas @ Flinders Ranges NP


Oracle Spa9al and Graph Op9on Bringing loca9on analysis and secure storage to enterprise data

Oracle Spatial and Graph

“Points” “Lines” “Polygons”

Rasters

Topologies

3D, point clouds (Lidar)

f1 f2 n1 n2

e1 e2 e3

e4 Network Graphs

Web Services (OGC)

RDF Seman9c Graphs

SPARQL End Point

Geocoding Rou9ng

Inferencing

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 40


DinoDate Applica9on Development Summary Client applica9on tuning Efficient use of PL/SQL Splijng a monolithic architecture – Oracle Advanced Queuing

Extending the applica9on – Oracle Text – Oracle Locator

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 41


You might be interested in . . . • Best Prac9ces For Applica9on Performance, Scalability & Availability – [CON8311] Thursday 12:00 -‐ 12:45pm, Moscone South -‐ 307

• Demo Booths in Moscone South –  Applica9on Development: SLD-‐163 –  PL/SQL: SLD-‐160 –  Advanced Queuing: SLD-‐146 –  Oracle Text: SLD-‐117 –  Oracle Spa9al Technologies: SLD-‐114, SLD-‐109

• OOW Sessions and Events – Oracle Spa9al: hlp://9nyurl/spa9al2014 – Applica9on Development: hlp://9nyurl.com/AppDevOOW2014

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 42


Safe Harbor Statement The preceding is intended to outline our general product direc9on. It is intended for informa9on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func9onality, and should not be relied upon in making purchasing decisions. The development, release, and 9ming of any features or func9onality described for Oracle’s products remains at the sole discre9on of Oracle.

-‐-‐-‐-‐ DRAFT SLIDES -‐-‐-‐-‐ 43


Safe Dinosaur Statement No dinosaurs were harmed in the making of this presenta9on. Timelines may not be accurate.


Documents

Oracle PL/SQL: The Scripting Language Liberator · Model$ View Controller. ... Oracle PL/SQL: The Scripting Language Liberator Author: Oracle Subject: Oracle OpenWorld 2014 Presentation