Upload
gpzunino
View
215
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Using and Extending the Data Tools Platform
Citation preview
© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009
Using and Extending the Data Tools Platform
Brian Fitzpatrick (Sybase)
Linda Chan (Actuate)
Brian Payton (IBM)
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Welcome!
Goals for this tutorial:Get a working knowledge of DTP ToolingGet a glimpse into how to enable or extend database support for a particular vendorGet familiar with some DTP APIsUse ODA to extend DTP for a non-database data sourceSee how to expand the SQL Query Model and Parser for a particular DBAnd finally see how DTP is being leveraged in other Eclipse projects
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Before we begin…
What do you hope to get out of the talk today?
What are your goals? What do you know already about DTP?
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Beginning DTP
What is the Data Tools Platform (DTP)?
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Important Concepts in the Tooling
Connection Profiles & Driver Definitions
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Available Perspectives for DTP
Database Development
Database Debug
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Basic Components
Data Source Explorer (DSE)
SQL Scrapbook
SQL Results View
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Creating a Database Connection in DTP
In the Data Source Explorer, right-click on “Database Connections” and select “New…”Specify your database type, click NextIf no driver available, click the New Driver Definition button
Select a driver template, set your jar paths, click OKCustomize database properties, click FinishRight-click on new Connection Profile and select “Connect”
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Expanding and Exploring the Connected Profile
Once you connect to a profile, it starts to load the model a level at a time
So you can expand and see catalogs, schemas, tables, procedures, and so on – to represent what your database (and the DTP implementation for that database) supports
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Using the SQL Scrapbook to Execute Queries
Right-click on a database Connection Profile and select “Open SQL Scrapbook”
Type your query, select it, and select “Execute Selected Text”
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
How to Use the SQL Query Builder from the SQL Scrapbook
To create more complicated queries, you can use the SQL Query Builder (SQB), which simplifies query creation by allowing you to click, drag, and drop to customize the SQLTo open the SQB, right-click and select “Edit in SQL Query Builder”Right-click where it says “To add a table…” and select “Add Table…”Select one or more tables touse in the query and click OKThen you can select columns, specify conditions, and so on
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Executing a Query from the SQB
Once you’ve customized your query in the SQB, right-click in the SQL text area at the top and select “Run SQL” to show the results
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Interacting with Stored Procedures and UDFs
In SQL, you can define and call stored procedures in user-defined functions if your database supports them. DTP allows you to call them as well.
In a Derby database, you can call the SQLTABLES procedure:call SYSIBM.SQLTABLES(null, 'APP', null, null, null);
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Stored Procedures and UDFs continued…
You can also create and then use user-defined functions…
In a Derby database, try this:create function getSystemProperty(name varchar(128)) returns varchar(128) language java external name 'java.lang.System.getProperty' parameter style java no sql;
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Creating Tables and Modifying Content
In addition to simply running SQL, you can use DTP tools to create tables and view/modify their data
In the Data Source Explorer, connect to and drill down into a database connection profile. On the “Tables” folder in the tree, right-click and select “New Table…”
The New Table wizard creates DDL that you can then execute to create the table.
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Creating Tables and Modifying Content, continued
Execute the table DDL by right-clicking and selecting “Execute All” – this will run the SQL and create the table.
Then right-click on the “Tables” node in the tree, right-click and select “Refresh” to repopulate the table list.
Once you have a table, you can edit its content by right-clicking and selecting “Data->Edit”, which opens a tabular window for editing table data that you can use for data entry
When you’re done, right-click in the editor and select Save from the popup menu to persist your changes
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Creating Tables and Modifying Content, continued
Another way of loading data is by CSV file. To load a table froma CSV file, right-click on a table and select Data->Load…
Specify the input file, the delimiter, and the string delimiter, and click Finish.
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Tooling Exercise…
Set up a new Derby connection profile and connect to a local database
Connect to the connection profile
Create a new Product table with three columns – product_id (varchar), name (varchar), inventory (number)
Create a new Orders table with three columns – order_id (varchar), date (date/time), product_id (varchar), number_ordered (number)
Import data from the Products.csv and Orders.csv files into your new tables
In the SQL Scrapbook, create a query (in the text editor or the SQB) to correlate information from the Product and Orders tables to get a readable report of what products were ordered on a given date (product name, date, number ordered)
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Extending DTP
Now that you have a bit of knowledge about what DTP can do for you… How do you leverage and extend it?
A few possible avenues…Use DTP components in your own applications and simply leverage what’s already thereExtend DTP into new Data Sources and Databases (“enabling”new databases, adding custom items to the tree, adding custom SQL syntax to what DTP already knows, etc.)Take advantage of DTP APIs to utilize core DTP functionality under the covers
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
“Enabling” a New Database
Rather than spend a long time on this topic, I’ll direct you to a series of blog posts I wrote while I worked through “enabling” SQLite for DTP Enablement. Not all databases will require all of this, but it’s not difficult to extend the existing frameworks
Driver Framework: http://fitzdtp.blogspot.com/2008/06/dtp-driver-framework-repost.htmlCustom Driver Template: http://fitzdtp.blogspot.com/2008/06/how-do-you-add-your-own-custom-driver.htmlAdding a Custom Catalog Loader: http://fitzdtp.blogspot.com/2008/08/dtp-sqlite-support-continued-on-to.htmlCreating the SQLite Connection Profile (minus UI): http://fitzdtp.blogspot.com/2008/08/creating-actual-sqlite-connection.htmlCreating the SQLite Connection Profile UI Bits: http://fitzdtp.blogspot.com/2008/08/creating-sqlite-connection-profile-ui.html
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Leveraging DTP APIs
Though we have many cool tools in DTP, you may find that you don’t need or want to use them. In these cases, you may still be able to take advantage of DTP APIs.
There are too many APIs to really talk about in the time allowed, so I’m just going to hit upon a few of the high-level ones
In this section, we’re going to use DTP APIs to create a connection profile and write a utility class that takes a file, loads it into memory, and loads a table from the file
Consider that APIs could also be used to map/replicate data fromone database to another via a combination of JDBC and DTP
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
APIs for Creating Connection Profiles…
There are two types of connection profiles…persisted and transient.
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Accessing Existing Connection Profiles (Persisted)
Access to existing connection profiles (persisted or transient) is done using the ProfileManager class (org.eclipse.datatools.connectivity.ProfileManager)
If you know the name of your connection profile, you can find itpretty easily
IConnectionProfile myProfile = ProfileManager.getInstance().getProfileByName(“MyProfile”);
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Creating a Transient Connection Profile
In the second case, where you may not want to use the whole connection profile UI framework but want to leverage other components, you use the Transient API
These are the typical JDBC properties you think of…String profileUserName = "";String profilePassword = "";String profileURL = "jdbc:derby:C:\\DerbyDatabases\\MyDB;create=true";String profileDriverClass = "org.apache.derby.jdbc.EmbeddedDriver";String profileJarList = "C:\\Derby10.2.2\\db-derby-10.2.2.0-bin\\lib\\derby.jar";
Then there’s the DTP-specific properties you need…String providerID =
"org.eclipse.datatools.connectivity.db.derby.embedded.connectionProfile";String vendor = "Derby";String version = "10.1";
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Transient Profile, continued
So once you have your properties, you can do this:Properties baseProperties = new Properties();
baseProperties.setProperty( IDriverMgmtConstants.PROP_DEFN_JARLIST, jarList );baseProperties.setProperty(IJDBCConnectionProfileConstants.DRIVER_CLASS_PROP_ID,
driverClass);baseProperties.setProperty(IJDBCConnectionProfileConstants.URL_PROP_ID, driverURL);baseProperties.setProperty(IJDBCConnectionProfileConstants.USERNAME_PROP_ID, userName);baseProperties.setProperty(IJDBCConnectionProfileConstants.PASSWORD_PROP_ID, password);baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VENDOR_PROP_ID,
vendor);baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VERSION_PROP_ID,
version);baseProperties.setProperty( IJDBCConnectionProfileConstants.SAVE_PASSWORD_PROP_ID,
String.valueOf( true ) );ProfileManager pm = ProfileManager.getInstance();IConnectionProfile icp = pm.createTransientProfile(providerID, baseProperties );
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Transient Profile, continued…
So the upside to the Transient profile is that you don’t have to pre-define it in the UIThe downside is that you have some additional properties to figure out.
This is *new* API for Galileo. There are other changes to help with getting the correct vendor and version in Galileo so you can hand it a JDBC connection you managed yourself, it gets the properties it needs, and then returns the correct vendor, and version.For now, you can use the provider ID of the Generic JDBC profile(org.eclipse.datatools.connectivity.db.generic.connectionProfile)
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Using your Connection Profile reference
Now that you have an IConnectionProfile instance, what can you do with it?If it’s not connected, you can connect to it:
IConnectionProfile profile = ProfileManager.getInstance().getProfileByName("myprofile");
IStatus status = profile.connect();if (status.equals(IStatus.OK)) {
// success} else {
// failure :(if (status.getException() != null) {
status.getException().printStackTrace();}
}
If you get an exception, you can react accordingly…
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Using your Connection Profile reference…
So now that you have a connected profile, you can get the JDBC connection from the ProfileConnectionManager:
public java.sql.Connection getJavaConnectionForProfile(IConnectionProfile profile) {
IConnection connection =ProfileConnectionManager.getProfileConnectionManagerInstance().getConnection(profile, "java.sql.Connection");
if (connection != null) {return (java.sql.Connection) connection.getRawConnection();
}return null;
}
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Using the Connection…
From there, you can do anything you might have done with any other JDBC connection:
try{Statement st = connection.createStatement();String statement = “select * from mytable”;int val = st.executeUpdate(statement);
}catch (SQLException s){
// exception}
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Other good API bits…
You can also get the SQL model from a database connection. First, you have to get the ConnectionInfo class, which acts as a bridge between the JDBC connection and the SQL models
IManagedConnection managedConnection = ((IConnectionProfile)profile).getManagedConnection("org.eclipse.datatools.connectivity.sqm.core.connection.ConnectionInfo");
if (managedConnection != null) { try {
ConnectionInfo connectionInfo = (ConnectionInfo) managedConnection.getConnection().getRawConnection(); if (connectionInfo != null) {
Database database = connectionInfo.getSharedDatabase();// do something with the database reference…
}} catch (Exception e) {
e.printStackTrace();}
}
This is how to get the SQL Model’s root Database node for your connection profile
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Using the Database reference…
Once you have the Database reference, you can drill into your database and get the abstracted SQL structure, which you can then use to develop more flexible SQL calls for a variety of reasons…
Imagine being able to connect to two databases and providing a tool that allows you to map data between disparate tables…Imagine writing tooling or a set of APIs that could be used to inject data via DTP’s abstracted modelsImagine abstracting a data source class for a given table or set of tables that would hide SQL complexity and allow a developer to more easily add/remove/edit data via simple callsThe sky’s the limit!
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
API Exercise…
Using the base plug-in provided, in the action’s run method, dive into the connection profile and print a few things to the console for the Product table in the App schema of our Derby database…
MyDatabase.getCatalogs() returns an EList of catalogs. Derby has a dummy Catalog object, so you’ll have to get it and then go deeper ( (Catalog)MyDatabase.getCatalogs().get(0) )MyCatalog.getSchemas() returns an EList of schemas you can iterate through until you find APPOnce you find the Schema, MySchema.getTables() returns an EList of tables you can iterate through until you find ProductMyTable.getColumns() returns an EList of columns…And so on… simply work down the hierarchy and show a list of the columns in the Product table
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
In Summary… Tools + API = Flexibility
DTP’s tooling provides useful database and data source functionality that’s also extendable and reusable in your own code
And DTP’s APIs allow you to go deeper than the tooling to do many cool things with JDBC, the SQL model, and your own tools/APIs
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
More DTP Information
Main DataTools site at http://www.eclipse.org/datatoolsNewsgroup - news://news.eclipse.org/eclipse.dtpMailing list – http://dev.eclipse.org/mailman/listinfo/dtp-devWiki - http://wiki.eclipse.org/index.php/Data_Tools_Platform_ProjectBugzilla for bug or enhancement requests
Feel free to ask questions!
© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009
DTP Open Data Access (ODA) Framework
Extend DTP for Non Relational Database Data Sources
Linda Chan – Actuate® Corporation
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Topics
What is Open Data Access1 (ODA)?
Overview – ODA Framework
Enabling non-RDBMS data source with an ODA Data Provider
Demo/Exercise: creating ODA provider plug-ins
1 Actuate® patent pending
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Open Data Access Framework
Enables data access to any data source• Provides an abstraction for accessing heterogeneous data
sources• Highly scalable data retrieval and easy end-user experience• Built using familiar Eclipse extension points
Extends & customizes Applications Systems• Applications can provide custom data driver and query builder
for accessing their data
Provides proven framework• BIRT’s built-in ODA data connectors• Commercial products’ ODA data connectors
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Overview – Open Data Access Architecture
ODA Data Driver
ODA Query Builder
Application Client ToolsBIRT Report Designer
Data Sources
Application Server
ODA RuntimeInterfaces
ODA DesignInterfaces
Data Access & QueryDefinition
Complete control of data access and branding
Run-Time Data AccessDesign-Time Query Builder
Complete control of data access and branding
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Overview – Open Data Access Framework
Run-timePublic API defines a set of Java™ interfaces that allow heterogeneous data to be accessed in a generic way
Design-timeAllows data source-specific user interface to be embedded in any ODA-compliant consumer application
ODA Consumer HelperBuilt-in support for any ODA-compliant consumer application to consume data from any ODA data provider
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Run-time Framework
Stable run-time API since DTP 0.9
Integrates with DTP Connection Profile and JDBC™ Databases framework
Implementation wraps data source-specific APIs
Run-time plug-in implementsextension points
org.eclipse.datatools.connectivity.oda.dataSourceorg.eclipse.datatools.connectivity.connectionProfile
ODA run-time API interfacesorg.eclipse.datatools.connectivity.oda package
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Run-time API Interfaces
Defines the primary run-time operations of an ODA data provider to access and retrieve data from a data source
Java interfacesJDBC-like, extended to support additional capabilities of non-RDBMS data sources Emphasis on scalable data retrieval
Main run-time operationsData Source Connection – IConnection
Establishes a live connection to any type of data sourceObtains provider capabilities for each type of data set queriesCreates one or multiple concurrent data set-specific queries
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Run-time API Interfaces
Main run-time operations (cont’d)Data Set Query – IQuery, IAdvancedQuery
Prepares and executes a data set-specific query text command, e.g. XPath, MDX, SQL, Stored Procedure callsHandles one or more sets of data rows, i.e. result sets, retrieved by a single data set query
Query Parameters – IParameterMetaData, IParameterRowSet
Provides run-time metadata of parameters specified in a prepared queryHandles scalar and complex input/output parameters
Result Sets – IResultSet, IResultSetMetaData
Fetches tabular data rows Allows sequential or concurrent access to multiple result sets
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Design-time Framework
Eclipse Modeling Framework (EMF) model-based design-time interfaces
communicate connection properties, query and parameter definitions to an ODA consumer application
Integrates with DTP Data Source Explorer view
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Design-time Framework
ODA Consumer Designer (e.g. BIRT Data Source Wizard) Customized Wizard Page
contributed by ODA Data Providers
ODA Design Session model
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Designer UI Plug-in
Implements extension points
org.eclipse.datatools.connectivity.connectionProfileorg.eclipse.ui.propertyPagesorg.eclipse.datatools.connectivity.oda.design.ui.dataSourceorg.eclipse.datatools.connectivity.ui.connectionProfileImage(optional)
customizes ODA Designer UI pagesorg.eclipse.datatools.connectivity.oda.design.ui.wizardspackage
Communicates its Data Source and Data Set design definitions in an ODA Design Session model
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Design Session model
Model based on Eclipse Modeling Framework (EMF)org.eclipse.datatools.connectivity.oda.design package
Allows customized data source and query builders to design the slice of data to access at run-time
Communicates connection information, query and parameter definitions to an ODA consumer applicationTransient Objects
root element: org.eclipse datatools.connectivity.oda.design.OdaDesignSession
ODA design-time consumer application, e.g. BIRT Report Designer, initiates an ODA design session
Consumes a data provider’s UI page contributionsAdapts an edited data access design to host-specific design components Provides persistent services for editing designs
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Build a Custom ODA Data Provider
From scratchPDE New Plug-in Project Wizard
Demo / Exercise
Extends an existing ODA providerdriverBridge extension point
org.eclipse.datatools.connectivity.oda.consumer.driverBridge
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Build a Custom ODA Data Provider – from Scratch
PDE New Plug-in Project WizardDemo / Exercise
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Extends an Existing ODA Data Provider
Modify/enhance behavior of an existing ODA data providerorg.eclipse.datatools.connectivity.oda.consumer.driverBridge extension point
Ideal for minor enhancementspreserves existing ODA extension IDSupports “chained” bridges
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
DriverBridge Extension Point
ODA Consumer Application(e.g. BIRT)
ODA Run-time Driver
ODA RuntimeInterfaces
oda.consumer.helper
Driver Bridge
driverType driver class name or first matching interface name
Driver BridgebridgeId oda.dataSource extension id
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Enhance an Existing ODA Data Provider
DriverBridge extension implementsExtension Point
org.eclipse.datatools.connectivity.oda.consumer.driverBridge
Sample driverBridge extension element<bridge
driverType="org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver"bridgeId="org.eclipse.birt.report.data.testjdbc">
</bridge>
Bridge driver is handed its underlying driver instanceIDriver.setAppContext( Map )
Key: OdaDriver.ODA_BRIDGED_DRIVERValue: underlying driver instance
May use own API between Bridge and underlying Driver instances Sample code attached in Bugzilla 135246
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
ODA Data Providers
Out-of-the-Box ODA Data ProvidersCSV data file
org.eclipse.datatools.connectivity.oda.flatfile* plug-insXML data
org.eclipse.datatools.enablement.oda.xml* plug-insWeb Services
org.eclipse.datatools.enablement.oda.ws* plug-insJDBC Drivers – SQL Query Builder & SQL Textual Editor
org.eclipse.birt.report.data.oda.jdbc* plug-ins
DTP Incubation ProjectECore ODA Data Provider (Bugzilla 132958)
org.eclipse.datatools.enablement.oda.ecore* plug-insSeparate incubator download package
Welcomes contribution
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
What is New in ODA – DTP 1.7 Release
ODA API EnhancementsJava Object data typeAvailability of Data Set Property and Input Parameter Values prior to Query PreparationCancel a Query ExecutionConnection-level Locale SettingConsumer Resource Identifiers
http://wiki.eclipse.org/Galileo_DTP_ODA_API_Changes_Spec
Dynamic Query Result Specification API – Experimental APIFiltering Result ProjectionRow Ordering
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Resources
ODA Overview documentlink on DTP Connectivity home page
http://wiki.eclipse.org/Connectivity
Developers' Guide for Connectivity FrameworksDTP Help Contents – Javadoc™ and Extension Points’ SchemaPresentation slides
https://www.eclipsecon.org/submissions/2009/view_talk.php?id=686
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Open Data Access Framework
Discussions, Feedbacks, Q&As
© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009
Using and Extending the Data Tools Platform
SQL Query Model and Parser
Brian Payton (IBM)
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Agenda
Introduction and backgroundCharacteristicsWhat is it good for?Tour of the Query Model and Parser components
Query ModelQuery ParserParser FactoryPost-parse ProcessorsSQL Source WriterQuery Model Helpers
Extending the Query Model and ParserFun with the interactive tester
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Introduction and background
The Data Studio Query Model and Parser is a system for representing, analyzing, manipulating, and generating SQL query statements in client-side database applications.
Originally created for the SQL Assist component of IBM® DB2 Control Center, later used in IBM® WebSphere® Studio Application Developer, IBM Rational® Application Developer, and IBM Data Studio Developer.
Donated by IBM to the Eclipse Data Tool Platform project 2006 and used in the DTP SQL Query Builder component.
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Characteristics
Deep coverage of SQL DML statements (Query, Insert, Update, Delete)But DML only!Extensible, componentized system
Extension points to register extensions for different databasesParser grammar inheritance
Model is based on EMF technologyParser is based on LPG (LALR Parser Generator) technology
SourceForge project, but available in Eclipse through Orbit
Think of the model as an SQL AST (Abstract Syntax Tree) on steriods!
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
What is it good for?
Provides underlying support for the SQL Query BuilderMakes “round-trip” editing possible
Can do SQL Syntax validation outside of the databaseEnables query analysis
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Tour of the Query Model and Parser components
Query ModelRepresent the essential structure of a query statement
Query ParserValidate SQL source syntax and generate the query model
Parser FactorySet of methods used by the parser to generate the model
Post-parse ProcessorsValidate semantics and modify the model after parsing
SQL Source WriterGenerate SQL source from the model
Model HelpersSet of helpful methods to analyze or modify the model
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Query ModelRepresents the essential syntactic structure of a query statement as an object model
Covers DML statements
SELECT, INSERT, UPDATE, DELETE
Implemented using EMF (Eclipse Modeling Facility)
Derived from object model diagrams created using Rational Rose
Extends and complements the SQL ModelSQL Model is the foundation for the Data Studio tool setSQL Model models the database catalog while the Query Model models database statements
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Query Model - plugins
org.eclipse.datatools.modelbase.sql.queryBase query model
org.eclipse.datatools.modelbase.sql.xml.querySQL/XML extension
Depends on:org.eclipse.datatools.modelbase.sql
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Query Model – EMF Model
Java code is generated directly from the Rose model using EMF (Eclipse Modeling Facility)
Each Rose diagram class becomes a Java class
Model classes extend EMF EObject class
EMF handles details of getters, setters, collections, notifications
Each Rose diagram class becomes two EMF classes: an interface class and an implementation (Impl) class
Generated code can be customized, and changes won't be lost when regenerated
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Query Parser
Parses a SQL DML statement and generates an instance of the Query Model
Implemented using LPG (LALR Parser Generator) from IBM Research
Parser is generated from BNF-like grammar files
Like the Query Model, is actually an extendable set of parsers: base SQL, SQL/XML
Base SQL query grammar derived from ISO SQL 2003
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Parser Factory
Set of methods used by the parser to create a Query Model instance
Each parser plug-in has a parser factory class
Used in the “action” clauses of the LPG grammar file rules
In turn the parser factory methods call model factory methods tocreate individual model elements
Model factory code is generated by EMF, parser factory methods are hand-coded
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Post-parse Processors
Query Parser provides a mechanism to allow clients to modify a generated Query Model after the Parser Factory creates it but before it is returned as a parse result“Post-parse processors” are attached to the parser, register what model elements they are interested in, and after the parse get achance to modify the modelTwo default post-parse processors are provided:
DataTypeResolverTries to determine datatypes of value expressions in the model using available information
TableReferenceResolverReplaces place-holder table references in generated model with table objects from the SQL Model
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
SQL Source Writer
Generates SQL source from a Query Model instanceTogether with the Query Parser, provides “round-trip” capability from SQL to model and backLike the model and parser, is configurable and extendableWalks the model tree and uses reflection to find the right source writer method for each model element
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Query Model Helpers
Query Model Helpers are a set of classes and methods that provide useful utility functions for the Query Model
DataTypeHelperJoinHelperStatementHelperTableHelperValueExpressionHelper
Used both for analyzing and manipulating the model. Examples: Find information, such as all table references or all effective result columnsModify the model, such as remove a table and all column references for that table from a query
Packaged in org.eclipse.datatools.modelbase.sql.query
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Extending the Query Model and Parser
Process:Analyze the grammar feature you want to add. Is it specific to one database type, or common to several?Modify the appropriate to add the new feature
Getting this right is not easy! Modeling is hard work.Regenerate the EMF modelAdd Parser Factory methods for the new model elementsModify the appropriate parser grammar to cover the new syntaxUpdate the SQL Source Writer to generate SQL for the new syntaxUpdate the Query Model Helper classes as neededUpdate the JUnit tests for the parserMake sure it all works!
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Fun with the interactive tester
Along with JUnit tests, the test plugins for the Query Parsers include an interactive testing tool
In org.eclipse.datatools.sqltools.parsers.sql, it's SQLQueryParserInteractiveTest
A command-line tool, works with the Eclipse console viewType in a SQL statement, and it displays the generated model elements, then the SQL generated back out from the model
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
DTP Adopters
Eclipse ProjectsBusiness Intelligence and Reporting Tools (BIRT)
SQL Query BuilderDatabase EnablementConnection Profile ManagementOpen Data Access
http://www.eclipse.org/birtWeb Tools Project (WTP) – EJB or Dali JPA Tools development
http://www.eclipse.org/webtools
Other Adoptershttp://www.eclipse.org/datatools/community.php
DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0
Legal Notices
Actuate is a registered trademark of Actuate Corporation and/or its affiliates in the U.S. and certain other countries.
IBM, Rational, Rational Rose, and WebSphere are registered trademarks of International Business Machines Corporation in the United States, other countries, or both.
Sybase and Adaptive Server are registered trademarks of Sybase, Inc. or its subsidiaries.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.