Embed Size (px)
Citation preview
WEB BASED DATA TRANSFORMATIONUSING XML, JAVA
Group members:
Darius Balarashti &
Matt Smith
Idea
Web application that acts as an intermediary among several databases in exchanging data
• highly customizable
• dynamic
• platform independent
Background – technology used
XML SOAP HTTP Java Server Pages (JSP)
XML - eXtensible Markup Language
called extensible since it is not a fixed format e.g. HTML
XML is a ‘meta-language’ – a language for describing other languages- e.g. Wireless Markup Language (WML)
designed to describe any structured data “universal format for structured data on the
Web” – W3C web site
When should I use XML? When you need a buzzword in your resume.W3SCHOOLS.COM
XML vs. HTML
designed to describe data & focus on what data is
No predefined tags No inherit structure of
your tags used for data
manipulation and transfer
designed to display data & focus on how it looks
Predefined tags Inherit structure built into tags
e.g. <h1> used for displaying that same
data
XML is not a replacement for HTML
Simple Example
<?xml version="1.0"?> <CATALOG> <CD>
<TITLE>EmpireBurlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST> <COUNTRY>USA</COUNTRY> <COMPANY>Columbia</COMPANY> <PRICE>10.90</PRICE> <YEAR>1985</YEAR>
</CD> </CATALOG>
<html> <h2>Catalog</h2>
<body><table
width=“100%”><tr> <td>EmpireBurl</td> <td>Bob Dylan</td>
<td>USA</td></tr><tr> <td>Columbia</td> <td>10.90</td>
<td>1985</td></tr></table></body></html>
XML HTML
<?xml version="1.0"?> <CATALOG> <CD>
<TITLE>EmpireBurlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST> <COUNTRY>USA</COUNTRY> <COMPANY>Columbia</COMPANY> <PRICE>10.90</PRICE> <YEAR>1985</YEAR>
</CD> </CATALOG>
Advantages of XML
data exchange between incomparable systems– Software independent– Hardware independent– plain text files
• data is stored outside of HTML
• can code documents more precisely
- reflects structure and semantics of that document
Transforming XML
DTD – document type definition
- defines tags in XML
- number, sequence, attributes, and values of those tags
XSL – eXtensible Style Sheet
- browsers can’t display XML
- transforms XML into HTML CSS – Cascading Style Sheet
- less control than XSL
SOAP – Simple Object Access Protocol
- defines a framework for describing what is in a message & how to process it
- protocol for exchange of information decentralized environment
All SOAP messages encoded in XML
XML and HTTP based Protocol
Java Server Pages (JSP)
servlet = server side applet - Java’s answer to CGI - no GUI static HTML with dynamic content from servlets
and/or JavaBeans some Java advantages
– Platform independent– Can utilize Java API for XML Processing (JAXP),
Java API for XML Messaging (JAXM), and Simple Access API for XML (SAX).
XML & Java Application
XML document is parsed, data becomes available to application
DOM (Document Object Model)– represents elements as tree nodes– use if need random access to data
SAX (Simple API for XML)- fires events based on what it encounters- write code to make sense of these events
XML & Java
Java is portable code, XML portable data Applications completely portable Java provides most robust set of
- API’s
- processors
- parsers
Why XML & JSP ?
can use SAX
3 main steps
1. Create object model
2. Create parser
3. Create handler
Overall Approach Web app. that is intermediary between 2 databases•User can select source data and transfer it to different database(s)
• 2 distinct process
- configuration
- transformation
Third Party Server
User
XHTML THINCLIENT
Microsoft Internet Information Server
Source DB(Oracle,
MySQL, SQLServer) Target
(Destination)DBS
XML DATAHTTP Protocol HTTP Protocol
System ServerThird Party Server
Internet
Client Side
System Overview and Environments
System Overview
Configuration Process
User driven
- user selects source data
- user selects destination database User select transformation options, if any
- direct mapping of data
- string manipulations
- simple calculations
Configuration Process
Load Source Schema
Source Database
Target Database[0]
Target Database[1] Load Target Schema(s)
<<include>>
Set User Priveleges
Third Party Administrator
<<include>>
<<include>>
No Access Allowed
<<extend>>
Database not Available<<extend>>
Database not Available
<<extend>>
<<include>>
<<include>>
<<include>>
Configure Target Database(s)
Configure Source Database
Configure Data Mapping
Describe Transformations
User
<<include>>
Use Case Model
Example Interface
System driven - takes user specified configurations and performs actual data
transformation
How? - System sends SOAP request to the module controlling the source
database.- Module connects to database and receives the data tuple.- System sends data to the interpreter which transforms the data.- Data is sent to the module controlling the destination database.- Module loads the data into the database.
Transformation Process
Use Case ModelData Transformation Process
Initiate TransformationUser
Request/Receive Data ProcessTarget Database
<<include>>
<<include>>
<<include>>
Send Data Process Source Database
System Transform Data
<<include>>
Costs
Time
- done by April 2002 Money – can vary greatly
1. Software
- currently, none
2. Hardware
- none other than available on campus
Performance Requirements
• handle large databases 10,000+ records
• less than 1 sec per record
Constraints
Environmental
• Unknown bandwidth between system and source/destination
• Unknown database optimizations
System
• Bandwidth
• Processor power
• Code optimization
Constraints [cont’d]
User
• Access Privileges
• Security
Maintenance
• Basic code/server maintenance
• Updating database specific modules
• Optimization
Alternative Designs
1. Application server using Enterprise Java Beans Adv: any application environment, direct
transformations Dis: complex, expensive, unreliable
2. Microsoft .NET platform Adv: based on XML, any language Dis: documentation, Beta version
Testing Methods
User Interface TestingTesting the user interface against necessary functionality.
Unit TestingTesting the classes as individual components.
Development and implementation of test cases.
System TestingTesting the classes as components in the system.
Development and implementation of test scenarios.
Testing Methodology: Rational Unified Process
Scheduling Diagram [First Semester]
ID Task Name Start End DurationOct 2001Sep 2001 Nov 2001
11/49/30 10/28 11/189/9 9/23 10/14 11/11 11/2510/79/16
2 8d10/10/200110/1/2001Business Modeling Phase
5 20d11/7/200110/11/2001Requirements Phase
8 13d11/26/200111/8/2001Analysis and Design Phase
1 53d12/12/200110/1/2001First Semester Design
3 0d10/1/200110/1/2001MILESTONE: ProblemDefinition[1]
4 0d10/10/200110/10/2001MILESTONE: ProblemDefinition[2]
6 0d10/22/200110/22/2001MILESTONE: Req Analysis[1]
7 0d11/7/200111/7/2001MILESTONE: Req Analysis[2]
9 0d11/26/200111/26/2001MILESTONE: DesignSpecs[1]
ResourceName
Team
Team
Team
Team
Team
Team
Team
Team
Team
PercentComplete
30%
100%
100%
100%
50%
100%
0%
0%
0%
10/219/2
Dec 2001
12/2 12/9 12/16 12/23 12/30
Scheduling Diagram (Second Semester)
ID Task Name Start End DurationJan 2002 Feb 2002 Mar 2002
1/6 1/13 1/20 1/27 2/3 2/10 2/17 2/24 3/3 3/10
2 69d4/5/20021/1/2002Implementation Phase
6 20d5/3/20024/8/2002Testing Phase
1 102d5/22/20021/1/2002Second Semester Design
3 0d1/10/20021/10/2002MILESTONE: User InterfacePrototype
4 0d2/18/20022/18/2002MILESTONE: ConfigurationProcess Functionality
7 0d3/25/20023/25/2002MILESTONE: Developmentof Test Cases
ResourceName
Team
Team
Team
Team
Team
Team
PercentComplete
0%
0%
0%
0%
0%
0%
3/17 3/24
5 0%Team0d3/18/20023/18/2002MILESTONE: TransformationProcess Functionality
Apr 2002 May 2002
3/31 4/7 4/14 4/21 4/28 5/5 5/12 5/19 5/26
8 0%Team0d4/12/20024/12/2002MILESTONE: Verification ofTest Cases
9 0%Team8d4/25/20024/16/2002Deployment Phase
10 0%0d4/25/20024/25/2002MILESTONE: WorkingSystem Deployed
