DATABASE MANAGEMENT & IMPLEMENTATION Distributed Database Systems 1

DATABASE MANAGEMENT & IMPLEMENTATION Distributed Database Systems

1

Distributed Distributed Database Database SystemsSystems


2

Distributed Database Systems: What we will cover ?Distributed Database Systems: What we will cover ?

Architecture of a distributed database systems

Date’s Rules for DDBS

Tradeoffs in distributing the database

Distributed Oracle: An overview


3

Recommended ReferencesRecommended References

Date C. J. (1995)An Introduction to Database Systems. Volume 1, 6th Edition

Addison Wesley

Elmasri R. & Navathe S. B. (1994)Fundamentals of Database Systems, Second Edition

Benjamin Cummings

Korth HF & Silberschatz A (1991)Database System Concepts Second Edition

McGraw-Hill

Date,C.J.(1990)Relational Database: Writings 1985 - 1989, Chapter 10

Addison-Wesley

Oszu,M.,Valduriez,P.(1991) Principles of Distributed Database Systems Prentice Hall


4

Reasons for Data DistributionReasons for Data Distribution

Centralized DBMS vs. Distributed Database System

A distributed database is a collection of data that belongs logically to the same system but is physically spread over the sites of a computer network

Several factors have led to the development of DDBS:

Distributed nature of some database applications

Increased reliability and availability

Allowing data sharing while maintaining some measure of local control

Improved performance


5

Additional Functionality of DDBSAdditional Functionality of DDBS

Distribution leads to increased complexity in the system design and implementation

DDBMS must be able to provide additional functions to those of a centralized DBMS. Some of these are: To access remote sites and transmit queries and data among the various

sites via a communication network. To keep track of the data distribution and replication in the DDBMS catalog. To devise execution strategies for queries and transactions that access data

from more than one site. To decide on which copy of a replicated data item to access. To maintain the consistency of copies of a replicated data item. To maintain the global conceptual schema of the distributed database To recover from individual site crashes and from new types of failures such as

failure of a communication link.


6

DBMS Implementation AlternativesDBMS Implementation Alternatives

Distribution

Logically integratedand homogeneousmultiple DBMSs

DistributedheterogeneousDBMS

Distributedheterogeneousfederated DBMS

Heterogeneity

Heterogeneousintegrated DBMS Single site

heterogeneousfederated DBMS

Heterogeneousmultidatabasesystem

Distributedheterogeneousmultidatabasesystem

Autonomy

Multidatabasesystem

Distributedmultidatabasesystem

Distributedfederated DBMS

Distributedhomogeneous DBMS


7

Physical Architecture of DDBSPhysical Architecture of DDBS

Site 1

DP

AP

Site N

DP

AP

Site 3

DP

Site 2

AP

....

Communication network

Back-endmachine

Front-endmachine


8

Possible Network TopologiesPossible Network Topologies

A B

F

E D

C

Fully connected network

A

B

FE D

C

Partially connected network

AB

F

E

DC

Star networkA

B

F

E D

C

Ring network

A

B

F

E

DC

Tree-structured network


9

Components of Distributed DBMSComponents of Distributed DBMS

USER

User Interface Handler

Semantic Data Controller

Global Query Optimiser

Global Execution Monitor

ExternalSchema

GlobalConceptual Schema

GD/D

SystemResponses

UserRequests

AP

LocalConceptual Schema

Local Internal Schema

System Log

Runtime Support Processor

Local Recovery Manager

Local Query Processor

Database

DP


10

Date’s 12 Rules for Distributed SystemsDate’s 12 Rules for Distributed Systems

Rule 0. TO THE USER, A DISTRIBUTED SYSTEM SHOULD LOOK EXACTLY LIKE A NONDISTRIBUTED SYSTEM

1. Local autonomy 2. No reliance on a central site 3. Continuous operation 4. Location independence5. Fragmentation independence6. Replication independence7. Distributed query processing8. Distributed transaction management9. Hardware independence10. Operating system independence11. Network independence12. DBMS independence


11

Rule 1: Local AutonomyRule 1: Local Autonomy

Autonomy objective: Sites should be autonomous to the maximum extent possible

Local data is locally owned and managed, with local accountability security considerations integrity considerations

Local operations remain purely local All operations at a given site are controlled by that site; no site X

should depend on some other site Y for its successful functioning In some situations some slight loss of autonomy is inevitable

fragmentation problem - Rule 5 replication problem - Rule 6 update of replicated relation - Rule 6 multiple-site integrity constraint problem - Rule 7 a problem of participation in a two-phase commit process - Rule 8


12

Rule 2: No Reliance on a Central SiteRule 2: No Reliance on a Central Site

There must not be any reliance on a central "master" site for some central service, such as centralized query processing or centralized transaction management, such that the entire system is dependent on that central site

Reliance on a central site would be undesirable for at least the following two reasons: that central site might be a bottleneck the system would be vulnerable

In a distributed system, therefore, the following functions (among others) must all be distributed: Dictionary management Query processing Concurrency control Recovery control


13

Rule 3: Continuous OperationRule 3: Continuous Operation

There should ideally never be any need for a planned entire system shutdown

Incorporating a new site X into an existing distributed system D should not bring the entire system to a halt

Incorporating a new site X into an existing distributed system D should not require any changes to existing user programs or terminal activities

Removing an existing site X from the distributed system should not cause any unnecessary interruptions in service

Within the distributed system, it should be possible to create and destroy fragments and replicas of fragments dynamically

It should be possible to upgrade the DBMS at any given component site to a newer release without taking the entire system down


14

Rule 4: Location Independence(Transparency)Rule 4: Location Independence(Transparency)

Users should not have to know where data is physically stored, but rather should be able to behave - at least from a logical standpoint - as if the data was all stored at their own local site

Simplifies user programs and terminal activities Allows data to migrate from site to site It is easier to provide location independence for simple retrieval

operations than it is for update operations Distributed data naming scheme and corresponding support from

the dictionary subsystem User naming scheme

User U has to have a valid logon ID at each of multiple sites to operate User profile for each valid logon ID in the dictionary Granting of access privileges at each component site


15

Rule 5: Fragmentation Independence (Transparency)Rule 5: Fragmentation Independence (Transparency)

A distributed system supports data fragmentation if a given relation can be divided up into pieces or "fragments" for physical storage purposes

A system that supports data fragmentation should also support fragmentation independence (also known as fragmentation transparency)

Fragmentation is desirable for performance reasons Horizontal fragmentation SELECT Vertical fragmentation PROJECT Fragmentation must be defined within the context of a distributed

database Fragmentation independence (like location independence) is desirable

because it simplifies user programs and terminal activities Fragmentation independence implies that users should normally be

presented with a view of the data in which the fragments are logically combined together by means of suitable joins and unions

Users should be able to behave (at least from a logical standpoint) as if the data were in fact not fragmented at all


16

Rule 5: An Example of FragmentationRule 5: An Example of Fragmentation

EMP # DEPT # SALARY

E1 DX 45K E2 DY 40K E3 DZ 50K E4 DY 63K E5 DZ 40K

EmployeeUser Perception

EMP # DEPT # SALARY

E1 DX 45K E3 DZ 50K E5 DZ 40K

EMP # DEPT # SALARY

E2 DY 40K E4 DY 63K

physical storage New York

physical storage London

New York fragment London fragment


17

Rule 6: Replication Independence (Transparency)Rule 6: Replication Independence (Transparency)

User should be able to behave as if the data were in fact not replicated at all

A distributed system supports data replication if a given relation (more generally, a given fragment of a relation) can be represented at the physical level by many distinct stored copies or replicas, at many distinct sites.

Replication, like fragmentation, should be “transparent to the user” Replication is desirable for at least two reasons:

Performance Availability

Update propagation problem Replication independence (like location and fragmentation independence)

is desirable because it simplifies user programs and terminal activities Snapshots


18

Rule 6: Example of Data Replication Rule 6: Example of Data Replication

EMP # DEPT # SALARY

E1 DX 45K E2 DY 40K E3 DZ 50K E4 DY 63K E5 DZ 40K

EmployeeUser Perception

EMP # DEPT # SALARY


EMP # DEPT # SALARY

E2 DY 40K E4 DY 63K

physical storage New York

physical storage London

New York fragment

London fragment

EMP # DEPT # SALARY


Replica ofNew Yorkfragment

EMP # DEPT # SALARY

E2 DY 40K E4 DY 63K

Replica ofLondonfragment


19

Rule 7: Distributed Query ProcessingRule 7: Distributed Query Processing

It is crucially important for distributed database systems to choose a good strategy for distributed query processing

Query processing in a distributed system involve local CPU and I/O activity at several distinct sites some amount of data communication among those sites

Amount of data communication is a major performance factor

Query compilation ahead of time Views that span multiple sites Integrity constraints within a DDBS that span multiple

sites

Ry

Rz

Query Q

300 records

2 Mrecords


20

Rule 8: Distributed Transaction ManagementRule 8: Distributed Transaction Management

Two major aspects of transaction management, recovery control and concurrency control, require extended treatment in the distributed environment

In a distributed system, a single transaction can involve the execution of code at multiple sites and can thus involve updates at multiple sites

Each transaction is therefore said to consist of multiple "agents," where an agent is the process performed on behalf of a given transaction at a given site

Global deadlock: neither site can detect it using only information that is internal to that site

Site X

Site Y

T1x T2x

T1y T2y

holds lock Lx

holds lock Ly

wait for T2yto release Ly

wait for T1xto release Lx

wait forT1y to complete

wait forT2x to complete


21

Rule 9: Hardware Independence(Transparency)Rule 9: Hardware Independence(Transparency)

It is desirable to be able to run the same DBMS on different hardware systems

It is desirable to have those different hardware systems all participate as equal

partners (where appropriate) in a distributed system

The strict homogeneity assumption is not relaxed; it is still assumed that the same

DBMS is running on all those different hardware systems

User should be presented with the “single-system image” regardless any particular hardware platform


22

Rule 10: Operating System IndependenceRule 10: Operating System Independence

From a commercial point of view, the most important operating system environments, and hence the ones that (at a minimum) the DBMS should support, are probably MVS/XA, MVS/ESA, VM/CMS, VAX/VMS, UNIX (various flavors), OS/2, MS/DOS, Windows

It is obviously desirable, not only to be able to run the same DBMS on different hardware systems, but also to be able to run it on different operating systems - even different operating systems on the same hardware


23

Rule 11: Network IndependenceRule 11: Network Independence

From the point of view of the distributed DBMS, the network is merely the provider of a reliable message transmission service

By "reliable" here is meant that, if the network accepts a message from site X for delivery to site Y, then it will eventually deliver that message to site Y;

Messages will not be garbled, will not be delivered more than once, and will be delivered in the order sent.

The network should also be responsible for site authentication Ideally the system should support both local area networks and wide-area

networks Distributed system should support a variety of different network architectures

It is obviously desirable to be able to support a variety of disparate communication networks


24

Rule 12: DBMS IndependenceRule 12: DBMS Independence

Ideal distributed system should provide DVBMS independence (or transparency)

INGRES user

INGRES INGRES/ (SQL) STAR

INGRESdatabase

ORACLEdatabase

ORACLE (SQL)

GATE WAY

distributed INGRES database


25

Distributed DB System With OracleDistributed DB System With Oracle

An Oracle distributed system can be a blend of distributed database and distributed processing systems.


26

Oracle: Cooperative Server ArchitectureOracle: Cooperative Server Architecture

Note also that the INSERT statement includes the location of the database to be accessed (SALES) because the client from which the SQL statement is issued is connected directly to the HQ database server only.


27

Oracle SQL Statement ExecutionOracle SQL Statement Execution

SQL statement execution of remote and distributed statements is essentially the same in a distributed system as in a non–distributed database.

All remote queries and remote updates are sent to the remote node for statement execution. The remote node executes the statement and returns any results back to the local node, which returns them to the user or application.

The statements in a procedure or trigger that constitute a distributed update are sent individually to the correct remote node for execution. Results are returned to the local node and then to the calling user or application.

Optimization of SQL Statements in a Distributed System. The optimization approach available for SQL statements can vary depending on the type and complexity of the statement.


28

Oracle: Network Product ArchitectureOracle: Network Product Architecture


29

Oracle Network Products &

Distributed Systems

Oracle Network Products &

Distributed Systems

A Possible distributed systems scenario


30

SQL*Net and Network Environment IndependenceSQL*Net and Network Environment Independence

SQL*Net runs on most networks. Media/Topology Independence. SQL*Net supports most third-party

network software packages, which in turn, support a wide variety of network hardware devices. On some platforms, a single Oracle Protocol Adapter can operate on hundreds of different network interface cards.

When a request for a connection is made successfully, SQL*Net passes control of the connection to the underlying protocol. At that point, all media and/or topologies supported by the underlying network protocol are indirectly inherited by SQL*Net. SQL*Net allows the network protocol to use any means of data transmission, such as Ethernet, Token Ring, FDDI, or SDLC, to accomplish the low-level data link transmission between the two computers.

Protocol Independence. An application using SQL*Net can run over any network protocol.

In addition to supporting different protocols, SQL*Net also supports many vendor's protocol stacks.


31

The MultiProtocol InterchangeThe MultiProtocol Interchange

Oracle's client-server and server-to-server models provide connectivity across multiple network protocols.

Oracle's MultiProtocol Interchange can be used with SQL*Net to enable transparent data access across protocols, allowing a client using one protocol to communicate with a server using a different protocol.

Applications simply ask for services by name, and SQL*Net automatically calculates the most efficient route to take through the network and establishes the connection.

With SQL*Net version 2, the client and server can belong to different communities connected by one or more MultiProtocol Interchange(s).

In a server-to-server configuration, this same heterogeneous network capability is extended to include database-to-database communications. Two types of server-to-server connections are possible using SQL*Net: a direct connection between two servers in the same community a connection between servers in different communities through one or more

MultiProtocol Interchanges


32

Heterogeneous Networking with Client-Server ConnectionHeterogeneous Networking with Client-Server Connection

The Interchange has adapters for both Protocol A and Protocol B. In a sense, it is bilingual (or poly-lingual). When communication is requested between the two communities, the MultiProtocol Interchange translates between the two protocols.


33

MultiProtocol Interchanges in Server-to-Server ConfigurationMultiProtocol Interchanges in Server-to-Server Configuration

Server 1 is a member of two communities, Community A and Community B. A client application in Community A accesses the database server (Server 1) within the same community. Server 1 determines that the transaction must be distributed further to retrieve data from tables in Server 2 and Server 3. Server 1 initiates a connection to Server 2 in Community B to which Server 1 also belongs. Server 1 also initiates a connection to Server 3 through the MultiProtocol Interchange installed between Community B and Community C.


34

SQL*Net Architecture (1)SQL*Net Architecture (1)

Transparent Network Substrate SQL*Net's communication role Distributed processing with SQL*Net SQL*Net operations SQL*Net and the network listener


35


Transparent Network Substrate (TNS) enables Oracle to provide a network of applications above all existing networks of computers. With TNS, peer-to-peer application connectivity is possible where no direct machine-level connectivity exists. Peer-to-peer is an architecture in which two or more nodes can communicate with each other directly, without the need for any intermediary devices. In a peer-to-peer system, a node can be both a client and a server.

TNS provides two key features to a TNS-based network product and, in turn, any application built using TNS: a single, common interface to all industry-standard protocols the ability to connect to applications in physically separate networks through

one or more MultiProtocol Interchanges In a distributed transaction, SQL*Net is responsible for sending

information across various networks on behalf of a client application or database server.

The Oracle client and server exchange SQL statements and data rows.


36


SQL*Net is responsible for enabling communications between the cooperating partners in a distributed transaction, either client-server or server-to-server. Specifically, SQL*Net enables clients and servers to connect to each other, send data such as SQL statements and data responses, initiate interrupts from the client or server, and disconnect when the session is complete. During the life of the connection, SQL*Net resolves all differences between the internal data representations and/or character sets of the computers being used.


37


When two servers are communicating to complete a distributed transaction, the process and dialogues are the same as in the client-server scenario, except that there is no client application.

The server has its own version of UPI, called NPI. The NPI interface can perform all of the functions that the UPI does for clients, allowing a coordinating server to construct SQL requests for additional servers.


38


SQL*Net provides the following functions connect operations

connect to server (open) disconnect from server (close)

data operations send data synchronously receive data synchronously send data asynchronously receive data asynchronously

exception operations initiate a break over the TNS connection reset a connection for synchronization after a break test the condition of the connection for incoming break


39


TNS includes a protocol independent application listener that receives connections on behalf of any TNS application, over any underlying protocol. Referred to as a network listener, it runs as a single process or task and can service the needs of all TNS applications over all protocols available on a machine.


40

Network Directories and Global Database NamesNetwork Directories and Global Database Names


41

Network Issues to Consider (1)Network Issues to Consider (1)

Planning your Network When planning your SQL*Net network, think about future needs as well as

present requirements. Draw the Network Layout

It is a good idea to draw a picture of your network layout as you decide about its composition. Two types of diagrams are useful: physical diagrams logical diagrams

Select Network Protocols to be Used The first decision to make when designing a network is whether it will include

only one protocol or more than one protocol.


42


Choose Nodes as Interchanges If you decide on a multicommunity network, you must choose what nodes to

use for Interchanges to connect the communities. Choose a Node to Run Network Manager Decide on the Structure of Network Administration Decide which Nodes will Run Oracle Names Servers

If you use Oracle Names to provide a centralized naming service for your network, you must decide what nodes should contain Names servers, which provide name and address information to enable connections throughout the network.

Decide on a Organizational Naming Standard When you name entities such as nodes in a networked environment, you

should ensure that object names are unique within the network. This can be a challenge if your organization is large, and network administration is not handled centrally.


43


Selecting Domain Names A recommended network naming

technique is to use hierarchical groups or domains in which each administrative unit is assigned to a unique domain based on the function it provides.

Default Domain Every client and server has a default

domain listed in its SQLNET.ORA file. The default domain is the domain to which most of the clients' connection requests are directed. The service names of databases in the default domain do not need to be fully qualified; that is, the domain name does not need to be included.


44

Network Administration UtilitiesNetwork Administration Utilities

SQL*Net has administrative, performance, and security related utilities, making it even easier to use in large environments. Some of these utilities are: password encryption client-side load balancing connection route testing with ping SNMP support diagnostic logging and tracing capabilities

Network Manager Oracle Network Manager is a utility that assists you in creating the

configuration files needed for Oracle networking products. It also provides graphical views of your network.


45

DDBS: Query Processing and OptimizationDDBS: Query Processing and Optimization

From Date V2

SUPPLIER (SUPPLIER#,CITY) 10,000 RECORDS STORED IN DETROITPART(PART#,COLOR) 100,000 RECORDS STORED IN CHICAGOSHIPMENT (SUPPLIER#,PART#) 1M RECORDS STORED IN DETROIT

SELECT SUPPLIER.SUPPLIER#FROM SUPPLIER,SHIPMENT,PARTWHERE SUPPLIER.CITY = ‘CLEVELAND’AND SUPPLIER.SUPPLIER# = SHIPMENT.SUPPLIER#AND SHIPMENT.PART# = PART.PART#AND PART.COLOR = ‘RED’

100 character records - 10,000 character per second data rate1 second access daily to send a message


46

DDBS: Query OptimizationDDBS: Query Optimization

6 strategies

Move part to Detroit and process whole query at Detroit computer

Time to complete query = Access Delay + (Data Volume / Data Rate)= Number of Messages * 1 + 100,000 part records * 100 / 10,000= 1 + 1,000 = 1,001 seconds

= 16.67 minutes

Strategies range from 1 sec Select just red parts at Chicago to 2.3 days join supplier and shipment at Detroit


47

DDBS: Global Transaction ManagementDDBS: Global Transaction Management

Goals of Transaction Management in Distributed DBMS resiliency efficiency (message minimization) correctness

Global Transaction a logical concept

Local Transaction the physical subtransactions of a global transaction

Example ITEMS A B NODES N1 T1 = A - 10 N2, N3 N4,N5 B + 10 Local Tx to N1,N2,N3,N4,N5


48

DDBS: Distributed LockingDDBS: Distributed Locking

Goals of Distributed Locking efficient and correct

1 Logical Lock = n(Physical Locks)

Problems - convert logical locks to physical locks - choose a strategy which reduces communication

Locking Strategies

Writes Locks All - Reads Lock 1 Majority Locking Primary Site Primary Copy


49

DDBS: Comparison of Locking StrategiesDDBS: Comparison of Locking Strategies

Method Control Msgs Control Msgs Comment to Write to Read Write Locks All 2n 1 Good if reads dominate

Majority Locking >= n + 1 >= n Avoids some deadlock

Primary Site 2 1 Vulnerable to Crash

Primary Copy 2 1 Removes traffic from Central Site


51

DDBS: Distributed CommitmentDDBS: Distributed Commitment

TWO-PHASE COMMIT Required when more than one resource manager is involved in the COMMIT

process, i.e Database Manager and Communication Manager All these resource manager have separate logs and independent COMMIT

processes One Resource Manager is the Coordinator and the others are Participants Tx issues global commit to the coordinator not the the individual participants

Phase 1 . Coordinator requests each participant to get ready to commit byforcing changes to the local log. If successful they respond OK.

Phase 2. If all responses are OK. Coordinator broadcasts COMMIT to allparticipants. COMMIT now forced to the local log. If any absent or NOK responses then coordinator broadcasts rollback to all participants


53

Distributed Database Systems: ConclusionsDistributed Database Systems: Conclusions

Architecture of a distributed database systems

Date’s Rules for DDBS

Tradeoffs in distributing the database

Advantages

Disadvantages

Problems of distributed database systems

Distributed Oracle: An overview


54

Distributed Database Systems: KeywordsDistributed Database Systems: Keywords

data distribution data replication global failure DBMS catalog global conceptual schema site crash external schema local conceptual schema local internal schema Rule 0 local autonomy no reliance on central site continuous operation location transparency fragmentation independence

replication independence distributed transaction

manager hardware independence OS independence network independence DBMS independence Oracle/Net createdb distributed locking distributed commitment recovery of nodes


55

What’s next ? - ReplicationWhat’s next ? - Replication

Basic Architectural Models Classification of Replica Control Strategies

Consistency & Request Ordering The Gossip architecture Process Groups and ISIS Replication in Oracle

Documents

DATABASE MANAGEMENT & IMPLEMENTATION Distributed Database Systems 1