Upload
byron-mccarthy
View
224
Download
3
Embed Size (px)
Citation preview
B AC K G R O U N D
OVERVIEW OF RELATIONAL DBS
2
BASICS
• Highly structured• Schema based - we can leverage this to address volume
• Semantics• SQL• App• Middleware• Users
• Structure• Tables/relations, rows/tuples, columns/attributes• User defined data types• PKs and FKs• Null or not null• Triggers as a catch-all integrity constraint• Normalization for formal table minimization
• Uses• Bank checks• Insurance claims• Credit card payments
3
NOSQL
• Cluster based broad distribution• Semi structured• More flexible access of data• Hierarchical• Similar structure
4
RELATIONAL DBS:FORMALLY UNDERSTOOD
• Set theoretic
•Originally defined with an algebra, with Selection, Projection, Join, and Union/Difference/Intersection
•Declarative calculus that is based on the algebra and supports large grained queries
• Clean implementation spec
• Unambiguous optimization - with its own algebra of query parse tree transformations
5
SEMANTICS ARE IN QUERIES
• Relational algebra compliant
•Queries written in declarative calculus
• Set-oriented
• But at least Programmers tend to follow PK/FK pairs, and infer semantics from attribute names and associations in tuples
•Query results are legal tables (Views)
6
ALSO WE GET(GOOD AND BAD)
• Fixed size tuples for easy row-optimization
• 2P transactions
• Table, Row distribution
• Two language based, with lowest common denominator semantics
• Security
• Checkpointing
• Powerful query optimizers
7
OBJECT-RELATIONAL DBS
• This runs somewhat counter to NoSQL trends - we make the data types even more complex
•We make domains out of type constructors
•Object IDs
• A row can be a tuple - or an object, with an object ID and a tuple, making all relational DBs also O-R
8
OBJECT-ORIENTED DBS
• No tuple rows
• Blend SQL and the app language
• This avoids lowest common denominator semantics
• These bombed, as relational DBs were not O-O
• And they are tough to optimize
THE RELATIONAL ALGEBRA AND CALCULUS: THE HEART OF
RELATIONAL DBS
… S Q L
10
THE BIG 3:
• Selection and projection are unary ops• Join is binary• Selection is based on a formula and returns a
table that contains all tuples from a given table where the formula is valid• Projection returns a table consisting of a subset of
attributes from a given table, with dupes removed• Join creates tuples with attributes from two given
tables, where a specific attribute in one matches a specific attribute in another (often a PK, FK pair)
11
ALGEBRAIC CLOSURE
• Any relational algebra operation returns a legal derived table• The set operators are also part of the algebra• From a formal perspective, the join operator is not
a minimal operator, and is therefore represented as a cross product followed by a selection (where the PK equals the FK)• Note that joins are symmetric
12
JOINS CAN BE GENERALIZED
• Complex join conditions• Non-equi joins• A “natural” join is based on matching all
attributes with equal names in both tables• “Outer” join creates null-packed tuples when
tuples on the left do not match any on the right; there is also a right outer join
13
THE CALCULUS
• It is a tuple calculus, not a domain calculus• SQL is equivalent• Select From Where• The part after the Where is declarative• A tuple calculus (SQL) • Notice that the variables are indeed tuples• Note that set operators often act on tables that
are being created in the query
SQL
CREATE PROCEDURE test()BEGINDECLARE sql_error TINYINT DEFAULT FALSE;DECLARE CONTINUE HANDLER FOR SQLEXCEPTIONSET sql_error = TRUE;START TRANSACTION; INSERT INTO invoicesVALUES (115, 34, 'ZXA-080', '2012-01-18', 14092.59, 0, 0, 3, '2012-04-18', NULL); INSERT INTO invoice_line_items VALUES (115, 1, 160, 4447.23, 'HW upgrade');INSERT INTO invoice_line_items VALUES (115, 2, 167, 9645.36, 'OS upgrade');IF sql_error = FALSE THENCOMMIT;SELECT 'The transaction was committed.';ELSEROLLBACK;SELECT 'The transaction was rolled back.';END IF;END//
15
MORE
• IN operator is “element of”• EXISTS • Nesting• FOR ALL• FOR SOME• Putting computations in the SELECT clause• COUNT, SUM, AVG, MAX, MIN operators
STORED PROGRAMS
• Stored procedures (can be called by an application)• Stored functions (can be called by an SQL
program)• Triggers (tied to an operation like INSERT)• Events (tied to a clock)
VARIABLES
• DECLARE statement• SET statement• DEFAULT statement• INTO (from a SELECT clause)
EXAMPLE…
CREATE PROCEDURE test()BEGIN DECLARE max_invoice_total DECIMAL(9,2); DECLARE min_invoice_total DECIMAL(9,2); DECLARE percent_difference DECIMAL(9,4); DECLARE count_invoice_id INT; DECLARE vendor_id_var INT; SET vendor_id_var = 95; SELECT MAX(invoice_total), MIN(invoice_total), COUNT(invoice_id) INTO max_invoice_total, min_invoice_total, count_invoice_id FROM invoices WHERE vendor_id = vendor_id_var;
EXAMPLE, CONTINUED
SET percent_difference = (max_invoice_total - min_invoice_total) / min_invoice_total * 100; SELECT CONCAT('$', max_invoice_total) AS 'Maximum invoice', CONCAT('$', min_invoice_total) AS 'Minimum invoice', CONCAT('%', ROUND(percent_difference, 2)) AS 'Percent difference', count_invoice_id AS 'Number of invoices';END//
DOMAIN TYPES – CHAPTER 8
• Character• Integers • Reals• Date• Time• Large object, BLOB and CLOB• 2D vector spatial types• Enumerated
THE ACID PROPERTIES, NORMALIZATION, AND DATABASE DESIGN
ACID TRANSACTIONS
• Atomic: Either all of a transaction or None of it affects the database• Consistent: When a transaction ends, the
database obeys all constraints• Isolated: Two running transactions cannot pass
values to each other, via the database or other data store• Durable: Once a transaction has “committed”, its
updates are permanent
ATOMICITY
• Use a local log to store a transaction’s partial result• If a transaction does something illegal, toss out
the log
CONSISTENT
• Check constraints in phase 1• Some are immediate, like domains• Others don’t have to be true until the commit point, like
FKs
ISOLATED
• Transactions commit in a linear order • Serializability is enforced• Results become available only after atomic
commit point
DURABLE
• Database has one state and it is in nonvolatile storage• Keep checkpoints and transaction logs
DEADLOCK
• Loops of transactions wait on each other• Detection: use time-outs• Prevention: use “waits for” graph
STORED PROGRAMS
• Stored procedures (can be called by an application)• Stored functions (can be called by an SQL
program)• Triggers (tied to an operation like INSERT)• Events (tied to a clock)
VARIABLES
• DECLARE statement• SET statement• DEFAULT statement• INTO (from a SELECT clause)
ANOTHER VIEW OF TRANSACTIONS
• Prevents• Lost updates from one of two transactions• Dirty reads when a transaction reads an uncommitted
value• Nonrepeatable reads in one transaction because the
value gets updated in between• Phantom reads are when a subset of updated rows are
simultaneously updated by another transaction
CONTINUED…
• Options• Serializable isolates transactions completely and is the
highest level of protection• Read uncommitted lets our four problems occur – no
locks• Read committed prevents dirty reads• Repeatable read is the default and it means that a
transaction will always read a given value the same because the values are locked
DEADLOCK
• Detect by closing transactions that have been open a long time• Use the lowest acceptable locking level• Try to do heavy update transactions when
database can be completely reserved
STORED PROGRAMS
• Stored procedures (can be called by an application)• Stored functions (can be called by an SQL
program)• Triggers (tied to an operation like INSERT)• Events (tied to a clock)
THE DB DESIGN PROCESS
• Start with an entity model• Map to tables• Create PKs and FKs• Create other constraints• Normalize tables
OUR FOCUS: NORMALIZATION
• Goals• Minimize redundant data• Minimize “update anomalies”
FUNCTIONAL AND MULTIVALUED DEPENDENCIES
• FD • We say that ai FD-> aj• Or “ai functionally determines aj”
• MVD->• We say that ai MVD-> aj• Or “ai multivalued determines aj”
• Note: the right side of an FD or an MVD can be a set of attributes
FIRST 3 NORMAL FORMS
• First (1NF) The value stored at the intersection of each row and column must be a scalar value, and a table must not contain any repeating columns.• Second (2NF) Every non-key column must
depend on the entire primary key.• Third (3NF) Every non-key column must depend
only on the primary key.
NF3 FIXED AND NF4
• Boyce-Codd (BCNF) A non-key column can’t be dependent on another non-key column. • Fourth (4NF) A table must not have more
than one multivalued dependency, where the primary key has a one-to-many relationship to non-key columns.
EXAMPLE: 1NF
EXAMPLE: 2NF
EXAMPLE: 2NF, CONTINUED
3NF: REMOVE TRANSITIVE DEPENDENCIES
Customer ID Address ZIP18 112 First 80304 17 123 Ash 80303 16 123 Ash 80303
3NF, CONTINUED
Break into two tables:
Customer ID AddressAddress Zip
4NF: SEPARATE PAIRS OF MVDS
Mothers_Phone Fathers_Phone Child_Name
Break into: Mothers_Phone Child_Name 3030000000 Sue 3031111111 SueAnd Fathers_Phone Child_Name 3032222222 Sue
3033333333 Sue
Note: both fields needed for PK
TRADEOFFS
• “Decomposition” makes it harder to misunderstand the database schema• But Decomposition create narrow tables that
might not correspond to forms in the real world• And Decomposition leads to extra joins• One solution is to pre-join data