subqueries - University of Washington · SUBQUERIES A subqueryis a SQL query nested inside a larger...

CSE 344JANUARY 17TH – SUBQUERIES

GROUPING AND AGGREGATIONPurchase(product, price, quantity)

SELECT product, Sum(quantity) AS TotalSalesFROM PurchaseWHERE price > 1GROUP BY product

How is this query processed?

Find total quantities for all sales over $1, by product.

SELECT product, Sum(quantity) AS TotalSalesFROM PurchaseGROUP BY product

Do these queries return the same number of rows? Why?

SELECT product, Sum(quantity) AS TotalSalesFROM PurchaseGROUP BY product

Do these queries return the same number of rows? Why?

Empty groups are removed, hencefirst query may return fewer groups

GROUPING AND AGGREGATION

CSE 344 - 2017au 5

1. Compute the FROM and WHERE clauses.

2. Group by the attributes in the GROUPBY

3. Compute the SELECT clause:grouped attributes and aggregates.

FWGSTM

1,2: FROM, WHERE

Product Price QuantityBagel 3 20Bagel 1.50 20

Banana 0.5 50Banana 2 10Banana 4 10

WHEREprice>1

3,4. GROUPING, SELECT

Product TotalSalesBagel 40

Banana 20

Product Price QuantityBagel 3 20Bagel 1.50 20

Banana 0.5 50Banana 2 10Banana 4 10

ORDERING RESULTS

CSE 344 - 2017au 8

SELECT product, sum(price*quantity) as revFROM PurchaseGROUP BY productORDER BY rev desc

Purchase(pid, product, price, quantity, month)

Note: some SQL engineswant you to say ORDER BY sum(price*quantity) desc

HAVING CLAUSE

CSE 344 - 2017au 9

SELECT product, sum(price*quantity)FROM PurchaseWHERE price > 1GROUP BY productHAVING sum(quantity) > 30

Same query as before, except that we consider only productsthat had at least 30 sales.

HAVING clause contains conditions on aggregates.

GENERAL FORM OF GROUPINGAND AGGREGATION

S = may contain attributes a1,…,ak and/or any aggregates but NO OTHER ATTRIBUTES

C1 = is any condition on the attributes in R1,…,Rn

C2 = is any condition on aggregate expressionsand on attributes a1,…,ak

CSE 344 - 2017au 10

SELECT SFROM R1,…,RnWHERE C1GROUP BY a1,…,akHAVING C2

SEMANTICS OF SQL WITH GROUP-BY

CSE 344 - 2017au 11

Evaluation steps:1. Evaluate FROM-WHERE using Nested Loop Semantics2. Group by the attributes a1,…,ak

3. Apply condition C2 to each group (may have aggregates)4. Compute aggregates in S and return the result

SELECT SFROM R1,…,RnWHERE C1GROUP BY a1,…,akHAVING C2

FWGHOS

EXERCISE

CSE 344 - 2017au 12

Compute the total income per monthShow only months with less than 10 items soldOrder by quantity sold and display as “TotalSold”

EXERCISE

CSE 344 - 2017au 13

FROM Purchase

EXERCISE

CSE 344 - 2017au 14

FROM PurchaseGROUP BY month

EXERCISE

CSE 344 - 2017au 15

FROM PurchaseGROUP BY monthHAVING sum(quantity) < 10

EXERCISE

CSE 344 - 2017au 16

SELECT month, sum(price*quantity), sum(quantity) as TotalSold

FROM PurchaseGROUP BY monthHAVING sum(quantity) < 10

EXERCISE

CSE 344 - 2017au 17

SELECT month, sum(price*quantity), sum(quantity) as TotalSold

FROM PurchaseGROUP BY monthHAVING sum(quantity) < 10ORDER BY sum(quantity)

WHERE VS HAVING

WHERE condition is applied to individual rows• The rows may or may not contribute to the aggregate• No aggregates allowed here• Occasionally, some groups become empty and are

removed

HAVING condition is applied to the entire group• Entire group is returned, or removed• May use aggregate functions on the group

CSE 344 - 2017au 18

MYSTERY QUERY

SELECT month, sum(quantity), max(price)FROM PurchaseGROUP BY month

What do they compute?

SELECT month, sum(quantity)FROM PurchaseGROUP BY month

SELECT monthFROM PurchaseGROUP BY month

MYSTERY QUERY

SELECT month, sum(quantity), max(price)FROM PurchaseGROUP BY month

What do they compute?

SELECT month, sum(quantity)FROM PurchaseGROUP BY month

SELECT monthFROM PurchaseGROUP BY month

Lesson:DISTINCT isa special caseof GROUP BY

AGGREGATE + JOIN

Product(pid,pname,manufacturer)Purchase(id,product_id,price,month)

For each manufacturer, compute how many products with price > $100 they sold

AGGREGATE + JOIN

Problem: manufacturer is in Purchase, price is in Product...

AGGREGATE + JOIN

-- step 1: think about their joinSELECT ...FROM Product x, Purchase yWHERE x.pid = y.product_idand y.price > 100

manufacturer ... price ...

Hitachi 150

Canon 300

Hitachi 180

AGGREGATE + JOIN

-- step 2: do the group-by on the joinSELECT x.manufacturer, count(*)FROM Product x, Purchase yWHERE x.pid = y.product_idand y.price > 100

GROUP BY x.manufacturer

manufacturer count(*)

Hitachi 2

Canon 1

-- step 1: think about their joinSELECT ...FROM Product x, Purchase yWHERE x.pid = y.product_idand y.price > 100

manufacturer ... price ...

Hitachi 150

Canon 300

Hitachi 180

AGGREGATE + JOIN

SELECT x.manufacturer, y.month, count(*)FROM Product x, Purchase yWHERE x.pid = y.product_idand y.price > 100

GROUP BY x.manufacturer, y.month

manufacturer month count(*)

Hitachi Jan 2

Hitachi Feb 1

Canon Jan 3

Variant:For each manufacturer, compute how many products with price > $100 they sold in each month

INCLUDING EMPTY GROUPSIn the result of a group by query, there is one row per group in the result

CSE 344 - 2017au 26

SELECT x.manufacturer, count(*)FROM Product x, Purchase yWHERE x.pname = y.productGROUP BY x.manufacturer

Count(*) is never 0

FWGHOS

INCLUDING EMPTY GROUPS

CSE 344 - 2017au 27

SELECT x.manufacturer, count(y.pid)FROM Product x LEFT OUTER JOIN Purchase yON x.pname = y.productGROUP BY x.manufacturer

Count(pid) is 0when all pid’s in

the group areNULL

SUBQUERIES

A subquery is a SQL query nested inside a larger querySuch inner-outer queries are called nested queriesA subquery may occur in:

• A SELECT clause• A FROM clause• A WHERE clause

Rule of thumb: avoid nested queries when possible• But sometimes it’s impossible, as we will see

CSE 344 - 2017au 28

SUBQUERIES…Can return a single value to be included in a SELECT clauseCan return a relation to be included in the FROMclause, aliased using a tuple variableCan return a single value to be compared with another value in a WHERE clauseCan return a relation to be used in the WHERE or HAVING clause under an existential quantifier

CSE 344 - 2017au 29

1. SUBQUERIES IN SELECT

CSE 344 - 2017au 30

Product (pname, price, cid)Company (cid, cname, city)

For each product return the city where it is manufactured

SELECT X.pname, (SELECT Y.cityFROM Company YWHERE Y.cid=X.cid) as City

FROM Product X

What happens if the subquery returns more than one city?

We get a runtime error(and SQLite simply ignores the extra values…)

“correlated subquery”

CSE 344 - 2017au 31

Whenever possible, don’t use a nested queries:

SELECT X.pname, Y.cityFROM Product X, Company YWHERE X.cid=Y.cid

SELECT X.pname, (SELECT Y.cityFROM Company YWHERE Y.cid=X.cid) as City

FROM Product X

We have “unnested”the query

CSE 344 - 2017au 32

Compute the number of products made by each company

SELECT DISTINCT C.cname, (SELECT count(*)FROM Product P WHERE P.cid=C.cid)

FROM Company C

CSE 344 - 2017au 33

Compute the number of products made by each company

SELECT DISTINCT C.cname, (SELECT count(*) FROM Product P WHERE P.cid=C.cid)

FROM Company C

Better: we can unnest using a GROUP BY

SELECT C.cname, count(*)FROM Company C, Product PWHERE C.cid=P.cidGROUP BY C.cname

CSE 344 - 2017au 34

But are these really equivalent?

FROM Company C

CSE 344 - 2017au 35

But are these really equivalent?

FROM Company C

No! Different results if a company has no products

SELECT C.cname, count(pname)FROM Company C LEFT OUTER JOIN Product PON C.cid=P.cidGROUP BY C.cname

2. SUBQUERIES IN FROM

CSE 344 - 2017au 36

Find all products whose prices is > 20 and < 500

SELECT X.pnameFROM (SELECT *

FROM Product AS Y WHERE price > 20) as X

WHERE X.price < 500

CSE 344 - 2017au 37

WHERE X.price < 500

Try unnest this query !

CSE 344 - 2017au 38

WHERE X.price < 500

Try unnest this query !

Side note: This is not a correlated subquery. (why?)

Sometimes we need to compute an intermediate table only to use it later in a SELECT-FROM-WHEREOption 1: use a subquery in the FROM clauseOption 2: use the WITH clause

CSE 344 - 2017au 39

CSE 344 - 2017au 40

WHERE X.price < 500

=WITH myTable AS (SELECT * FROM Product AS Y WHERE price > 20)SELECT X.pnameFROM myTable as XWHERE X.price < 500

A subquery whoseresult we called myTable

3. SUBQUERIES IN WHERE

CSE 344 - 2017au 41

Find all companies that make some products with price < 200

CSE 344 - 2017au 42

Existential quantifiers

CSE 344 - 2017au 43

SELECT DISTINCT C.cnameFROM Company CWHERE EXISTS (SELECT *

FROM Product PWHERE C.cid = P.cid and P.price < 200)

Using EXISTS:

CSE 344 - 2017au 44

SELECT DISTINCT C.cnameFROM Company CWHERE C.cid IN (SELECT P.cid

FROM Product PWHERE P.price < 200)

Using IN

CSE 344 - 2017au 45

SELECT DISTINCT C.cnameFROM Company CWHERE 200 > ANY (SELECT price

FROM Product PWHERE P.cid = C.cid)

Using ANY:

CSE 344 - 2017au 46

SELECT DISTINCT C.cnameFROM Company CWHERE 200 > ANY (SELECT price

Using ANY:

Not supported in sqlite

CSE 344 - 2017au 47

SELECT DISTINCT C.cnameFROM Company C, Product PWHERE C.cid = P.cid and P.price < 200

Now let’s unnest it:

CSE 344 - 2017au 48

SELECT DISTINCT C.cnameFROM Company C, Product PWHERE C.cid = P.cid and P.price < 200

Existential quantifiers are easy! J

Now let’s unnest it:

CSE 344 - 2017au 49

same as:

Find all companies that make only products with price < 200

Find all companies s.t. all their products have price < 200

CSE 344 - 2017au 50

same as:

Universal quantifiers

CSE 344 - 2017au 51

Universal quantifiers are hard! L

same as:

CSE 344 - 2017au 52

1. Find the other companies that make some product ≥ 200

FROM Product PWHERE P.price >= 200)

CSE 344 - 2017au 53

2. Find all companies s.t. all their products have price < 200

1. Find the other companies that make some product ≥ 200

SELECT DISTINCT C.cnameFROM Company CWHERE C.cid NOT IN (SELECT P.cid

CSE 344 - 2017au 54

SELECT DISTINCT C.cnameFROM Company CWHERE NOT EXISTS (SELECT *

FROM Product PWHERE P.cid = C.cid and P.price >= 200)

Using EXISTS:

CSE 344 - 2017au 55

SELECT DISTINCT C.cnameFROM Company CWHERE 200 >= ALL (SELECT price

Using ALL:

CSE 344 - 2017au 56

SELECT DISTINCT C.cnameFROM Company CWHERE 200 >= ALL (SELECT price

Using ALL:

Not supported in sqlite

QUESTION FOR DATABASE THEORY FANSAND THEIR FRIENDS

Can we unnest the universal quantifierquery?

We need to first discuss the concept of monotonicity

CSE 344 - 2017au 57

MONOTONE QUERIESDefinition A query Q is monotone if:

• Whenever we add tuples to one or more input tables, the answer to the query will not lose any of the tuples

CSE 344 - 2017au 58

CSE 344 - 2017au 59

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c004

Camera 149.99 c003

cid cname city

c002 Sunworks Bonn

c001 DB Inc. Lyon

c003 Builder Lodtz

Product Company

Q pname city

Gizmo Lyon

Camera Lodtz

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c004

Camera 149.99 c003

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c004

Camera 149.99 c003

iPad 499.99 c001

cid cname city

c002 Sunworks Bonn

c001 DB Inc. Lyon

c003 Builder Lodtz

Product Companypname city

Gizmo Lyon

Camera Lodtz

pname city

Gizmo Lyon

Camera Lodtz

iPad Lyon

Product Company

Qcid cname city

c002 Sunworks Bonn

c001 DB Inc. Lyon

c003 Builder Lodtz

So far it looks monotone...

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c004

Camera 149.99 c003

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c004

Camera 149.99 c003

iPad 499.99 c001

cid cname city

c002 Sunworks Bonn

c001 DB Inc. Lyon

c003 Builder Lodtz

Product Companypname city

Gizmo Lyon

Camera Lodtz

pname city

Gizmo Lodtz

Camera Lodtz

iPad Lyon

Product Company

Qcid cname city

c002 Sunworks Bonn

c001 DB Inc. Lyon

c003 Builder Lodtz

c004 Crafter Lodtz

Q is not monotone!

MONOTONE QUERIESTheorem: If Q is a SELECT-FROM-WHERE query that does not have subqueries, and no aggregates, then it is monotone.

CSE 344 - 2017au 62

MONOTONE QUERIESTheorem: If Q is a SELECT-FROM-WHERE query that does not have subqueries, and no aggregates, then it is monotone.

Proof. We use the nested loop semantics: if we insert a tuple in a relation Ri, this will not remove any tuples from the answer

CSE 344 - 2017au 63

SELECT a1, a2, …, akFROM R1 AS x1, R2 AS x2, …, Rn AS xnWHERE Conditions

for x1 in R1 dofor x2 in R2 do

…for xn in Rn doif Conditionsoutput (a1,…,ak)

MONOTONE QUERIESThe query:

is not monotone

pname price cid

Gizmo 19.99 c001

cid cname city

c001 Sunworks Bonn

Sunworks

is not monotone

Consequence: If a query is not monotonic, then we cannot write it as a SELECT-FROM-WHERE query without nested subqueries

pname price cid

Gizmo 19.99 c001

cid cname city

c001 Sunworks Bonn

Sunworks

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c001

cid cname city

c001 Sunworks Bonn

QUERIES THAT MUST BE NESTEDQueries with universal quantifiers or with negation

CSE 344 - 2017au 67

QUERIES THAT MUST BE NESTEDQueries with universal quantifiers or with negation

Queries that use aggregates in certain ways

• sum(..) and count(*) are NOT monotone, because they do not satisfy set containment

• select count(*) from R is not monotone!

CSE 344 - 2017au 68

INTRODUCTION TO DATA MANAGEMENTCSE 344

LECTURE 7-8: SQL WRAP-UP

RELATIONAL ALGEBRA

CSE 344 - 2017au 69

ANNOUNCEMENTSYou received invitation email to @csYou will be prompted to choose passwd

• Problems with existing account?• In the worst case we will ask you to create a new @outlook

account just for this classIf OK, create the database server

• Choose cheapest pricing tier!Remember: WQ2 due on Friday

GROUP BY V.S. NESTED QUERIES

CSE 344 - 2017au 71

SELECT DISTINCT x.product, (SELECT Sum(y.quantity)FROM Purchase yWHERE x.product = y.productAND y.price > 1)

AS TotalSalesFROM Purchase xWHERE x.price > 1

Why twice ?

Purchase(pid, product, quantity, price)

MORE UNNESTING

CSE 344 - 2017au 72

Author(login,name)Wrote(login,url)

Find authors who wrote ≥ 10 documents:

MORE UNNESTING

CSE 344 - 2017au 73

SELECT DISTINCT Author.nameFROM AuthorWHERE (SELECT count(Wrote.url)

FROM WroteWHERE Author.login=Wrote.login)

This isSQL bya novice

Attempt 1: with nested queries

MORE UNNESTING

CSE 344 - 2017au 74

Attempt 1: with nested queries

SELECT Author.nameFROM Author, WroteWHERE Author.login=Wrote.loginGROUP BY Author.nameHAVING count(wrote.url) >= 10

This isSQL by

an expert

Attempt 2: using GROUP BY and HAVING

FINDING WITNESSES

CSE 344 - 2017au 75

For each city, find the most expensive product made in that city

FINDING WITNESSES

CSE 344 - 2017au 76

SELECT x.city, max(y.price)FROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city;

Finding the maximum price is easy…

But we need the witnesses, i.e., the products with max price

For each city, find the most expensive product made in that city

FINDING WITNESSES

CSE 344 - 2017au 77

To find the witnesses, compute the maximum pricein a subquery (in FROM or in WITH)

WITH CityMax AS (SELECT x.city, max(y.price) as maxpriceFROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city)

SELECT DISTINCT u.city, v.pname, v.priceFROM Company u, Product v, CityMax wWHERE u.cid = v.cid

and u.city = w.cityand v.price = w.maxprice;

FINDING WITNESSES

CSE 344 - 2017au 78

To find the witnesses, compute the maximum pricein a subquery (in FROM or in WITH)

SELECT DISTINCT u.city, v.pname, v.priceFROM Company u, Product v,

(SELECT x.city, max(y.price) as maxpriceFROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city) w

WHERE u.cid = v.cidand u.city = w.cityand v.price = w.maxprice;

FINDING WITNESSES

CSE 344 - 2017au 79

Or we can use a subquery in where clause

SELECT u.city, v.pname, v.priceFROM Company u, Product vWHERE u.cid = v.cidand v.price >= ALL (SELECT y.price

FROM Company x, Product y WHERE u.city=x.cityand x.cid=y.cid);

FINDING WITNESSES

CSE 344 - 2017au 80

There is a more concise solution here:

SELECT u.city, v.pname, v.priceFROM Company u, Product v, Company x, Product yWHERE u.cid = v.cid and u.city = x.cityand x.cid = y.cidGROUP BY u.city, v.pname, v.priceHAVING v.price = max(y.price)

subqueries - University of Washington · SUBQUERIES A subqueryis a SQL query nested inside a larger...

Documents

Chapter 9 Advanced Query Formulation with SQL. Outline Outer join problems Type I nested queries Type II nested queries and difference problems Nested

SQL Overview Data Definition Basic Queries Set Operations Null Values Aggregate Functions Nested Subqueries Modification of the Database

SQL Queries and Subqueries Zaki Malik September 04, 2008

Hoofdstuk 8: Werken met SQL voor eindgebruikers. Overzicht Inleiding op SQL: de select-instructie Eenvoudige queries Join queries Nested queries Queries

Basic Nested Queries and Views

Optimization of Nested Queries Sujatha Thanigaimani COSC 6421

Nested Queries (Sub-Queries). Objectives Learn how to run a query as a nested sub-query Condition on nested query Application of nested query Restriction

Chapter 4: SQL - WordPress.com 3: SQL Data Definition Basic Query Structure Set Operations Aggregate Functions Null Values Nested Subqueries Complex Queries Views …

Chapter Eleven Data Manipulation Language (DML) Nested Queries Dr. Chitsaz

SQL QUERIES - California Institute of Technologyusers.cms.caltech.edu/~donnie/cs121/CS121Lec05.pdf · ¨SQL provides broad support for nested subqueries ¤A SQL query is a “select-from-where”

Subqueries to Solve Queries Questions

Multiple Table Queries 2: Outer Joins, Self Joins, Nested Queries, and Views CS 320

Structural and Content Queries on the Nested …silvello/file/2012SEBD.pdf · Structural and Content Queries on the Nested ... and we describe how it is possible to reconsider structural

UNIT – III Form of basic SQL Query Examples of Basic SQL Queries Introduction to Nested Queries Correlated Nested Queries set Comparison operators Aggregative

CREATING COMPLEX QUERIES WITH NESTED QUERIES CS1100: Data, Databases, and Queries CS1100Microsoft Access1

Aggregate functions and nested queries - ITUmogel/SIDD2012/lectures/SIDD.2012.03.pdf · Aggregate functions and nested queries Introduction to Database Design 2012, Lecture 3 Rasmus

Introduction to SQL Introduction Select-From-Where Statements Queries over Several Relations Subqueries

SQL (Chapter 2: Simple queries; Chapter 7 and 8: Nested and DML queries)

Subquery - WordPress.comSubquery: It is also known as Nested query. Sub queries are queries nested inside other queries, marked off with parentheses, and sometimes referred to as "inner"

Database · B.E., MCA, M.Phil., Head of Computer Science and Information Technology Department, ... Subqueries – Subqueries with IN, EXISTS, Subqueries Restrictions, Nested Subqueries,