B211 Parser

7/27/2019 B211 Parser

http://slidepdf.com/reader/full/b211-parser 1/22

Module 11: SQL Parser

After completing this module, you will be able to:

Describe internal, channel, and LAN parcels.

Explain software cache functionality.

List and identify the function of the main components

(phases) of the parser.

Describe the functionality of Request-To-Steps cache.

7/27/2019 B211 Parser


Internal, Channel and LAN Parcels

• A REQUEST parcel is followed by zero or one DATA parcel plus one

RESPOND parcel.

• The RESPOND parcel identifies response buffer size.

• A RESPOND parcel may be sent by itself as a continuation request for

additional data.

• A SUCCESS parcel may be followed by RECORD parcels.

• Every REQUEST parcel generates a SUCCESS/FAIL parcel.

Teradata

RDBMS

REQUEST

PARCEL

1

DATA

PARCEL

RESPOND

PARCEL

RECORD

PARCEL

RECORD

PARCEL

Success / Fail

PARCEL

HOST

2 3

123

INTERNAL CHANNEL PARCELS

7/27/2019 B211 Parser


Request Parcel

A Request Parcel:

• Must contain at least one SQL statement.

• May contain two or more SQL statements (multi-statement request).

• May be a transaction by itself (default).

• May be one parcel of a multi-request transaction.

A Request Parcel is the parsing unit.

7/27/2019 B211 Parser


The Data Parcel

• Explicitly declare a USING data parcel in BTEQ.

• FastLoad and MultiLoad do not use the SQL protocol, but use this conceptthrough the DEFINE or LAYOUT statements.

• The Preprocessor generates a USING data parcel from macro parameters or

SQL statements that reference host program variables.

• CLI programs must create and manage their own DATA parcels.

USING x INTEGER, y CHAR(5), z INTEGER

EXEC My_Macro (:x, :y, :z); 100, 'ABC', 321

CREATE MACRO My_Macro(var1 INTEGER

,var2 CHAR(5)

,var3 INTEGER )

AS ( INSERT INTO Table_1

VALUES (:var1, :var2, :var3);

UPDATE Table_2

SET Col_C = :var3 + 1WHERE Col_A = :var1; ) ;

User executes

Request Parcel USING x INTEGER, y CHAR(5), z INTEGER

EXEC My_Macro (:x, :y, :z);

Within Teradata

100, 'ABC', 321 Data Parcel

7/27/2019 B211 Parser


7/27/2019 B211 Parser


Software Cache

• ALL Teradata vprocs use some SMP memory as software cache.

• Software cache retains data dictionary information and/or processing

steps.

– Dictionary Cache - hold DD/D information in memory

– Requests-To-Steps Cache - holds processing steps in memory

• Cache eliminates regenerating and re-fetching needed information.

• Larger memory gives the processors more software cache and reduces

disk accesses.

7/27/2019 B211 Parser


Dictionary Cache

• The Parser needs data from the DD to convert SQL into AMP steps.

• DD cache holds the most recently used items in PE memory:

– SQL names and their related numeric IDs.

– Statistical information for the Optimizer.

• SQL statements that change the DD generate “spoiling” messages.

• PEs drop the “spoiled” entries from their cache.

• The system purges cache every four hours:

– Data demographics can change significantly within four hours.

– Purging cache forces the Parser to re-optimize all current requests.

•

Processors purge their cache on a sequential basis.

• DD tables used by the resolver to parse a request:

– Dbase – TVM – AccessRights

– TVFields – Indexes – RoleGrants (V2R5)

7/27/2019 B211 Parser


Request-to-Steps Cache

• Cache stores the SQL text and the AMP steps generated by the Parser

without binding in data from the Using DATA parcel.

– Plans do not have to be re-parsed by the user.

– Requests with hard-coded values are not immediately cached.

• For cached plans, the plan is retrieved and the SYNTAXER, RESOLVER,

OPTIMIZER and GENERATOR steps are bypassed.• Cached steps may be shared by sessions/logons within a Parsing Engine.

• Cache is maintained in Least Recently Used sequence so that lower activity

plans swap out to disk.

• The system purges unmarked entries from cache every four hours.

– All PEs also purge affected cache entries on receipt of a DDL “Spoiling” message.

• Demographically independent plans (UPI, USI and some Nested Joins) are

marked when placed into cache.

• DDL requests are never cached.

7/27/2019 B211 Parser


Request-to-Steps Cache Check

REQUEST

Parcel

Request/Steps

CACHE

REQUESTParcel

PlasticSteps

Cached?

To SYNTAXER To SECURITY and APPLY

No Yes

If an identical Request exists in Request-To-Steps cache:

• Call SECURITY and APPLY and pass them the memory address of the AMP steps.

• These steps do not have DATA parcel values bound into them; they are called Plastic

steps.

Otherwise, the Request Parcel passes the request to the SYNTAXER.

The larger the Request Parcel, the longer these steps take.Macros reduce parcel size, dramatically improving performance.

7/27/2019 B211 Parser


Request-To-Steps Cache Logic

CREATE

HASH of

PARCEL

HASH

in REQ.AREA

?

TEXT

MATCH?

PARCEL

CONTAIN

: var

?

HASH

in TEXT

AREA

?

Move HASH

and TEXT to

REQUEST AREA

Y

N

Y

N

Y

N

Y

N

PARSE

Request

STORE HASH

and TEXT in

REQUEST AREA

STORE HASH

in TEXT AREA

SECURITY

PARSE

Request

OPTIMIZE Request and GENERATE Steps

STORE

Steps

Apply data parcels and dispatch steps to AMP(s)

7/27/2019 B211 Parser


Syntaxer

REQUEST

Parcel

InitialPARSE TREE

To RESOLVER

SYNTAXER

• This module checks the syntax of an incoming Request Parcel.

• If no errors are found, it produces an initial parse Tree and calls the

RESOLVER.

• The larger the Request Parcel, the longer these steps take.

•

Macros reduce parcel size, dramatically improving performance.

7/27/2019 B211 Parser


Resolver

Initial

PARSE TREE

AnnotatedPARSE TREE

To SECURITY

RESOLVERViews and Macros

From SYNTAXER

DD Cache

DD Dbase

AccessRights

RoleGrants (V2R5)

TVM

TVFieldsIndexes

• RESOLVER replaces all View and Macro names with their underlying text.

• Nesting of Views and macros could add substantial time to this process.

• Views and/or Macros can be nested 10 levels (V2R3) or 64 levels (V2R4).

• The RESOLVER uses information from the DD cache when possible.

• It accesses the DD tables if additional information is needed.

7/27/2019 B211 Parser


Security

SECURITY verifies that the requesting User ID has the necessary permissions

on the referenced objects.

Permissions can be granted or revoked dynamically.

Each execution of a Request includes a SECURITY check.

• Initial parse and execute checks ALL access rights.

• Execution from cache checks only user rights (i.e., EXEC).

Annotated

PARSE Tree

Plastic

Steps

To OPTIMIZER To APPLY

AnnotatedPARSE Tree

SECURITY

PlasticSteps

7/27/2019 B211 Parser


Optimizer

• DD/D operations replace DDL statements in the Parse tree.

• The OPTIMIZER evaluates DML statements for possible access paths: – Available indexes referenced in the WHERE clause.

– Possible join plans from the WHERE clause.

– Full Table Scan possibility.

• It uses COLLECTed STATISTICS or dynamic samples to make a choice.

•It generates Serial, Parallel, Individual and Common steps.

• OPTIMIZER output is passed to either the Generator or the Explain facility.

Annotated

PARSE TREE

OptimizedPARSE TREE

To GENERATOR

OPTIMIZER

From SECURITY

Statistics

EXPLAIN

7/27/2019 B211 Parser


Generator

• Plastic Steps are AMP steps without data values from the DATA parcel boundin.

• Hard-coded literal values are embedded in the Plastic Steps.

• Plastic Steps are stored in Request-to-Steps cache.

Optimized

PARSE TREE

PlasticSteps

To APPLY

GENERATOR

From OPTIMIZER

R-T-S CACHE

7/27/2019 B211 Parser


Apply

APPLY binds the DATA parcel values into the Plastic Steps:

• This produces Concrete Steps.

•

The DISPATCHER sends the Concrete Steps to the AMPs.• a.k.a., GncApply in V2 or OptApply in V1

Plastic

Steps

ConcreteSteps

APPLY

From SECURITY or

GENERATOR

DATA

Parcel

DISPATCHER

AMPs

7/27/2019 B211 Parser


SQL Parser Review

DATA parcel

STATISTICS

GENERATOR OPTIMIZER

APPLY

EXPLAIN

Plastic STEPS

Concrete STEPS

R-T-S Cache

Teradata’s “Late Binding” Parser

provides maximum flexibility. DISPATCHER

RESOLVER

SYNTAXER

CACHED? No

SECURITY

REQUEST Parcel

Yes

DD Cache

DD Dbase

AccessRights

RoleGrants

TVM

TVFields

Indexes

7/27/2019 B211 Parser


Parser Summary

• Plastic Steps for Requests with DATA parcels are cached immediately.

• Views and Macros cannot be nested beyond 10 levels (V2R3) or 64 levels

(V2R4).

• Nested Views and macros take longer for initial parsing.

• Multi-statement requests (including macros) generate more Parallel and

Common steps.

• Execution plans remain current in cache for up to four hours.

• DDL “spoiling” messages may purge DD cache entries at any time.

• Requests against purged entries must be re-parsed and re-optimized.

7/27/2019 B211 Parser


Review Questions

1. What must a REQUEST parcel contain? ________________

2. Which statement(s) about the RESPOND parcel are true? ___

a. Identifies response buffer size.

b. Generates a SUCCESS/FAIL parcel.

c. Always followed by one or more DATA parcels.

d. May be sent by itself as a continuation request.

3. Match the six SQL Parser sections (phases) listed below with its correct description.

__ Syntaxer a. Determines whether the Requesting User ID has the necessary permissions

__ Resolver b. Create concrete steps

__ Security c. Checks the Request for valid syntax.

__ Optimizer d. Creates the steps for execution.

__ Generator e. Breaks down Views and Macros into their underlying table references

__ Apply f. Chooses the execution plan.

4. Select the Parser phase that benefits the most from the use of macros? ___

a. Generator

b. Resolver

c. Syntaxer

d. Apply

7/27/2019 B211 Parser


Module 11: Review Question Answers

1. What must a REQUEST parcel contain? At least 1 SQL statement

2. Which statement(s) about the RESPOND parcel are true? ___

a. Identifies response buffer size.

b. Generates a SUCCESS/FAIL parcel.

c. Always followed by one or more DATA parcels.

d. May be sent by itself as a continuation request.

3. Match the six SQL Parser sections (phases) listed below with its correct description.

C Syntaxer a. Determines whether the Requesting User ID has the necessary permissions

E Resolver b. Create concrete steps

A Security c. Checks the Request for valid syntax.

F Optimizer d. Creates the steps for execution.

D Generator e. Breaks down Views and Macros into their underlying table references

B Apply f. Chooses the execution plan.

4. Select the Parser phase that benefits the most from the use of macros? ___

a. Generator

b. Resolver

c. Syntaxer

d. Apply

7/27/2019 B211 Parser


Review Questions (cont.)

5. What is the function of the Request-to-Steps (R-T-S) Cache? ____

a. Stores the SQL text and AMP steps generated by the Parser.b. Resolves View and Macro references down to table references.

c. Stores the most recently used DD information including SQL names, their related numeric IDs

and Statistics.

d. Analyzes the various ways an SQL Request can be executed and determines which of these is

the most efficient.

6. Teradata's Late Binding Parser refers to Apply, which acts upon the _____________ from the Generator and produces __________ by binding in the data values from the DATA parcel.

a. Interpretive Steps / Compiled Steps

b. Processing Steps / Execution Steps

c. AMP steps / Request-to-Steps Cache

d. Plastic Steps / Concrete Steps

7/27/2019 B211 Parser


Module 11: Review Question Answers

5. What is the function of the Request-to-Steps (R-T-S) Cache? ____

a. Stores the SQL text and AMP steps generated by the Parser.b. Resolves View and Macro references down to table references.

c. Stores the most recently used DD information including SQL names, their related numeric IDs

and Statistics.

d. Analyzes the various ways an SQL Request can be executed and determines which of these is

the most efficient.

6. Teradata’s Late Binding Parser refers to Apply, which acts upon the _____________ from the Generator and produces __________ by binding in the data values from the DATA parcel.

a. Interpretive Steps / Compiled Steps

b. Processing Steps / Execution Steps

c. AMP steps / Request-to-Steps Cache

d. Plastic Steps / Concrete Steps

Documents

B211 Parser