Sybase ASE Compatibility Mode 072309 WP

Using 'Compatibility Mode' In ASE

15.0.3 ESD#1 For ASE 15 Migration

TECHNIC AL WHITE P APER

www.sybase.com

Using 'Compatibility Mode' In ASE 15.0.3 ESD#1 For ASE 15 Migration

Version 1.1 - July 2009 1

Contents

Introduction............................................................................................................................................... 2

What is 'Compatibility Mode'? .............................................................................................................................. 2 Why is Compatibility Mode introduced? ............................................................................................................... 2

Enabling Compatibility Mode .................................................................................................................... 3

Determining whether Compatibility Mode is enabled .......................................................................................... 3

How Compatibility Mode works ................................................................................................................ 4

Influencing the choice of processing mode ........................................................................................................... 5

Example of different processing modes .................................................................................................... 6

Implications of using Compatibility Mode ................................................................................................ 9

Features Incompatible with Compatibility Mode ................................................................................... 10

Understanding why Compatibility Mode is not used .............................................................................. 11

sp_compatmode: identifying potentially conflicting settings ............................................................................. 11

Using Compatibility Mode for ASE 15 migration ..................................................................................... 13

Reference Documents ............................................................................................................................. 15

Author

Rob Verschoor [email protected]

Revision History

Version 1.0 - May 2009

Version 1.1 - July 2009 - added information about traceflag 441

2 Using 'Compatibility Mode' In ASE 15.0.3 ESD#1 For ASE 15 Migration

Version 1.1 - July 2009

Introduction

At the end of March 2009, version 15.0.3 ESD#1 of Sybase Adaptive Server Enterprise (ASE) was released. This ESD

contained a new feature named 'compatibility mode'.

This whitepaper explores what compatibility mode is, how it works and how it can be applied when migrating to

ASE 15.

What is 'Compatibility Mode'?

Compatibility mode is a query processing enhancement in ASE 15.0.3 ESD#1 that allows qualifying T-SQL queries to

be processed with a method of query optimization and query execution very similar to that used in ASE 12.5,

instead of using the ASE 15 query processing algorithms.

Why is Compatibility Mode introduced?

ASE 15 introduced a fully redesigned query processing engine with many query processing enhancements such as

hash joins and more efficient algorithms for calculating aggregates, to name just two features out of many. The

ASE 15 query processing engine was designed to yield greatly improved query performance. Indeed, many

customers have upgraded their systems to ASE 15 and benefited from these query processing enhancements.

The significant changes in ASE 15 query processing, however, do put a greater onus on customers to sufficiently

test their applications to determine the ASE 15 query processing configuration that yields best results for their

applications.

However, not all customers are able or willing to perform such testing at sufficiently thorough levels, often due to

inability to dedicate enough resources at a given point in time. In some cases, customers may also determine that

they do not expect some of their applications to benefit from the new ASE 15 features due to the nature of their

SQL workload, or that they need longer lead times to change their applications to take full advantage of the ASE 15

features.

Nevertheless, customers would still like to migrate to ASE 15 as ASE 12.5 is reaching end of life at the end of 2009.

In such cases, customers would prefer to limit the amount of testing effort spent when migration to ASE 15.

Against this background, compatibility mode allows customers greater flexibility in managing their migrations to

ASE 15. It aims to provide an easier migration path to ASE 15, requiring less application testing effort than when

migrating to ASE 15 without using compatibility mode. This is achieved by generating query plans in ASE 15 that

are as close as possible to the corresponding query plans in ASE 12.5, thus reducing the level of testing required for

migrating to ASE 15.

With compatibility mode, customers have more control over the precise moment when ASE 15-specific query

processing features become active in their queries and applications, thus allowing the required testing effort to be

better focused and planned than without compatibility mode. Having said that, Sybase still recommends to

perform sufficient testing on applications that are migrated to ASE 15, even when using compatibility mode.

Please refer to page 13 for a more detailed discussion on how compatibility mode may be used when migrating to

ASE 15.

It should be noted that, as a consequence of using compatibility mode, applications may not be able to fully

benefit from the query processing enhancements of ASE 15. This is a conscious choice that customers will need to

make when using compatibility mode.



Enabling Compatibility Mode

Compatibility mode can be enabled server-wide for the entire ASE server, as well as for an individual session only.

By default, compatibility mode is disabled. To enable it on server level, set the configuration parameter 'enable

compatibility mode' to 1 (when set to 0, it is disabled):

sp_configure 'enable compatibility mode', 1

To override the server-wide compatibility mode setting for an individual session, use this syntax:

set compatibility_mode on -- set to off to disable

This command can also be used in a login trigger.

In both cases, changing the setting is dynamic, meaning it takes effect immediately. However, please note what

'taking effect' means: since compatibility mode applies at the moment when a statement's query plan is compiled,

already existing plans will not be affected.

The next few sections discuss this in more detail.

Determining whether Compatibility Mode is enabled

To determine whether compatibility mode is enabled server-wide, simply run sp_configure

'compatibility mode' to display the current setting.

To determine whether compatibility mode has been explicitly enabled for the current session with set

compatibility_mode on, run sp_show_options:

1> sp_show_options

2> go

number name

----------- ------------------

7 arithabort

[...]

93 compatibility_mode

In the above output, the row with number = 93 indicates that compatibility mode has been enabled explicitly for

the current session. When compatibility mode is implicitly enabled for the current session as a result of the server-

wide default setting, sp_show_options will not show any indication of compatibility mode.

Note that it might seem more logical to use sp_options to determine whether compatibility mode is enabled

for a session, since the ASE 15 optimizer settings are all visible here. However, the information about the setting

for compatibility mode is internally maintained on a higher level than the optimizer-specific settings. Therefore,

sp_show_options must be used.



How Compatibility Mode works

When enabled, compatibility mode applies at the moment when a query is processed by the ASE query optimizer

in order to generate a query plan. Consequently, any query plans that already exist -either in the statement cache

or in the procedure cache- are not affected when the setting for compatibility mode is changed.

When compatibility mode is enabled, each optimizable SQL statement can be processed in one of three modes, as

described below. After each statement, the session-specific global variable @@qpmode contains a value of 0, 1, 2

or 3 indicating whether compatibility mode was applied.

The logic is as follows:

1. As the first choice, the optimizer determines if it can use both the ASE 12.5-like optimization and

execution logic; this is known as 'full compatibility mode', and is indicated by @@qpmode = 1 after the

statement completes.

Full compatibility mode produces query plans that are very similar, if not identical, to the corresponding

query plan in 12.5.

2. If full compatibility mode cannot be used, the optimizer will consider an intermediate approach, whereby

a 12.5-style initial abstract query plan is generated, which is then fed into the ASE 15 optimizer to obtain

the final query plan; the query is executed with the ASE 15 execution logic. This is known as 'restricted

compatibility mode', and is indicated by @@qpmode = 2.

Restricted compatibility mode should be seen as an attempt to still produce query plans that are as close

to 12.5 as possible, in cases where full compatibility mode cannot be used.

If, for some reason, it would appear to be undesirable for queries to be processed under restricted

compatibility mode, traceflag 446 will disable restricted compatibility mode, thus leaving only full

compatibility mode and ASE 15 mode. Traceflag 446 applies server-wide.

3. If the optimizer determines that restricted compatibility mode cannot be used either, it will use the full

ASE 15 optimization and execution logic; this is known as 'ASE 15 mode', and indicated by @@qpmode = 3.

In ASE 15 mode, query plans generated under the optimization goal allrows_oltp tend to be most

similar to those in 12.5, compared with the other ASE 15 optimization goals.

4. For statements where no optimization took place, @@qpmode will contain 0. For example, this applies to

statements that do not use a query plan, such as create index or update statistics, or when

a query plan resides in the statement cache or procedure cache.

The ASE query processor applies the above logic for every individual statement. It is therefore quite possible, and

in fact expected, that two subsequent statements in a batch or a stored procedure will be processed in different

modes.

Please keep in mind that compatibility mode is functionally neutral: a query will always return the correct result

regardless of whether full or restricted compatibility mode or ASE 15 mode is used. The only difference may be the

final performance since the query plan, as well as run-time query execution, will not be identical in the different

modes.

Which of the different query processing modes is used for a statement depends on whether the statement uses

particular ASE features. In general, when SQL code is taken from ASE 12.5 and migrated to ASE 15 without any

changes, full compatibility mode will be used in most cases, whereas using ASE 15-specific features will often result

in using ASE 15 mode. This is discussed in more detail on page 10.



Finally, it is worth pointing out that there is no performance impact due to ASE running through the above steps to

determine the query processing mode for every statement: ASE does not engage in significant additional

processing, but determines right from the start which mode will be used for every statement.

Influencing the choice of processing mode

Once compatibility mode is enabled, you cannot specify directly which of the different processing modes will be

used for a particular statement: ASE determines this by itself as outlined above. However, in principle it may be

possible to indirectly steer ASE towards using a particular processing mode, by making modifications to the query.

For example, for a query that is processed with full compatibility mode, adding an abstract plan clause (i.e. plan

('…') ) will cause the query to be processed in a different mode (most likely, in ASE 15 mode when a full or

partial abstract plan was specified in the plan clause).

This type of influencing the choice of processing mode should be seen as specialized fine-tuning that should only

be considered for very special problem cases, should those occur.



Example of different processing modes

In the example below, three select statements are executed, each using a different mode. Note that the first

query, which is processed in full compatibility mode, shows the 12.5-style showplan, clearly formatted differently

from the ASE 15-style showplan for the next two statements. Please refer to the subsequent section for more

details.

-- first, some setup:

create table tb ( a int, img_col image )

go

create table partitioned_tb ( a int )

partition by list (a) (p1 values(1), p2 values (2))

go

-- prepare for showing the different modes:

set compatibility_mode on

go

set showplan on

go

1> -- full compatibility mode; note the 12.5-style showplan

2> select a from tb

3> select @@qpmode "@@qpmode:full compat mode"

4> go

QUERY PLAN FOR STATEMENT 1 (at line 3).

STEP 1

The type of query is SELECT.

FROM TABLE

tb

Nested iteration.

Table Scan.

Forward Scan.

Positioning at start of table.

Using I/O Size 2 Kbytes for data pages.

With LRU Buffer Replacement Strategy for data pages.

a

-----------

[...]

@@qpmode:full compat mode

-------------------------

1

(1 row affected)



1> -- restricted compatibility mode: an image column is involved

2> select img_col from tb

3> select @@qpmode "@@qpmode:restricted comp. mode"

4> go


STEP 1


1 operator(s) under root

|ROOT:EMIT Operator (VA = 1)

|

| |SCAN Operator (VA = 0)

| | FROM TABLE

| | tb

| | Table Scan.

| | Forward Scan.

| | Positioning at start of table.

| | Using I/O Size 2 Kbytes for data pages.

| | With LRU Buffer Replacement Strategy for data pages.

img_col

-----------------------------------------------------------------

[...]

@@qpmode:restricted comp. mode

------------------------------

2

(1 row affected)

1> -- ASE 15 mode: accessing a semantically partitioned table

2> select cnt=count(*) from partitioned_tb

3> select @@qpmode "@@qpmode:ASE 15 mode"

4> go


STEP 1


2 operator(s) under root

|ROOT:EMIT Operator (VA = 2)

|

| |SCALAR AGGREGATE Operator (VA = 1)

| | Evaluate Ungrouped COUNT AGGREGATE.

| |

| | |SCAN Operator (VA = 0)

| | | FROM TABLE



| | | partitioned_tb

| | | Table Scan.

| | | Forward Scan.

| | | Positioning at start of table.

| | | Using I/O Size 16 Kbytes for data pages.

| | | With LRU Buffer Replacement Strategy for data pages.

cnt

-----------

[...]

@@qpmode:ASE 15 mode

--------------------

3

(1 row affected)



Implications of using Compatibility Mode

When a T-SQL statement is processed in full compatibility mode or restricted compatibility mode (i.e. @@qpmode

= 1 or 2), its query plan will be generated with 12.5-like optimizer logic. Consequently, all ASE 15-specific query

processing features such as optimization goals, hash joins or hash-based sorting, do not apply, and will therefore

not be used. In contrast, when a statement is optimized in ASE 15 mode, all ASE 15-specific optimizer features and

settings apply.

It should be noted that there is no risk of, for example, syntax errors due to ASE 15-specific settings not being

used: when these don't apply under full/restricted compatibility mode, they are simply ignored.

There is also a visible difference in the query plan as displayed by set showplan on. For queries processed in

full compatibility mode (i.e. @@qpmode = 1), the query plan is displayed in 12.5 format, so without the leading

vertical bars that are characteristic for ASE 15. For statements processed in restricted compatibility mode or in ASE

15 mode, the query plan is displayed in ASE 15 style.

In addition, compatibility mode has implications for some T-SQL commands that had become no-ops in ASE 15,

such as set table count, set sort_merge, set jtc, the configuration parameter 'enable sort-

merge join and JTC' as well as traceflags 334 and 384: for queries that are processed in full compatibility

mode or restricted compatibility mode, these commands and traceflags will apply and influence query optimizer

behaviour as they did in 12.5. When processing in ASE 15 mode, they will still be no-ops.

Furthermore, the classic series of optimizer traceflags 302, 310, 311, 317, 319 and 321, which had been

superseded by set option show_... in ASE 15, will function as in 12.5 when full compatibility mode is used.

Under restricted compatibility mode, the output for these traceflags may be a combination of the output

generated in 12.5 plus the output generated in ASE 15.



Features Incompatible with Compatibility Mode

When determining whether to process a query with use full or restricted compatibility mode or ASE 15 mode, ASE

examines the query for the use of specific features or constructs.

In general, if a statement uses features introduced in ASE 15, then that statement will usually be executed in ASE

15 mode -- although there are some exceptions, like using the rand2() built-in function (introduced in 15.0.1),

which can be processed in restricted compatibility mode.

When migrating from ASE 12.5 to ASE 15 without changing any SQL code, in most cases the SQL from 12.5 will be

processed with full compatibility mode in ASE 15. However, there are some cases where ASE 15 will have to use

restricted compatibility mode or ASE 15 mode, even for SQL code that has not changed compared with 12.5. These

are some examples:

• Queries requiring checking of referential integrity constraints will use restricted compatibility mode or

ASE 15 mode.

• Queries involving text/image columns or column-level encryption will use restricted compatibility mode.

• When the statement cache and literal autoparametrization are enabled in ASE 15, queries where literal

autoparametrization applies will, by default, use ASE 15 mode. The exception to this rule are

insert…values… statements which can still be processed in full compatibility mode.

By enabling traceflag 441, all queries where literal autoparametrization applies will be processed in full

compatibility mode. However, this includes a risk of data type mismatches for predicates like "where

smallintcolumn = 10", as the value of 10 will be parametrized as a 4-byte integer value, being

different from the 2-byte smallint datatype of the column in this predicate. Consequently, enabling

traceflag 441 could, under the 12.5 logic, cause an index on such a column not to be used and could

potentially result in a sub-optimal query plan.

• Queries against semantically partitioned tables will use ASE 15 mode.

• Under full or restricted compatibility mode, parallel plans are not possible, even when the ASE server is

configured for parallelism. When forcing a parallel plan with the '(parallel N)' optimizer hint, the

query will be processed in ASE 15 mode.

For a more detailed list of criteria governing the decision for using full or restricted compatibility mode or ASE 15

mode, see the "Migration Technology Guide", which is part of the ASE 15.0.3 documentation set.

Note that not all new ASE 15 features are incompatible with compatibility mode. For example, the database option

'delayed commit' can be enabled irrespective of the processing mode used. Another example are the row-

level-locked system tables in ASE 15.0.2, which do not interfere with compatibility mode.



Understanding why Compatibility Mode is not used

When investigating details of ASE performance, it may be necessary to have a better understanding why full or

restricted compatibility mode is not used for particular statements.

If a particular statement is not using compatibility mode in the way you expect, enable traceflag 477 (as well as

traceflag 3604!) to obtain more information. For every statement this will print a line:

Compatibility = false

or:

Compatibility = true

Here, Compatibility = true means that full compatibility mode is used. false means that either

restricted compatibility mode or ASE 15 mode is used; you can only tell which of these modes is used by examining

@@qpmode afterwards.

When full compatibility mode cannot be used (i.e. Compatibility = false), traceflag 477 will print an

additional line indicating the reason why. These messages tend to be telegram-style and somewhat cryptic

indications, like:

Unsupported functionality

� this indicates the query involves functionality that cannot processed with full compatibility mode, such

as new built-in functions

RI

� this indicates the query requires checking of referential integrity constraints

Auto params

� this indicates the query is using literal autoparametrization, while it is not an insert…values…

statement

The ASE product manual "Migration Technology Guide", which is part of the ASE 15.0.3 documentation set,

contains a full list of these indicators and their meaning.

sp_compatmode: identifying potentially conflicting settings

Some combinations of configuration parameter settings may affect the processing mode used for a statement.

The system stored procedure sp_compatmode reports about such interference between parameters in the

current server-wide configuration.

Here is an example:

1> sp_compatmode

2> go

Compatibility mode is enabled.

WARNING: Compatibility mode may not be used when statement cache and

literal autoparam are enabled.



WARNING: The configuration option 'histogram tuning factor' is

configured with value '20', which is not the default value in ASE 12.5.

This may lead to different accuracy of statistics and different query

plans.

(return status = 0)

Note that sp_compatmode takes only server-level configuration parameter settings into account: session-level

settings are ignored.

Also note that sp_compatmode reports only on full compatibility mode. So when it says "Compatibility mode

may not be used…" This really applies only to full compatibility mode, and does not exclude the possibility of

restricted compatibility mode being used instead.

In addition, sp_configure will print similar warnings when changing a configuration parameter in such a way

that interference with compatibility mode could occur:

1> sp_configure 'abstract plan dump', 1

2> go

WARNING: Compatibility mode will not be used when 'abstract plan

dump/load/replace' is on.

Parameter Name Default Memory Used Config Value Run Value Unit Type

-------------- ------- ----------- ------------ --------- ---- ----

[...]



Using Compatibility Mode for ASE 15 migration

As mentioned earlier in this document, the purpose of compatibility mode is to allow customers to upgrade from

ASE 12.5 to ASE 15 with less testing effort. This is achieved by generating query plans in ASE 15 that are as close as

possible to the corresponding query plans in ASE 12.5, thus reducing the level of testing required for migrating to

ASE 15.

While compatibility mode undeniably offers advantages in terms of ease of migrating to ASE 15, it should also be

noted that most of the query processing enhancements in ASE 15 will not be utilized for statements using 12.5-

style query plans as a result of using compatibility mode.

The reverse reasoning also applies: under compatibility mode, ASE 12.5-style query plans will only be generated

when most new ASE 15 features are not actively used. When customers want to take full advantage of new ASE 15

features, such as SQL UDFs or semantically partitioned tables, the query plans will be generated in ASE 15-style,

even when compatibility mode is active. When ASE 15-style query plans are used, this is likely to require more

testing than with 12.5-style plans (as always when changing applications or introducing new features).

In a sense, compatibility mode represents a tradeoff between utilizing the full ASE 15 query processing potential

and achieving better overall performance, versus minimizing the effort for migrating to ASE 15.

Sybase expects the value of ASE 15's query processing enhancements to be sufficiently compelling that customers

will eventually want to utilize these enhancements to achieve better query performance. Just a few examples of

such enhancements are:

• hash joins, improved merge joins and "Nary-nested loop joins"

• new processing algorithms to avoid creation of worktables, and reduce tempdb usage

• improved processing of queries against star- or snowflake schemas

In this context, the main advantage of compatibility mode is that customers have more control over the precise

moment when ASE 15-specific query processing features become active in their queries and applications, thus

allowing gradual introduction of ASE 15 query processing aspects after migrating to ASE 15. Consequently, the

required testing effort can be better focused and planned than when migrating to ASE 15 without using

compatibility mode: in the latter case, full-scale ASE 15 testing would be required for all applications at the

moment of migration.

As an example, a migration strategy for moving from ASE 12.5 to ASE 15 might follow an approach like this:

1. Migrate to ASE 15 with compatibility mode enabled server-wide; do not change any SQL code except

where absolutely required (see the whitepaper "Required SQL code changes when migrating to ASE 15" -

URL on page 15).

The expectation is that, after migration to ASE 15, the majority of queries will use the same query plans,

and deliver at least the same performance, as in 12.5.

As part of this migration, Sybase still recommends to perform sufficient testing on real-life databases

under realistic workload to ensure all applications perform correctly.

2. After migrating to ASE 15 as in the previous bullet, identify ASE 15 enhancements that are beneficial for

applications, and determine how and where to introduce these in those applications.

As an example, this could take the shape of switching compatibility mode off for specific application

connections, for example those performing reporting functions, and enabling the ASE 15 allrows_dss

optimization goal (both can be performed in a login trigger if the specific application connection can be

identified). Alternatively, specific queries might be rewritten to take advantage of new ASE 15 built-in

functions, or specific tables could be semantically partitioned.

In any of these cases, these specific changes should be carefully tested, in line with best-practice testing

standards, before deploying these in the ASE 15 production environment.



3. Eventually, after incrementally moving away from compatibility mode in various applications, there may

come a point where there is no longer a worthwhile positive effect from leaving compatibility mode

enabled. In that case, consider disabling compatibility mode server-wide to take full advantage of ASE 15's

query processing features in all applications.

In summary, compatibility mode provides an alternative approach to ASE 15 migration, allowing customers to

choose the migration path most suitable for their situation.



Reference Documents

• ASE product manual "Migration Technology Guide" :

http://sybooks.sybase.com (part of the 15.0.3 documentation set)

• Technical whitepaper "Required SQL code changes when migrating to ASE 15" :

http://www.sybase.com/detail?id=1063534

• Technical whitepaper "Sybase ASE 15 Best Practices: Query Processing & Optimization" :

http://www.sybase.com/detail?id=1056243

CONTACT INFORMATION

For Europe, Middle East,

or Africa inquiries:

+(31) 34 658 2999

For Asia-Pacific inquiries:

+852 2506 8900 (Hong Kong)

For Latin America inquiries:

+770 777 3131 (Atlanta, GA)

SYBASE, INC.

WORLDWIDE HEADQUARTERS

ONE SYBASE DRIVE

DUBLIN, CA 94568-7902 USA

Tel: 1 800 8 SYBASE

www.sybase.com

Copyright © 2009 Sybase, Inc. All rights reserved. Unpublished rights reserved under U.S.

copyright laws. Sybase, and the Sybase logo are trademarks of Sybase, Inc. or its subsidiaries.

All other trademarks are the property of their respective owners. ® indicates registration in the

United States. Specifications are subject to change without notice. 2/09.