Why Can't Johnny Tune?

Embed Size (px)

Citation preview

  • 8/9/2019 Why Can't Johnny Tune?

    1/7

    Why Cant Johnny Tune?

    Mike Ault

    I often see DBAs and developers who dont know the first thing about tuning SQL. Theyask Why do I need to know that, cant Oracle tune itself? The truth of the matter is that

    Oracle is, for the most part, able to do a pretty decent job of tuning itself if it is givenenough information and the queries or tuning tasks arent too complex. However, there

    are times when Johnny must tune.

    What do I mean by enough information? Well, Oracle utilizes a cost based optimizer

    (CBO) in most releases (if you have a release prior to 10g, then you will have both rule

    and cost based optimizer) and this CBO uses what it knows about database objects inregards to size and data distributions to determine the cost of accessing the objects

    through various paths, other databases utilize this type of statistic based optimization as

    well. These statistics are mostly manually gathered in releases prior to 10g andfreshness of these statistics determines how good the decisions are that Oracle makesregarding the access paths chosen for SQL statements. So if the statistics are invalid or

    stale or not present, the optimizer will make bad choices. In addition Oracle assumes that

    the data will be equally distributed about the key values, in situations where data isskewed with more data values clustered about specific keys than being equally

    distributed, the optimizer will make bad choices unless histograms (which show the

    optimizer how data is clustered around the keys) are also provided with the databaseobject statistics. Also, in releases greater than 9i the CPU costing can come into play.

    CPU costing involves just a few basic statistics such as CPU speed, the IO timing for

    single block reads and the IO timing for multi block reads. Using these basic speedrelated statistics Oracle can then determine the cost of a CPU based path verses that of an

    IO based path and then will choose whether or not to use the best CPU or IO based path

    in its SQL access determinations. The statistics for the CPU option are gathered by usingthe DBMS_STATS.GATHER_SYSTEM_STATS call to the DBMS_STATS package.

    So it should be clear that having up-to-date statistics is a vital key to getting properoptimization out of the Oracle optimizer for SQL access paths. The other half of the

    optimization puzzle is the complexity of the SQL being optimized. The number of paths

    that the optimizer will consider when looking at the optimal access path for a given SQLstatement is determined using the OPTIMIZER_MAX_PERMUTATIONS optimization

    parameter. In versions prior to Oracle9i the OPTIMIZER_MAX_PERMUTATIONSparameter was set at 80,000, in versions after 9i it was reduced to 2000 and in 10g it mas

    made into an undocumented setting and remained at 2000. But what exactly is thisparameter?

    The number of possible paths to reach a set of destinations is stated as the factorial (n!) ofthe number of destinations. In Oracle the number of ways to process a given SQL

    statement is roughly the factorial of the number of tables in the specific statement. Now,

  • 8/9/2019 Why Can't Johnny Tune?

    2/7

    things like the number of indexes, types of indexes and other factors may increase the

    number of possible paths above this factorial but you get the picture. A factorial is

    essentially the integer product of all integers from 1 to n where n is the number ofobjects. So for a 3 table join the factorial is 1*2*3 which is 6, for a 4 table join it is

    1*2*3*4 or 24. This doesnt seem so badhowever, when you get to a 6! You reach 720

    and with 7 you get to 5040 possible paths just considering the tables in the join and nottheir indexes. Generally speaking Oracle will consider the tables join order starting from

    the left side of the FROM clause and working its way right. What this left-right ordering

    means is that with a 6 or more table join you had better place the most important tables inthe join first (on the left side of the FROM clause.) Can Oracle do that for you? Not

    really, it has no clue (generally speaking) on the relative importance of the tables relative

    to your query.

    Another area that has an immense affect on the optimizer are the settings of various

    documented and undocumented initialization parameters. If you want to see what the

    optimizer considers when performing a SQL optimization, utilize an Oracle 10054 trace,

    in my 10gR2 database, the following is the list of documented and undocumentedparameters considered by the optimizer when making path choices for a simple two-table

    join:

    Optimizer environment:optimizer_mode_hinted = falseoptimizer_features_hinted = 0.0.0parallel_execution_enabled = trueparallel_query_forced_dop = 0parallel_dml_forced_dop = 0parallel_ddl_forced_degree = 0parallel_ddl_forced_instances = 0_query_rewrite_fudge = 90optimizer_features_enable = 10.2.0.1_optimizer_search_limit = 5cpu_count = 2active_instance_count = 1parallel_threads_per_cpu = 2hash_area_size = 131072bitmap_merge_area_size = 1048576sort_area_size = 65536sort_area_retained_size = 0

    _sort_elimination_cost_ratio = 0_optimizer_block_size = 8192_sort_multiblock_read_count = 2_hash_multiblock_io_count = 0_db_file_optimizer_read_count = 16_optimizer_max_permutations = 2000pga_aggregate_target = 198656 KB_pga_max_size = 204800 KB

  • 8/9/2019 Why Can't Johnny Tune?

    3/7

    _query_rewrite_maxdisjunct = 257_smm_auto_min_io_size = 56 KB_smm_auto_max_io_size = 248 KB_smm_min_size = 198 KB_smm_max_size = 39731 KB

    _smm_px_max_size = 99328 KB_cpu_to_io = 0_optimizer_undo_cost_change = 10.2.0.1parallel_query_mode = enabledparallel_dml_mode = disabledparallel_ddl_mode = enabledoptimizer_mode = all_rowssqlstat_enabled = false_optimizer_percent_parallel = 101_always_anti_join = choose_always_semi_join = choose_optimizer_mode_force = true_partition_view_enabled = true_always_star_transformation = false_query_rewrite_or_error = false_hash_join_enabled = truecursor_sharing = exact_b_tree_bitmap_plans = truestar_transformation_enabled = false_optimizer_cost_model = choose_new_sort_cost_estimate = true_complex_view_merging = true_unnest_subquery = true

    _eliminate_common_subexpr = true_pred_move_around = true_convert_set_to_join = false_push_join_predicate = true_push_join_union_view = true_fast_full_scan_enabled = true_optim_enhance_nnull_detection = true_parallel_broadcast_enabled = true_px_broadcast_fudge_factor = 100_ordered_nested_loop = true_no_or_expansion = false

    optimizer_index_cost_adj = 100optimizer_index_caching = 0_system_index_caching = 0_disable_datalayer_sampling = falsequery_rewrite_enabled = truequery_rewrite_integrity = enforced_query_cost_rewrite = true_query_rewrite_2 = true

  • 8/9/2019 Why Can't Johnny Tune?

    4/7

    _query_rewrite_1 = true_query_rewrite_expression = true_query_rewrite_jgmigrate = true_query_rewrite_fpc = true_query_rewrite_drj = true

    _full_pwise_join_enabled = true_partial_pwise_join_enabled = true_left_nested_loops_random = true_improved_row_length_enabled = true_index_join_enabled = true_enable_type_dep_selectivity = true_improved_outerjoin_card = true_optimizer_adjust_for_nulls = true_optimizer_degree = 0_use_column_stats_for_function = true_subquery_pruning_enabled = true_subquery_pruning_mv_enabled = false_or_expand_nvl_predicate = true_like_with_bind_as_equality = false_table_scan_cost_plus_one = true_cost_equality_semi_join = true_default_non_equality_sel_check = true_new_initial_join_orders = true_oneside_colstat_for_equijoins = true_optim_peek_user_binds = true_minimal_stats_aggregation = true_force_temptables_for_gsets = falseworkarea_size_policy = auto

    _smm_auto_cost_enabled = true_gs_anti_semi_join_allowed = true_optim_new_default_join_sel = trueoptimizer_dynamic_sampling = 2_pre_rewrite_push_pred = true_optimizer_new_join_card_computation = true_union_rewrite_for_gs = yes_gset_mvs_generalized_pruning_enabled = true_optim_adjust_for_part_skews = true_force_datefold_trunc = falsestatistics_level = typical

    _optimizer_system_stats_usage = trueskip_unusable_indexes = true_remove_aggr_subquery = true_optimizer_push_down_distinct = 0_dml_monitoring_enabled = true_optimizer_undo_changes = false_predicate_elimination_enabled = true_nested_loop_fudge = 100

  • 8/9/2019 Why Can't Johnny Tune?

    5/7

    _project_view_columns = true_local_communication_costing_enabled = true_local_communication_ratio = 50_query_rewrite_vop_cleanup = true_slave_mapping_enabled = true

    _optimizer_cost_based_transformation = linear_optimizer_mjc_enabled = true_right_outer_hash_enable = true_spr_push_pred_refspr = true_optimizer_cache_stats = false_optimizer_cbqt_factor = 50_optimizer_squ_bottomup = true_fic_area_size = 131072_optimizer_skip_scan_enabled = true_optimizer_cost_filter_pred = false_optimizer_sortmerge_join_enabled = true_optimizer_join_sel_sanity_check = true_mmv_query_rewrite_enabled = true_bt_mmv_query_rewrite_enabled = true_add_stale_mv_to_dependency_list = true_distinct_view_unnesting = false_optimizer_dim_subq_join_sel = true_optimizer_disable_strans_sanity_checks = 0_optimizer_compute_index_stats = true_push_join_union_view2 = true_optimizer_ignore_hints = false_optimizer_random_plan = 0_query_rewrite_setopgrw_enable = true

    _optimizer_correct_sq_selectivity = true_disable_function_based_index = false_optimizer_join_order_control = 3_optimizer_cartesian_enabled = true_optimizer_starplan_enabled = true_extended_pruning_enabled = true_optimizer_push_pred_cost_based = true_sql_model_unfold_forloops = run_time_enable_dml_lock_escalation = false_bloom_filter_enabled = true_update_bji_ipdml_enabled = 0

    _optimizer_extended_cursor_sharing = udo_dm_max_shared_pool_pct = 1_optimizer_cost_hjsmj_multimatch = true_optimizer_transitivity_retain = true_px_pwg_enabled = trueoptimizer_secure_view_merging = true_optimizer_join_elimination_enabled = trueflashback_table_rpi = non_fbt

  • 8/9/2019 Why Can't Johnny Tune?

    6/7

    _optimizer_cbqt_no_size_restriction = true_optimizer_enhanced_filter_push = true_optimizer_filter_pred_pullup = true_rowsrc_trace_level = 0_simple_view_merging = true

    _optimizer_rownum_pred_based_fkr = true_optimizer_better_inlist_costing = all_optimizer_self_induced_cache_cost = false_optimizer_min_cache_blocks = 10_optimizer_or_expansion = depth_optimizer_order_by_elimination_enabled = true_optimizer_outer_to_anti_enabled = true_selfjoin_mv_duplicates = true_dimension_skip_null = true_force_rewrite_enable = false_optimizer_star_tran_in_with_clause = true_optimizer_complex_pred_selectivity = true_gby_hash_aggregation_enabled = true

    All-in-all there are 33 parameters (documented) that the DBA has control over and about

    170 undocumented parameters that have an affect on how SQL is optimized. In somereleases Oracle has made changes to the optimizer which resulted in bad SQL

    optimization decisions (such as the change in 9i setting the parameter

    _unnest_subqueries from false to true and causing havoc in the Oracle community untilit was sorted out properly.) Another problem was with the optimization of the skip-scan

    access path, in most versions of 9i there was a bug in the way the path was being costed

    which resulted in this very good path not being chosen by the optimizer, forcing the DBAor developer to have to use a hint to force the execution plan to utilize this pathway.

    So what have we considered so far? First, we have determined that for simple queries (6

    tables or less, in fact I would hazard 5 tables or less is probably a better rule of thumbbased on also considering indexes) the Oracle optimizer will do a bang-up job of

    optimization provided it is given the proper statistics and has the proper base objects to

    work from. This is assuming Oracle hasnt shot us in the foot with a bad undocumentedsetting or an internal costing bug. What do I mean by base objects?

    Base objects are: Tables, Table Partitions, Indexes (b-Tree, function-based, Bitmap,bitmap-join, special) , Index Partitions and Index-Only-Tables. If there are no proper

    indexes, a good execution plan is impossible. If a table is too large and has not been

    partitioned, a good plan is impossible, if the proper special indexes havent been applied,then a proper plan is impossible.

    SQL tuning is more than just looking at a SQL statement. SQL tuning involves making

    sure that the optimizer has what it needs to do its job (statistics, histograms, CPU costingstatistics, proper optimizer settings) and in the case where the statement is too complex

    (>5-6 tables) you will need to do some manual tuning to get optimal performance form

    your database. In addition the DBA and developer have to be aware of the special indexes

  • 8/9/2019 Why Can't Johnny Tune?

    7/7

    available and when to apply them to a tuning situation. A prime example for indexes is

    the proper use of function-based indexes. In 10g and later versions of 9i Oracle will use a

    less efficient index scanning technique if a function is applied to the left-hand side of acomparison statement in the WHERE clause (to a column value), in earlier releases, a full

    table scan will result, while placing a function-based index will result in a more optimal

    plan.

    The DBA and developer have to be aware of the benefits and limitations of the CBO and

    be able to make proper SQL tuning decisions based on that knowledge.

    So, back to our original question: Why cant Johnny tune? The answer: He has not

    learned any better!