1. The Data Computing Division of EMC Greenplum Database 4.1Administrator GuideP/N: 300-012-428 Rev: A03 2. Copyright 2011 EMC Corporation. All rights reserved.EMC believes the information in this publication is accurate as of its publication date. The information is subject tochange without notice.THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONSOR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLYDISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.Use, copying, and distribution of any EMC software described in this publication requires an applicable softwarelicense.For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.comAll other trademarks used herein are the property of their respective owners. 3. Greenplum Database Administrator Guide 4.1 - ContentsGreenplum Database Administrator Guide 4.1 - Contents Preface ............................................................................................... 1About This Guide .............................................................................. 1Document Conventions .................................................................... 2 Text Conventions ........................................................................ 2 Command Syntax Conventions ................................................... 3Getting Support ............................................................................... 3 Product information .................................................................... 3 Technical support ....................................................................... 4Section I: Introduction to Greenplum Chapter 1: About the Greenplum Architecture ........................ 6About the Greenplum Master ............................................................ 7About the Greenplum Segments....................................................... 7About the Greenplum Interconnect .................................................. 7About Redundancy and Failover in Greenplum Database .................. 8 About Segment Mirroring ............................................................ 8 About Master Mirroring ............................................................... 9 About Interconnect Redundancy ................................................. 9About Parallel Data Loading ............................................................10About Management and Monitoring .................................................10 Chapter 2: About Distributed Databases .................................12Understanding How Data is Stored ..................................................12Understanding Greenplum Distribution Policies................................13 Chapter 3: Summary of Greenplum Features .........................14Greenplum SQL Standard Conformance ..........................................14 Core SQL Conformance ..............................................................14 SQL 1992 Conformance .............................................................15 SQL 1999 Conformance .............................................................16 SQL 2003 Conformance .............................................................16 SQL 2008 Conformance .............................................................17Greenplum and PostgreSQL Compatibility .......................................18 Chapter 4: About Greenplum Query Processing ....................25Understanding Query Planning and Dispatch ...................................25Understanding Greenplum Query Plans ...........................................26Understanding Parallel Query Execution ..........................................27Section II: Access Control and Security Chapter 5: Managing Roles and Privileges ..............................30Security Best Practices for Roles and Privileges ...............................30Creating New Roles (Users) .............................................................31 Altering Role Attributes ..............................................................31Creating Groups (Role Membership) ................................................32Managing Object Privileges .............................................................33 Simulating Row and Column Level Access Control .....................34Encrypting Data ..............................................................................34Table of Contentsiii 4. Greenplum Database Administrator Guide 4.1 - Contents Chapter 6: Configuring Client Authentication .........................36Allowing Connections to Greenplum Database .................................36Editing the pg_hba.conf File.......................................................37Limiting Concurrent Connections .....................................................38Encrypting Client/Server Connections .............................................39 Chapter 7: Accessing the Database ...........................................41Establishing a Database Session .....................................................41Supported Client Applications ..........................................................42 Greenplum Database Client Applications ....................................43 pgAdmin III for Greenplum Database ........................................44 Database Application Interfaces .................................................47 Third-Party Client Tools .............................................................48Troubleshooting Connection Problems .............................................49 Chapter 8: Managing Workload and Resources .....................50Overview of Greenplum Workload Management ..............................50 How Resource Queues Work in Greenplum Database .................50 Steps to Enable Workload Management .....................................54Configuring Workload Management .................................................55Creating Resource Queues ..............................................................56 Creating Queues with an Active Query Limit ..............................56 Creating Queues with Memory Limits .........................................57 Creating Queues with a Query Planner Cost Limits ....................57 Setting Priority Levels ................................................................58Assigning Roles (Users) to a Resource Queue..................................59 Removing a Role from a Resource Queue ..................................59Modifying Resource Queues.............................................................60 Altering a Resource Queue.........................................................60 Dropping a Resource Queue ......................................................60Checking Resource Queue Status ....................................................60 Viewing Queued Statements and Resource Queue Status ..........61 Viewing Resource Queue Statistics ............................................61 Viewing the Roles Assigned to a Resource Queue ......................61 Viewing the Waiting Queries for a Resource Queue ....................62 Clearing a Waiting Statement From a Resource Queue ..............62 Viewing the Priority of Active Statements ..................................63 Resetting the Priority of an Active Statement.............................63Section III: Database Administration Chapter 9: Defining Database Objects......................................65Creating and Managing Databases ..................................................65 About Template Databases ........................................................65 Creating a Database ..................................................................65 Viewing the List of Databases ....................................................66 Altering a Database ...................................................................66 Dropping a Database .................................................................66Creating and Managing Tablespaces ................................................67 Creating a Filespace...................................................................67 Creating a Tablespace ...............................................................68Table of Contents iv 5. Greenplum Database Administrator Guide 4.1 - Contents Using a Tablespace to Store Database Objects ..........................68 Viewing Existing Tablespaces and Filespaces .............................69 Dropping Tablespaces and Filespaces ........................................69Creating and Managing Schemas.....................................................69 The Default Public Schema .......................................................70 Creating a Schema ....................................................................70 Schema Search Paths ................................................................70 Dropping a Schema ...................................................................71 System Schemas .......................................................................71Creating and Managing Tables ........................................................72 Creating a Table ........................................................................72 Altering a Table .........................................................................79 Dropping a Table .......................................................................80Partitioning Large Tables .................................................................80 Understanding Table Partitioning in Greenplum Database ..........80 Deciding on a Table Partitioning Strategy ..................................81 Creating Partitioned Tables ........................................................82 Loading Partitioned Tables .........................................................86 Verifying Your Partition Strategy ................................................86 Viewing Your Partition Design ....................................................87 Maintaining Partitioned Tables ...................................................87Creating and Using Sequences ........................................................91 Creating a Sequence..................................................................91 Using a Sequence ......................................................................91 Altering a Sequence ...................................................................92 Dropping a Sequence.................................................................92Using Indexes in Greenplum Database ............................................92 Index Types ...............................................................................94 Creating an Index ......................................................................96 Examining Index Usage .............................................................96 Managing Indexes .....................................................................97 Dropping an Index .....................................................................97Creating and Managing Views..........................................................97 Creating Views ...........................................................................97 Dropping Views..........................................................................97 Chapter 10: Managing Data .........................................................99About Concurrency Control in Greenplum Database ........................99Inserting New Rows ...................................................................... 100Updating Existing Rows ................................................................. 101Deleting Rows ............................................................................... 101 Truncating a Table ................................................................... 102Working With Transactions ............................................................ 102 Transaction Isolation Levels ..................................................... 102Vacuuming the Database .............................................................. 103 Configuring the Free Space Map .............................................. 103 Chapter 11: Querying Data ........................................................ 105Defining Queries ........................................................................... 105 SQL Lexicon ............................................................................. 105Table of Contents v 6. Greenplum Database Administrator Guide 4.1 - Contents SQL Value Expressions ............................................................ 105Using Functions and Operators ...................................................... 114 Using Functions in Greenplum Database .................................. 114 User-Defined Functions ............................................................ 115 Built-in Functions and Operators .............................................. 115Query Profiling .............................................................................. 130 Reading EXPLAIN Output ......................................................... 131 Reading EXPLAIN ANALYZE Output .......................................... 132 What to Look for in a Query Plan ............................................. 133 Chapter 12: Loading and Unloading Data .............................. 135Greenplum Database Loading Tools Overview ............................... 135 About External Tables .............................................................. 135 About gpload ........................................................................... 136 About COPY .............................................................................. 136Loading Data into Greenplum Database ........................................ 136 Accessing File-Based External Tables ....................................... 137Defining External Tables - Examples ............................................. 139 Using the Greenplum Parallel File Server (gpfdist) ................... 141 Using Hadoop Distributed File System (HDFS) Tables .............. 144 Creating and Using Web External Tables .................................. 146 Loading Data Using an External Table ...................................... 148 Handling Load Errors ............................................................... 148Loading Data from Greenplum Database ....................................... 150 Loading Data with gpload ........................................................ 150 Loading Data with the gphdfs Protocol ..................................... 151 Loading Data with COPY .......................................................... 152 Data Loading Performance Tips ............................................... 152Unloading Data from Greenplum Database .................................... 153 Defining a File-Based Writable External Table .......................... 153 Defining a Command-Based Writable External Web Table ........ 155 Unloading Data Using a Writable External Table ...................... 156 Unloading Data Using COPY ..................................................... 157 Readable External Tables and Query Planner Statistics ............ 157Formatting Data Files .................................................................... 157 Formatting Rows...................................................................... 157 Formatting Columns ................................................................ 158 Representing NULL Values ....................................................... 158 Escaping .................................................................................. 158 Character Encoding.................................................................. 160Section IV: System Administration Chapter 13: Starting and Stopping Greenplum .................... 162Overview ....................................................................................... 162Starting Greenplum Database ....................................................... 162 Restarting Greenplum Database .............................................. 162 Uploading Configuration File Changes Only .............................. 163 Starting the Master in Maintenance Mode ................................ 163Stopping Greenplum Database ...................................................... 163Table of Contentsvi 7. Greenplum Database Administrator Guide 4.1 - Contents Chapter 14: Configuring Your Greenplum System .............. 165About Greenplum Master and Local Parameters ............................ 165Setting Configuration Parameters .................................................. 165 Setting a Local Configuration Parameter .................................. 166 Setting a Master Configuration Parameter ............................... 166Viewing Settings of Server Configuration Parameters ................... 167Configuration Parameter Categories .............................................. 167 Connection and Authentication Parameters .............................. 168 System Resource Consumption Parameters ............................. 169 Query Tuning Parameters ........................................................ 170 Error Reporting and Logging Parameters ................................. 172 System Monitoring Parameters ................................................ 172 Runtime Statistics Collection Parameters ................................. 173 Automatic Statistics Collection Parameters .............................. 173 Client Connection Default Parameters ...................................... 174 Lock Management Parameters ................................................. 174 Workoad Management Parameters .......................................... 174 External Table Parameters ....................................................... 175 Append-Only Table Parameters ................................................ 175 Database and Tablespace/Filespace Parameters ...................... 175 Past PostgreSQL Version Compatibility Parameters .................. 175 Greenplum Array Configuration Parameters ............................. 175 Chapter 15: Enabling High Availability Features ................. 177Overview of High Availability in Greenplum Database.................... 177 Overview of Segment Mirroring ............................................... 177 Overview of Master Mirroring ................................................... 178 Overview of Fault Detection and Recovery ............................... 179Enabling Mirroring in Greenplum Database .................................... 180 Enabling Segment Mirroring ..................................................... 180 Enabling Master Mirroring ........................................................ 181Knowing When a Segment is Down ............................................... 182 Enabling Alerts and Notifications .............................................. 182 Checking for Failed Segments .................................................. 182 Checking the Log Files ............................................................. 183Recovering a Failed Segment ........................................................ 183 Recovering From Segment Failures .......................................... 184Recovering a Failed Master............................................................ 187 Restoring Master Mirroring After a Recovery ............................ 188 Chapter 16: Backing Up and Restoring Databases .............. 190Overview of Backup and Restore Operations ................................. 190 About Parallel Backups ............................................................ 190 About Non-Parallel Backups ..................................................... 191 About Parallel Restores ............................................................ 191 About Non-Parallel Restores .................................................... 192Backing Up a Database ................................................................. 192 Backing Up a Database with gp_dump ..................................... 193 Automating Parallel Backups with gpcrondump ........................ 194Restoring From Parallel Backup Files ............................................. 195Table of Contentsvii 8. Greenplum Database Administrator Guide 4.1 - ContentsRestoring a Database with gp_restore ..................................... 195Restoring a Database Using gpdbrestore ................................. 197Restoring to a Different Greenplum System Configuration ....... 197 Chapter 17: Expanding a Greenplum System ....................... 199Planning Greenplum System Expansion ......................................... 199System Expansion Overview .................................................... 199System Expansion Checklist .................................................... 201Planning New Hardware Platforms ........................................... 202Planning Initialization of New Segments .................................. 202Planning Table Redistribution ................................................... 203Preparing and Adding Nodes ......................................................... 206Adding New Nodes to the Trusted Host Environment ............... 206Verifying OS Settings ............................................................... 208Validating Disk I/O and Memory Bandwidth ............................. 208Integrating New Hardware into the System ............................. 209Initializing New Segments ............................................................. 209Creating an Input File for System Expansion ........................... 209Running gpexpand to Initialize New Segments ........................ 212Rolling Back an Failed Expansion Setup ................................... 213Redistributing Tables ..................................................................... 213Ranking Tables for Redistribution ............................................ 213Redistributing Tables Using gpexpand...................................... 214Monitoring Table Redistribution................................................ 214Removing the Expansion Schema .................................................. 215 Chapter 18: Monitoring a Greenplum System....................... 216Monitoring Database Activity and Performance .............................. 216Monitoring System State ............................................................... 216 Enabling System Alerts and Notifications ................................. 217 Checking System State ............................................................ 223 Checking Disk Space Usage ..................................................... 224 Checking for Data Distribution Skew ........................................ 225 Viewing Metadata Information about Database Objects ........... 226Viewing the Database Server Log Files .......................................... 227 Log File Format ........................................................................ 227 Searching the Greenplum Database Server Log Files ............... 228Using gp_toolkit ............................................................................ 228 Chapter 19: Routine System Maintenance Tasks................. 230Routine Vacuum and Analyze ........................................................ 230 Transaction ID Management .................................................... 230 System Catalog Maintenance ................................................... 230 Vacuum and Analyze for Query Optimization ........................... 231Routine Reindexing ....................................................................... 231Managing Greenplum Database Log Files ...................................... 232 Database Server Log Files ....................................................... 232 Management Utility Log Files ................................................... 232Table of Contentsviii 9. Greenplum Database Administrator Guide 4.1 - ContentsSection V: Performance Tuning Chapter 20: Defining Database Performance ....................... 234Understanding the Performance Factors ........................................ 234 System Resources ................................................................... 234 Workload ................................................................................. 234 Throughput .............................................................................. 234 Contention ............................................................................... 235 Optimization ............................................................................ 235Determining Acceptable Performance ............................................ 235 Baseline Hardware Performance .............................................. 235 Performance Benchmarks ........................................................ 235 Chapter 21: Common Causes of Performance Issues......... 237Identifying Hardware and Segment Failures .................................. 237Managing Workload ....................................................................... 238Avoiding Contention ...................................................................... 238Maintaining Database Statistics ..................................................... 238 Identifying Statistics Problems in Query Plans ......................... 238 Tuning Statistics Collection ...................................................... 239Optimizing Data Distribution ......................................................... 239Optimizing Your Database Design.................................................. 239 Greenplum Database Maximum Limits ..................................... 240 Chapter 22: Investigating a Performance Problem ............ 241Checking System State ................................................................. 241Checking Database Activity ........................................................... 241 Checking for Active Sessions (Workload) ................................. 241 Checking for Locks (Contention) .............................................. 241 Checking Query Status and System Utilization ......................... 242Troubleshooting Problem Queries .................................................. 242Investigating Error Messages ........................................................ 242 Gathering Information for Greenplum Support ......................... 243Section VI: Extending Greenplum Database Chapter 23: Using Greenplum MapReduce ............................ 245About Greenplum MapReduce ....................................................... 245 The Basics of MapReduce ......................................................... 245 How Greenplum MapReduce Works .......................................... 246Programming Greenplum MapReduce ............................................ 247 Defining Inputs ........................................................................ 247 Defining Map Functions ............................................................ 250 Defining Reduce Functions ....................................................... 252 Defining Outputs...................................................................... 255 Defining Tasks ......................................................................... 256 Putting Together a Complete MapReduce Specification ............ 257Submitting MapReduce Jobs for Execution .................................... 257Troubleshooting Problems with MapReduce Jobs ........................... 258 Language Does Not Exist ......................................................... 258 Generic Python Iterator Error .................................................. 259 Function Defined Using Wrong MODE ....................................... 259Table of Contents ix 10. Greenplum Database Administrator Guide 4.1 - ContentsSection VII: References Appendix A: SQL Command Reference....................................... 264SQL Syntax Summary ................................................................... 266ABORT .......................................................................................... 293ALTER AGGREGATE ....................................................................... 294ALTER CONVERSION ..................................................................... 296ALTER DATABASE.......................................................................... 297ALTER DOMAIN ............................................................................. 299ALTER EXTERNAL TABLE ............................................................... 301ALTER FILESPACE ......................................................................... 304ALTER FUNCTION .......................................................................... 305ALTER GROUP ............................................................................... 308ALTER INDEX ................................................................................ 309ALTER LANGUAGE ......................................................................... 311ALTER OPERATOR ......................................................................... 312ALTER OPERATOR CLASS .............................................................. 313ALTER RESOURCE QUEUE ............................................................. 314ALTER ROLE .................................................................................. 317ALTER SCHEMA ............................................................................. 321ALTER SEQUENCE ......................................................................... 322ALTER TABLE ................................................................................ 325ALTER TABLESPACE ...................................................................... 337ALTER TRIGGER ............................................................................ 338ALTER TYPE ................................................................................... 339ALTER USER .................................................................................. 340ANALYZE ....................................................................................... 341BEGIN ........................................................................................... 343CHECKPOINT ................................................................................. 345CLOSE ........................................................................................... 346CLUSTER ....................................................................................... 347COMMENT ..................................................................................... 350COMMIT ........................................................................................ 353COPY ............................................................................................. 354CREATE AGGREGATE ..................................................................... 362CREATE CAST ................................................................................ 366CREATE CONVERSION ................................................................... 369CREATE DATABASE ....................................................................... 371CREATE DOMAIN ........................................................................... 373CREATE EXTERNAL TABLE ............................................................. 375CREATE FUNCTION........................................................................ 383CREATE GROUP ............................................................................. 389CREATE INDEX .............................................................................. 390CREATE LANGUAGE ....................................................................... 394CREATE OPERATOR ....................................................................... 398CREATE OPERATOR CLASS ............................................................ 403CREATE RESOURCE QUEUE ........................................................... 408CREATE ROLE ................................................................................ 412CREATE RULE ................................................................................ 417Table of Contentsx 11. Greenplum Database Administrator Guide 4.1 - ContentsCREATE SCHEMA ........................................................................... 420CREATE SEQUENCE ....................................................................... 422CREATE TABLE .............................................................................. 426CREATE TABLE AS ......................................................................... 437CREATE TABLESPACE .................................................................... 441CREATE TRIGGER .......................................................................... 443CREATE TYPE ................................................................................ 446CREATE USER ............................................................................... 453CREATE VIEW ............................................................................... 454DEALLOCATE ................................................................................. 457DECLARE ....................................................................................... 458DELETE ......................................................................................... 461DROP AGGREGATE ........................................................................ 464DROP CAST ................................................................................... 465DROP CONVERSION ...................................................................... 466DROP DATABASE........................................................................... 467DROP DOMAIN .............................................................................. 468DROP EXTERNAL TABLE ................................................................ 469DROP FILESPACE .......................................................................... 470DROP FUNCTION ........................................................................... 471DROP GROUP ................................................................................ 473DROP INDEX ................................................................................. 474DROP LANGUAGE .......................................................................... 475DROP OPERATOR .......................................................................... 476DROP OPERATOR CLASS ............................................................... 478DROP OWNED ............................................................................... 480DROP RESOURCE QUEUE .............................................................. 482DROP ROLE ................................................................................... 484DROP RULE ................................................................................... 485DROP SCHEMA .............................................................................. 486DROP SEQUENCE .......................................................................... 487DROP TABLE ................................................................................. 488DROP TABLESPACE ....................................................................... 489DROP TRIGGER ............................................................................. 490DROP TYPE .................................................................................... 491DROP USER ................................................................................... 492DROP VIEW ................................................................................... 493END .............................................................................................. 494EXECUTE ....................................................................................... 495EXPLAIN ........................................................................................ 496FETCH ........................................................................................... 499GRANT .......................................................................................... 503INSERT ......................................................................................... 508LOAD ............................................................................................ 510LOCK ............................................................................................. 511MOVE ............................................................................................ 515PREPARE ....................................................................................... 517REASSIGN OWNED ........................................................................ 520REINDEX ....................................................................................... 521Table of Contentsxi 12. Greenplum Database Administrator Guide 4.1 - ContentsRELEASE SAVEPOINT .................................................................... 523RESET ........................................................................................... 524REVOKE ........................................................................................ 525ROLLBACK ..................................................................................... 528ROLLBACK TO SAVEPOINT ............................................................ 529SAVEPOINT ................................................................................... 531SELECT ......................................................................................... 533SELECT INTO ................................................................................ 549SET ............................................................................................... 551SET ROLE ...................................................................................... 553SET SESSION AUTHORIZATION .................................................... 555SET TRANSACTION ....................................................................... 557SHOW ........................................................................................... 560START TRANSACTION ................................................................... 561TRUNCATE .................................................................................... 563UPDATE ......................................................................................... 564VACUUM ........................................................................................ 568VALUES ......................................................................................... 571 Appendix B: Management Utility Reference................................ 574Backend Server Programs ............................................................. 575Management Utility Summary ....................................................... 576gp_dump....................................................................................... 597gp_restore .................................................................................... 602gpaddmirrors ................................................................................ 606gpactivatestandby ......................................................................... 611gpbitmapreindex ........................................................................... 614gpcheck ........................................................................................ 616gpcheckperf .................................................................................. 618gpconfig ........................................................................................ 622gpcrondump .................................................................................. 626gpdbrestore .................................................................................. 632gpdeletesystem ............................................................................. 635gpdetective ................................................................................... 637gpexpand ...................................................................................... 640gpfdist........................................................................................... 644gpfilespace .................................................................................... 647gpinitstandby ................................................................................ 650gpinitsystem ................................................................................. 653gpload ........................................................................................... 660gplogfilter...................................................................................... 672gpmapreduce ................................................................................ 675gpmigrator .................................................................................... 677gpmigrator_mirror ........................................................................ 680gpperfmon_install ......................................................................... 683gprecoverseg ................................................................................ 685gpscp ............................................................................................ 690gpseginstall ................................................................................... 692gpsetupsanfailover ........................................................................ 695Table of Contentsxii 13. Greenplum Database Administrator Guide 4.1 - Contentsgpsnmpd ....................................................................................... 697gpssh ............................................................................................ 700gpssh-exkeys ................................................................................ 702gpstart .......................................................................................... 705gpstate.......................................................................................... 708gpstop ........................................................................................... 712gpsys1 .......................................................................................... 715 Appendix C: Client Utility Reference ............................................ 716Client Utility Summary .................................................................. 718clusterdb ....................................................................................... 727createdb ........................................................................................ 729createlang ..................................................................................... 731createuser ..................................................................................... 733dropdb .......................................................................................... 736gp_db_interfaces .......................................................................... 738droplang........................................................................................ 739dropuser ....................................................................................... 741ecpg .............................................................................................. 743pg_config ...................................................................................... 745pg_dump....................................................................................... 748pg_dumpall ................................................................................... 755pg_restore .................................................................................... 759psql ............................................................................................... 764reindexdb ...................................................................................... 787vacuumdb ..................................................................................... 789 Appendix D: Server Configuration Parameters .......................... 792add_missing_from ......................................................................... 794application_name .......................................................................... 794array_nulls .................................................................................... 794authentication_timeout ................................................................. 794backslash_quote ........................................................................... 794block_size ..................................................................................... 794bonjour_name ............................................................................... 794check_function_bodies .................................................................. 794client_encoding ............................................................................. 795client_min_messages .................................................................... 795cpu_index_tuple_cost ................................................................... 795cpu_operator_cost ........................................................................ 795cpu_tuple_cost .............................................................................. 795cursor_tuple_fraction .................................................................... 795custom_variable_classes ............................................................... 795DateStyle ...................................................................................... 795db_user_namespace ..................................................................... 796deadlock_timeout .......................................................................... 796debug_assertions .......................................................................... 796debug_pretty_print ....................................................................... 796debug_print_parse ........................................................................ 796debug_print_plan .......................................................................... 796Table of Contentsxiii 14. Greenplum Database Administrator Guide 4.1 - Contentsdebug_print_prelim_plan .............................................................. 796debug_print_rewritten................................................................... 796debug_print_slice_table ................................................................ 796default_statistics_target ................................................................ 796default_tablespace ........................................................................ 797default_transaction_isolation......................................................... 797default_transaction_read_only ...................................................... 797dynamic_library_path.................................................................... 797effective_cache_size ..................................................................... 797enable_bitmapscan ....................................................................... 797enable_groupagg .......................................................................... 797enable_hashagg ............................................................................ 798enable_hashjoin ............................................................................ 798enable_indexscan .......................................................................... 798enable_mergejoin ......................................................................... 798enable_nestloop ............................................................................ 798enable_seqscan ............................................................................. 798enable_sort ................................................................................... 798enable_tidscan .............................................................................. 798escape_string_warning.................................................................. 798explain_pretty_print ...................................................................... 799extra_float_digits .......................................................................... 799from_collapse_limit ....................................................................... 799gp_adjust_selectivity_for_outerjoins ............................................. 799gp_analyze_relative_error ............................................................. 799gp_autostats_mode ...................................................................... 800gp_autostats_on_change_threshold .............................................. 800gp_cached_segworkers_threshold ................................................. 800gp_command_count ...................................................................... 800gp_connectemc_mode .................................................................. 801gp_connections_per_thread .......................................................... 801gp_content .................................................................................... 801gp_dbid ......................................................................................... 801gp_debug_linger ........................................................................... 801gp_email_from .............................................................................. 801gp_email_smtp_password ............................................................. 801gp_email_smtp_server .................................................................. 801gp_email_smtp_userid .................................................................. 801gp_email_to .................................................................................. 802gp_enable_adaptive_nestloop ....................................................... 802gp_enable_agg_distinct ................................................................ 802gp_enable_agg_distinct_pruning ................................................... 802gp_enable_direct_dispatch ............................................................ 802gp_enable_fallback_plan ............................................................... 802gp_enable_fast_sri ........................................................................ 802gp_enable_gpperfmon .................................................................. 803gp_enable_groupext_distinct_gather ............................................ 803gp_enable_groupext_distinct_pruning ........................................... 803gp_enable_multiphase_agg ........................................................... 803Table of Contents xiv 15. Greenplum Database Administrator Guide 4.1 - Contentsgp_enable_predicate_propagation ................................................. 803gp_enable_preunique .................................................................... 803gp_enable_sequential_window_plans ............................................ 803gp_enable_sort_distinct ................................................................ 803gp_enable_sort_limit..................................................................... 804gp_external_enable_exec.............................................................. 804gp_external_grant_privileges ........................................................ 804gp_external_max_segs ................................................................. 804gp_fts_probe_interval ................................................................... 804gp_fts_probe_threadcount ............................................................ 804gp_fts_probe_timeout ................................................................... 804gp_gpperfmon_send_interval ........................................................ 804gp_hashjoin_tuples_per_bucket .................................................... 805gp_interconnect_hash_multiplier................................................... 805gp_interconnect_queue_depth ...................................................... 805gp_interconnect_setup_timeout .................................................... 805gp_interconnect_type.................................................................... 805gp_log_format .............................................................................. 805gp_max_csv_line_length ............................................................... 805gp_max_databases ....................................................................... 806gp_max_filespaces ........................................................................ 806gp_max_local_distributed_cache ................................................... 806gp_max_packet_size ..................................................................... 806gp_max_tablespaces ..................................................................... 806gp_motion_cost_per_row .............................................................. 806gp_num_contents_in_cluster ........................................................ 806gp_reject_percent_threshold......................................................... 806gp_reraise_signal .......................................................................... 806gp_resqueue_memory_policy ........................................................ 806gp_resqueue_priority .................................................................... 806gp_resqueue_priority_cpucores_per_segment .............................. 807gp_resqueue_priority_sweeper_interval ........................................ 807gp_role ......................................................................................... 807gp_safefswritesize ......................................................................... 807gp_segment_connect_timeout ...................................................... 807gp_segments_for_planner ............................................................. 807gp_session_id ............................................................................... 808gp_set_proc_affinity ..................................................................... 808gp_set_read_only ......................................................................... 808gp_snmp_community .................................................................... 808gp_snmp_monitor_address ........................................................... 808gp_snmp_use_inform_or_trap ...................................................... 808gp_statistics_pullup_from_child_partition ..................................... 808gp_statistics_use_fkeys ................................................................ 808gp_vmem_idle_resource_timeout ................................................. 808gp_vmem_protect_limit ................................................................ 809gp_vmem_protect_segworker_cache_limit .................................... 809gp_workfile_checksumming .......................................................... 809gp_workfile_compress_algorithm .................................................. 809Table of Contentsxv 16. Greenplum Database Administrator Guide 4.1 - Contentsgpperfmon_port ............................................................................ 809integer_datetimes ......................................................................... 809IntervalStyle ................................................................................. 810join_collapse_limit ........................................................................ 810krb_caseins_users ......................................................................... 810krb_server_keyfile ........................................................................ 810krb_srvname ................................................................................. 810lc_collate ....................................................................................... 810lc_ctype ........................................................................................ 810lc_messages ................................................................................. 811lc_monetary .................................................................................. 811lc_numeric .................................................................................... 811lc_time .......................................................................................... 811listen_addresses ........................................................................... 811local_preload_libraries .................................................................. 811log_autostats ................................................................................ 811log_connections ............................................................................ 812log_disconnections ........................................................................ 812log_dispatch_stats ........................................................................ 812log_duration .................................................................................. 812log_error_verbosity ....................................................................... 812log_executor_stats ........................................................................ 812log_hostname ............................................................................... 812log_min_duration_statement ........................................................ 812log_min_error_statement .............................................................. 813log_min_messages ........................................................................ 813log_parser_stats ........................................................................... 813log_planner_stats.......................................................................... 813log_rotation_age ........................................................................... 813log_rotation_size ........................................................................... 813log_statement ............................................................................... 813log_statement_stats ..................................................................... 814log_timezone ................................................................................ 814log_truncate_on_rotation .............................................................. 814maintenance_work_mem .............................................................. 814max_appendonly_tables................................................................ 814max_connections .......................................................................... 815max_files_per_process .................................................................. 815max_fsm_pages ............................................................................ 815max_fsm_relations ........................................................................ 815max_function_args ....................................................................... 815max_identifier_length ................................................................... 815max_index_keys ........................................................................... 815max_locks_per_transaction ........................................................... 816max_prepared_transactions .......................................................... 816max_resource_portals_per_transaction ......................................... 816max_resource_queues .................................................................. 816max_stack_depth .......................................................................... 816max_statement_mem ................................................................... 817Table of Contentsxvi 17. Greenplum Database Administrator Guide 4.1 - Contentsmax_work_mem ........................................................................... 817password_encryption .................................................................... 817pljava_classpath ........................................................................... 817pljava_statement_cache_size ........................................................ 817pljava_release_lingering_savepoints ............................................. 817pljava_vmoptions .......................................................................... 818port ............................................................................................... 818random_page_cost........................................................................ 818regex_flavor .................................................................................. 818resource_cleanup_gangs_on_wait ................................................. 818resource_select_only..................................................................... 818search_path .................................................................................. 818seq_page_cost .............................................................................. 819server_encoding ............................................................................ 819server_version .............................................................................. 819server_version_num ..................................................................... 819shared_buffers .............................................................................. 819shared_preload_libraries ............................................................... 819ssl ................................................................................................. 819ssl_ciphers .................................................................................... 819standard_conforming_strings ........................................................ 819statement_mem ............................................................................ 820statement_timeout ........................................................................ 820stats_queue_level ......................................................................... 820superuser_reserved_connections .................................................. 820tcp_keepalives_count .................................................................... 820tcp_keepalives_idle ....................................................................... 820tcp_keepalives_interval ................................................................. 820temp_buffers ................................................................................ 820TimeZone ...................................................................................... 821timezone_abbreviations ................................................................ 821track_activities .............................................................................. 821track_counts ................................................................................. 821transaction_isolation ..................................................................... 821transaction_read_only ................................................................... 821transform_null_equals................................................................... 821unix_socket_directory ................................................................... 821unix_socket_group ........................................................................ 821unix_socket_permissions .............................................................. 822update_process_title ..................................................................... 822vacuum_cost_delay ....................................................................... 822vacuum_cost_limit ........................................................................ 822vacuum_cost_page_dirty .............................................................. 822vacuum_cost_page_hit ................................................................. 822vacuum_cost_page_miss .............................................................. 822vacuum_freeze_min_age .............................................................. 823work_mem .................................................................................... 823 Appendix E: Greenplum MapReduce Specification ................... 824Table of Contents xvii 18. Greenplum Database Administrator Guide 4.1 - ContentsGreenplum MapReduce Document Format ..................................... 824Greenplum MapReduce Document Schema ................................... 827Example Greenplum MapReduce Document................................... 838 MapReduce Flow Diagram ........................................................ 845 Appendix F: Greenplum Environment Variables ........................ 846Required Environment Variables.................................................... 846Optional Environment Variables .................................................... 847 Appendix G: Greenplum Database Data Types .......................... 849 Appendix H: System Catalog Reference ...................................... 852gp_configuration_history ............................................................... 855gp_distributed_log ........................................................................ 856gp_distributed_xacts ..................................................................... 857gp_distribution_policy ................................................................... 858gp_fastsequence ........................................................................... 859gp_fault_strategy .......................................................................... 860gp_global_sequence ...................................................................... 861gpexpand.status ........................................................................... 862gpexpand.status_detail ................................................................. 863gp_id ............................................................................................ 865gp_interfaces ................................................................................ 866gp_master_mirroring .................................................................... 867gp_persistent_database_node ....................................................... 868gp_persistent_filespace_node ....................................................... 869gp_persistent_relation_node ......................................................... 870gp_persistent_tablespace_node .................................................... 871gp_relation_node .......................................................................... 872gp_san_configuration .................................................................... 873gp_segment_configuration ............................................................ 875gp_pgdatabase ............................................................................. 876gp_transaction_log........................................................................ 877gp_version_at_initdb..................................................................... 878gpexpand.expansion_progress ...................................................... 879pg_aggregate ................................................................................ 880pg_am .......................................................................................... 881pg_amop ....................................................................................... 883pg_amproc .................................................................................... 884pg_appendonly ............................................................................. 885pg_attrdef ..................................................................................... 887pg_attribute .................................................................................. 888pg_auth_members ........................................................................ 890pg_authid...................................................................................... 891pg_autovacuum ............................................................................ 892pg_cast ......................................................................................... 893pg_class ........................................................................................ 894pg_constraint ................................................................................ 897pg_conversion ............................................................................... 898pg_database ................................................................................. 899Table of Contents xviii 19. Greenplum Database Administrator Guide 4.1 - Contentspg_depend .................................................................................... 901pg_description .............................................................................. 902pg_exttable ................................................................................... 903pg_filespace .................................................................................. 904pg_filespace_entry ........................................................................ 905pg_index ....................................................................................... 906pg_inherits .................................................................................... 908pg_language ................................................................................. 909pg_largeobject .............................................................................. 910pg_listener .................................................................................... 911pg_locks........................................................................................ 912pg_namespace .............................................................................. 914pg_opclass .................................................................................... 915pg_operator .................................................................................. 916pg_partition .................................................................................. 917pg_partition_columns .................................................................... 918pg_partition_rule .......................................................................... 919pg_partition_templates ................................................................. 920pg_partitions ................................................................................. 921pg_pltemplate ............................................................................... 923pg_proc......................................................................................... 924pg_resqueue ................................................................................. 926pg_resourcetype ........................................................................... 927pg_resqueue_attributes ................................................................ 928pg_resqueue_status - Deprecated ................................................. 929pg_resqueuecapability................................................................... 930pg_rewrite .................................................................................... 931pg_roles ........................................................................................ 932pg_shdepend ................................................................................ 933pg_shdescription ........................................................................... 934pg_stat_activity ............................................................................ 935pg_stat_operations ....................................................................... 936pg_stat_partition_operations ........................................................ 937pg_stat_resqueues ........................................................................ 938pg_stat_last_operation ................................................................. 939pg_stat_last_shoperation .............................................................. 940pg_statistic ................................................................................... 941pg_tablespace ............................................................................... 943pg_trigger ..................................................................................... 944pg_type ........................................................................................ 945pg_window .................................................................................... 948 Appendix I: The gp_toolkit Administrative Schema ................... 950Checking for Tables that Need Routine Maintenance ..................... 950 gp_bloat_diag .......................................................................... 951 gp_stats_missing ..................................................................... 951Checking for Locks ........................................................................ 951 gp_locks_on_relation ............................................................... 952 gp_locks_on_resqueue ............................................................ 952Table of Contents xix 20. Greenplum Database Administrator Guide 4.1 - ContentsViewing Greenplum Database Server Log Files .............................. 953 gp_log_command_timings ....................................................... 953 gp_log_database ..................................................................... 954 gp_log_master_concise ........................................................... 955 gp_log_system ........................................................................ 955Checking Server Configuration Files .............................................. 956 gp_param_setting(parameter_name)..................................... 957 gp_param_settings_seg_value_diffs ........................................ 957Checking for Failed Segments ....................................................... 957 gp_pgdatabase_invalid ............................................................ 957Checking Resource Queue Activity and Status ............................... 958 gp_resq_activity ...................................................................... 958 gp_resq_activity_by_queue ..................................................... 959 gp_resq_priority_statement..................................................... 959 gp_resq_role ........................................................................... 959 gp_resqueue_status ................................................................ 960Viewing Users and Groups (Roles)................................................. 960 gp_roles_assigned ................................................................... 961Checking Database Object Sizes and Disk Space ........................... 961 gp_size_of_all_table_indexes .................................................. 962 gp_size_of_database ............................................................... 962 gp_size_of_index ..................................................................... 962 gp_size_of_partition_and_indexes_disk ................................... 963 gp_size_of_schema_disk ......................................................... 963 gp_size_of_table_and_indexes_disk ........................................ 963 gp_size_of_table_and_indexes_licensing ................................. 964 gp_size_of_table_disk ............................................................. 964 gp_size_of_table_uncompressed ............................................. 964 gp_disk_free ............................................................................ 965Checking for Uneven Data Distribution .......................................... 965 gp_skew_coefficients ............................................................... 965 gp_skew_idle_fractions ........................................................... 966 Appendix J: Oracle Compatibility Functions .............................. 967Installing Oracle Compatibility Functions ....................................... 967Oracle and Greenplum Implementation Differences ....................... 967Available Oracle Compatibility Functions ....................................... 968decode .......................................................................................... 969nvl ................................................................................................ 972 Appendix K: Character Set Support ............................................. 973Setting the Character Set .............................................................. 974Character Set Conversion Between Server and Client.................... 975 Appendix L: SQL 2008 Optional Feature Compliance................ 978 Glossary .......................................................................................... 999 Index ............................................................................................. 1008Table of Contents xx