Writing Basic Postgres Functions

By Lloyd Albin12/3/2013

Functions

The Basics - Languages

Pre-Installed Languages:

SQL C internal

Installable Languages that come with Postgres: plpgsql plperl plperlu pltcl plpython

Other Downloadable Languages: pljava plphp plpy plr plruby plscheme plsh

Languages

SELECT name, comment

FROM pg_available_extensions WHERE comment LIKE '%language%';

Finding Languages that can be Installed

Name Comment

plperl PL/Perl procedural language

plperlu PL/PerlU untrusted procedural language

plpgsql PL/pgSQL procedural language

http://

www.postgresql.org/docs/9.3/static/external-pl.html http://

www.postgresql.org/docs/9.2/static/external-pl.html http://

www.postgresql.org/docs/9.1/static/external-pl.html http://

www.postgresql.org/docs/9.0/static/external-pl.html http://

www.postgresql.org/docs/8.4/static/external-pl.html

Finding Downloadable Languages

Name Language

Website (Postgres 9.2 & 9.3)

PL/Java Java http://pljava.projects.postgresql.org/

PL/PHP PHP http://www.commandprompt.com/community/plphp/

PL/Py Python http://python.projects.postgresql.org/backend/

PL/R R http://www.joeconway.com/plr/

PL/Ruby Ruby http://raa.ruby-lang.org/project/pl-ruby/

PL/Scheme Scheme http://plscheme.projects.postgresql.org/

PL/sh Unix shell http://plsh.projects.postgresql.org/

CREATE EXTENSION plperl; CREATE EXTENSION plperlu; CREATE EXTENSION plpgsql;

How to Install a Language

DROP EXTENSION plperl; DROP EXTENSION plperlu; DROP EXTENSION plpgsql;

Uninstalling a Language

Functions

The Basics – Function Behavior

IMMUTABLE indicates that the function cannot

modify the database and always returns the same result when given the same argument values; that is, it does not do database lookups or otherwise use information not directly present in its argument list.

If this option is given, any call of the function with all-constant arguments can be immediately replaced with the function value.

What this means is these types of functions can be used as a type converter or indexer.

IMMUTABLE

STABLE indicates that the function cannot modify the

database, and that within a single table scan it will consistently return the same result for the same argument values, but that its result could change across SQL statements.

This is the appropriate selection for functions whose results depend on database lookups, parameter variables (such as the current time zone), etc.

It is inappropriate for AFTER triggers that wish to query rows modified by the current command.

Also note that the current_timestamp family of functions qualify as stable, since their values do not change within a transaction.

STABLE

VOLATILE indicates that the function value can

change even within a single table scan, so no optimizations can be made.

Relatively few database functions are volatile in this sense; some examples are random(), currval(), timeofday().

But note that any function that has side-effects must be classified volatile, even if its result is quite predictable, to prevent calls from being optimized away; some example are random(), currval(), timeofday()

VOLATILE

Functions

The Basics – NULL INPUT

CALLED ON NULL INPUT – This is the default if

you don’t say anything. You will need to handle the NULL’s within your code.

RETURNS NULL ON NULL INPUT or STRICT – This will cause the function to return NULL when any of the input values are NULL. The body of the function is never executed.

What to do with NULL?

Functions

The Basics - SECURITY

SECURITY INVOKER – This is the default. The

function is to be executed with the privileges of the user that calls it.

SECURITY DEFINER – This specifies that the function is to be executed with the privileges of the user that created it. This could be used to have a function update a table that the calling user does not have permissions to access, etc.

Functions

The Basics - Syntax

CREATE OR REPLACE FUNCTION (

argname text)RETURNS numeric AS$body$….$body$LANGUAGE 'sql'IMMUTABLE | STABLE | VOLATILE RETURNS NULL ON NULL INPUTSECURITY DEFINER;

CREATE FUNCTION

void – This is for when the trigger should not

return anything. It is just doing some backgroud process for you.

trigger – This must be set for all trigger functions. boolean, text, etc – This is for a single values

being passed back. SET OF schema.table – This is for returning

multiple rows of data. This can either point to an existing table or a composite type to get the table/field layout.

Additional Return Types

Simple Functions

Basic SQL functions – Converting a field type

CREATE TABLE tools.lloyd_test ( mynumber VARCHAR);

ALTER TABLE tools.lloyd_test ALTER COLUMN mynumber TYPE INTEGER COLLATE pg_catalog."default";

ERROR: collations are not supported by type integer

Converting a tables field

ALTER TABLE tools.lloyd_test ALTER COLUMN mynumber TYPE INTEGER USING tools.chartoint(mynumber);

Using a function to do the conversion

CREATE OR REPLACE FUNCTION tools.chartoint ( chartoconvert varchar)RETURNS integer AS$body$ SELECT CASE WHEN trim(chartoconvert) SIMILAR TO '[0-9,]+' THEN CAST(trim(REPLACE(chartoconvert,',','')) AS integer) ELSE NULL END;$body$LANGUAGE 'sql'IMMUTABLERETURNS NULL ON NULL INPUTSECURITY INVOKER;

Convert varchar to int

CREATE OR REPLACE FUNCTION tools.chartonumeric ( chartoconvert varchar)RETURNS numeric AS$body$ SELECT CASE WHEN trim(chartoconvert) SIMILAR TO '[0-9,.-]+' THEN CAST(trim(REPLACE(chartoconvert,',','')) AS integer) ELSE NULL END;$body$LANGUAGE 'sql'IMMUTABLERETURNS NULL ON NULL INPUTSECURITY INVOKER;

Convert varchar to int

Simple Functions

Basic PL/pgSQL functions – Using as an index

We receive faxes that are multi-page TIFF’s.

The TIFF file name are called the raster id. We have data where we have the full path of the file name, the raster id and the raster id with page number.

Examples: 0000/000000 0000/0000001111 /studydata/studyname/0000/000000

The Problem

The first thing to do, is to be able to find the Raster ID, no matter which format is supplied.

CREATE FUNCTION find_raster (raster varchar)RETURNS VARCHAR(11) AS$$BEGIN CASE length(raster) WHEN 11 THEN -- Format: 1234/567890 -- Returns: 1234/567890 RETURN raster; WHEN 15 THEN -- Format: 1234/5678901234 -- Returns: 1234/567890 RETURN substr(raster, 1, 11); ELSE -- Format: /study_data/study_name/1234/567890 -- Returns: 1234/567890 RETURN substr(raster, length(raster)-10, 11); END CASE;END;$$LANGUAGE plpgsql IMMUTABLE RETURNS NULL ON NULL INPUT;

Finding the Raster ID

-- Test returning of Raster ID when Submitting Raster IDSELECT find_raster('1234/567890');-- Returns: 1234/567890

-- Test returning of Raster ID when Submitting Raster ID with 4 Digit Page NumberSELECT find_raster('1234/5678901234');-- Returns: 1234/567890

-- Test returning of Raster ID when Submitting Filename that includes Raster IDSELECT find_raster('/study_data/study_name/1234/567890');-- Returns: 1234/567890

Examples of the find_raster Function

CREATE INDEX [ name ] ON table ( expression )

expression

An expression based on one or more columns of the table. The expression usually must be written with surrounding parentheses, as shown in the syntax. However, the parentheses can be omitted if the expression has the form of a function call.

CREATE INDEX raster_raster_idx ON raster find_raster(raster); CREATE INDEX raster_file_raster_idx ON raster_file find_raster(raster); CREATE INDEX raster_page_raster_idx ON raster_page find_raster(raster);

Adding the Index

The benefits of the Index

Without IndexSELECT raster.*, raster_page.* FROM (SELECT * FROM raster OFFSET 50000 LIMIT 100) raster LEFT JOIN raster_page ON raster.raster = substr(raster_page.raster, 1, 11);

Total runtime: 4982.527 ms

Total runtime: 4.982527 s

With IndexSELECT raster.*, raster_page.* FROM (SELECT * FROM raster OFFSET 50000 LIMIT 100) raster LEFT JOIN raster_page ON raster.raster = find_raster(raster_page.raster);

Total runtime: 141.809 ms

Total runtime: 0.141809 s

Simple Functions

Basic PL/pgSQL functions – Triggers

Sometimes we want to have a copy of a table

and know when everything happened to the original table, insert, update, delete, and truncate. This is possible to have happen automatically with a trigger function.

Shadow Tables

Table 1CREATE TABLE public.table1 ( key SERIAL, value INTEGER, value_type VARCHAR, PRIMARY KEY(key)) ;

Table 2CREATE TABLE public.table2 ( key INTEGER, value INTEGER, value_type VARCHAR, user_name NAME, action VARCHAR, action_time TIMESTAMP) ;

Creating the base tables

CREATE FUNCTION public.shadow_table1 ()RETURNS trigger AS$body$BEGIN

IF TG_OP = 'INSERT' THENINSERT INTO public.table2 VALUES(NEW.key, NEW.value, NEW.value_type, current_user, TG_OP, now());RETURN NEW;

END IF;IF TG_OP = 'UPDATE' THEN

INSERT INTO public.table2 VALUES(NEW.key, NEW.value, NEW.value_type, current_user, TG_OP, now());RETURN NEW;

END IF;IF TG_OP = 'DELETE' THEN

INSERT INTO public.table2 VALUES(OLD.key, OLD.value, OLD.value_type, current_user, TG_OP, now());RETURN OLD;

END IF;END;$body$LANGUAGE 'plpgsql'VOLATILECALLED ON NULL INPUTSECURITY DEFINER;

The Shadow Function

CREATE TRIGGER table1_tr BEFORE INSERT OR UPDATE OR DELETE ON public.table1 FOR EACH ROW EXECUTE PROCEDURE public.shadow_table1();

Adding the trigger to the table

INSERT INTO public.table1 (value, value_type) VALUES ('30', 'meters');INSERT INTO public.table1 (value, value_type) VALUES ('10', 'inches');UPDATE public.table1 SET value = '20' WHERE value_type = 'inches';DELETE FROM public.table1 WHERE value_type = 'inches';INSERT INTO public.table1 (value, value_type) VALUES ('50', 'inches');

Working with Table1

What they look like

Table1key valu

evalue_type

1 30 meters

3 50 inches

Table2key

value

value_type

user_name

action action_time

1 30 meters postgres INSERT 12/3/2013 4:58:04 PM

2 10 inches postgres INSERT 12/3/2013 4:58:04 PM

2 20 inches postgres UPDATE 12/3/2013 4:58:04 PM

2 20 inches postgres DELETE 12/3/2013 4:58:04 PM

3 50 inches postgres INSERT 12/3/2013 4:58:04 PM

Simple Functions

Basic PL/pgSQL functions – Write to a hidden table

Sometimes you may want a normal user to be

able to write a table, but that user also not be able to view/select any contents from the table aka INSERT only.

This does not play with some front end applications.

Some people will write functions where you pass in the variables, but that is not always possible depending on the front end that is being written.

Hidden Tables

Table 1SET ROLE a;CREATE TABLE public.table1 ( key SERIAL, value INTEGER, value_type VARCHAR, PRIMARY KEY(key)) ;

Table 2SET ROLE b;CREATE TABLE public.table2 ( key INTEGER, value INTEGER, value_type VARCHAR, PRIMARY KEY(key)) ;

Creating the base tables

SET ROLE b;CREATE FUNCTION public.hidden_table1 ()RETURNS trigger AS$body$BEGIN

INSERT INTO public.table2 VALUES(NEW.key, NEW.value, NEW.value_type);

RETURN NULL;END;$body$LANGUAGE 'plpgsql'VOLATILECALLED ON NULL INPUTSECURITY DEFINER;

The Hidden Function

CREATE TRIGGER table1_tr BEFORE INSERT ON public.table1 FOR EACH ROW EXECUTE PROCEDURE public.hidden_table1();

Adding the trigger to the table

INSERT INTO public.table1 (value, value_type) VALUES ('30', 'meters');INSERT INTO public.table1 (value, value_type) VALUES ('10', 'inches');INSERT INTO public.table1 (value, value_type) VALUES ('50', 'inches');

Working with Table1

What they look like

Table1key valu

evalue_type

Table2key

value

value_type

1 30 meters

2 10 inches

3 50 inches

Simple Functions

Basic PL/pgSQL functions – Returning a Table

CREATE OR REPLACE FUNCTION tools.count_schema_rows (search_schema_name name)RETURNS SETOF tools.schema_row_count AS$body$DECLAREschema_results RECORD; table_results RECORD;BEGIN FOR schema_results IN SELECT table_schema, table_name FROM information_schema.tables WHERE table_schema = search_schema_name AND table_type = 'BASE TABLE' ORDER BY table_name LOOP -- looping through tables in schema here EXECUTE 'SELECT ' || quote_literal(schema_results.table_schema) || '::NAME AS schema_name, ' || quote_literal(schema_results.table_name) || '::NAME AS table_name, count(*), pg_total_relation_size(''' || schema_results.table_schema || '.' || schema_results.table_name || ''') AS total_size, pg_size_pretty(pg_total_relation_size(''' || schema_results.table_schema || '.' || schema_results.table_name || ''')) AS total_size_pretty FROM ' || quote_ident(schema_results.table_schema) || '.' || quote_ident(schema_results.table_name) INTO table_results; RETURN NEXT table_results; END LOOP;END;$body$LANGUAGE 'plpgsql'VOLATILECALLED ON NULL INPUTSECURITY DEFINER;

Counting the tables

SELECT * FROM tools.count_schema_rows

('dffax');

Counting the tables

schema_name

table_name row_count total_size total_size_pretty

dffax dfx_time1_p052

925323 566116352 540 MB

dffax dfx_time1_h021

1007840 622043136 593 MB

dffax dfx_time1_m003

1241989 767385600 732 MB

Writing Advanced Postgres Functions

By Lloyd Albin1/7/2014

Materialized Views Single Table Shadow functions. Lookup User Dependencies Update sequences for an entire schema

Sample of Functions