PostgreSQL Data Types for Oracle Developers - OpenSCG · PDF fileIntroduction Welcome to the world of PostgreSQL where data types are more than just a passing fancy. They are a passion

White Paper

PostgreSQL Data Types for Oracle Developers

This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/.

PostgreSQL Data Types for Oracle Developers 2

http://creativecommons.org/licenses/by-nc/4.0/

Table of ContentsIntroduction..............................................................................................4

Character Data Types..................................................................................4

TEXT..................................................................................................5VARCHAR...........................................................................................5CHAR.................................................................................................6

Numeric Data Types....................................................................................6

Integers.............................................................................................6Floating-Points....................................................................................7Numeric.............................................................................................7

Date Data Types.........................................................................................8

Timestamp.........................................................................................8Date................................................................................................10Time................................................................................................10

Large Objects..........................................................................................10

Bytea...............................................................................................11Purpose Built Types...................................................................................11

Boolean............................................................................................11JSON................................................................................................12XML.................................................................................................13UUID................................................................................................14Geometry.........................................................................................14Network Addresses.............................................................................15

Data Type Constructs................................................................................16

Composites.......................................................................................16Domains...........................................................................................17Arrays..............................................................................................17Enums..............................................................................................18Ranges.............................................................................................19

Conclusion...............................................................................................21

About OpenSCG.......................................................................................21


IntroductionWelcome to the world of PostgreSQL where data types are more than just a passing fancy. They are a passion. There are the equivalent main three data types(VARCHAR2, NUMBER and DATE) that Oracle has, but to really leverage the features, functionality and performance of PostgreSQL, there is another level of understanding.

The philosophy of PostgreSQL is a bit different than Oracle. In Oracle, the base data types are fairly basic and the put the responsibility on the developer to handle the logic of the data. PostgreSQL optimizes at the lower level data type forcommon business problems freeing up developer time and effort for more important things.

As more and more people move from Oracle to PostgreSQL, the first thing they discover is that the data types in PostgreSQL while similar to Oracle behave differently. A common first step in exploring PostgreSQL is to create some tables, load some data from Oracle and test out some existing queries. An example of a false assumption almost everyone initially makes is that an Oracle NUMBER type is equivalent to a PostgreSQL NUMERIC type. This is just wrong most of the time. This paper will delve into PostgreSQL’s built-in data types and talk about when and how to use them most effectively to avoid common pitfalls and mistakes Oracle developers often make using PostgreSQL.

Character Data TypesOracle has two main character data types, VARCHAR2 and CHAR with their “N” sister types. There is also the LONG data type for larger strings. PostgreSQL’s equivalent types behave similarly to Oracle, but have some subtle differences.

PostgreSQL does not have the equivalent of “N” sister types for Unicode values. In PostgreSQL, the character set is set at the database level so if you have the need for multi-byte character set data, setting the character set of the database to UTF8 will allow multi-byte data in all character columns. Luckily, the default encoding for PostgreSQL is UTF8 so it is likely that you will just be able to start inserting your data.

Like Oracle, PostgreSQL builds its character data types off of the same internal structures. Each PostgreSQL type has just small variances on that structure. So, choosing the right data type is a matter of understanding those variances and deciding if you need they functionality provided by the type.


TEXT

The TEXT data type in PostgreSQL will yield the best performance compared to the other PostgreSQL character data types. This is from just being a thin layer of the internal structure with no restrictions placed on it. You can store up to 1GB of text in the column without the use of special function calls.

# CREATE TABLE demo (val TEXT);CREATE TABLE

# INSERT INTO demo (val) VALUES ('Hello World');INSERT 0 1

# SELECT val FROM demo; val ------------- Hello World(1 row)

A TEXT column behaves the same way you would use a VARCHAR2 in Oracle, but without the size restrictions of 4000 characters. It also provides the large sizes of a LONG in Oracle, but it is limited to 1GB instead of 2GB.

VARCHAR

The VARCHAR data type extends the base structure with a defined maximum length. If you do not give it a maximum length, a VARCHAR is essentially a synonym for TEXT.

# CREATE TABLE demo (val VARCHAR(5));CREATE TABLE

# INSERT INTO demo (val) VALUES ('Hello World');ERROR: value too long for type character varying(5)STATEMENT: INSERT INTO demo (val) VALUES ('Hello World');ERROR: value too long for type character varying(5)

# INSERT INTO demo (val) VALUES ('Hello');INSERT 0 1

# SELECT val FROM demo; val ------- Hello(1 row)


Use a VARCHAR if you put a constraint on the length of the stored value. That length can be up to 1GB in size just like the TEXT data type so you can create a VARCHAR(1000000) if you really needed.

CHAR

The CHAR data type takes this a bit further and pads the value to make what is stored a fixed size. Like Oracle, there is no performance gain to this. The only reason to use CHAR over VARCHAR or TEXT would be if you need a CHAR(1) and want to save a little typing.

Numeric Data TypesThe character data types in PostgreSQL are very similar to Oracle, but the numeric data types are where you start seeing some differences in PostgreSQL. In Oracle, everything is of type NUMBER and you use the scale and precision values to control the value you can store. In PostgreSQL, you can control the values with scale and precision with some data types, but others are optimized and controlled just by their type itself.

Integers

When you need an integer, use one of the three types of integers provided by PostgreSQL based on the size you need. The are all fixed sizes and extremely efficient. They should be used for all primary keys and foreign key relationships.

Name Size Range

smallint 2 bytes -32768 to +32767

integer 4 bytes -2147483648 to +2147483647

bigint 8 bytes -9223372036854775808 to +9223372036854775807

When trying to decide between which size integer to choose, if you are not sure, always choose the larger size. In reality, very rarely will the space saving be meaningful and when it is, you usually need a larger integer value.


Floating-Points

In PostgreSQL, there are two floating point data types that follow the implementations of IEEE Standard 754. This allows you to store numbers with decimals, but it uses inexact means to store and act on those numbers. Managingthis inexactness is beyond the scope of this paper, but if you need exact numerical values for things like money, do not use floating-points.

As an example, rounding the numbers 2.5 and 3.5 result in different behaviors.

# CREATE TABLE demo (val real);CREATE TABLE

# INSERT INTO demo VALUES (3.5), (2.5);INSERT 0 2

# SELECT val, round(val) FROM demo; val | round -----+------- 3.5 | 4 2.5 | 2(2 rows)

Use floating-points if you need a simple data type to store numerical data, but themajority of the time, they should be avoided.

Name Size Range

real 4 bytes 6 decimal digits precision

double precision 8 bytes 15 decimal digits precision

Numeric

For the majority of your numerical data that is not integers, you should be using the data type, numeric. A numeric type is very precise and is the ideal data type for storing things like financial data. A numeric can hold 131072 digits before the decimal point and 16383 digits after, but this ability comes at a price. It is a very slow data type compared to integers.


# CREATE TABLE demo (val numeric);CREATE TABLE

# INSERT INTO demo VALUES (3.5), (2.5);INSERT 0 2

# SELECT val, round(val) FROM demo; val | round -----+------- 3.5 | 4 2.5 | 3(2 rows)

When coming from Oracle, many people use numeric in PostgreSQL as an alias for NUMBER in Oracle. A numeric can hold any value that was a NUMBER in Oracle, but the performance penalty shows itself when they are used as primary and foreign keys. If something is an integer, use one of the integer fields and onlyuse a numeric for data that really needs the precision.

Date Data TypesThe date data types in PostgreSQL essentially behave the same as in Oracle, but the naming convention adds some confusion. In Oracle the DATE data type actually has time information in it as well, so it really is a timestamp with 1 second precision.

Timestamp

A PostgreSQL timestamp is largely the same as an Oracle DATE. The only real difference is that a PostgreSQL timestamp will have microsecond precision insteadof second precision. To get the equivalent in PostgreSQL, setting the precision of the timestamp to 0 will do the trick. In addition to precision, there is the choice ofstoring with timezone or without. PostgreSQL will convert timestamp with timezone AKA timestamptz to UTC on input and store it. There is no storage overhead for storing timezones.


Name Size Low High Resolution

timestamp [ (p) ] 8 bytes 4713 BC 294276 AD 1 microsecond / 14 digits

timestamptz [ (p) ] 8 bytes 4713 BC 294276 AD 1 microsecond / 14 digits

The timezone of the client will be applied on retrieval so all of the timestamps arein an equivalent timezone regardless of how they are entered. This makes for better applications so please always use timestamptz.

# CREATE TABLE demo (val timestamptz(0));CREATE TABLE

# show timezone; TimeZone ------------ US/Eastern(1 row)

# INSERT INTO demo VALUES ('01-01-2010 00:00:00.000');INSERT 0 1

# SELECT * FROM demo; val ------------------------ 2010-01-01 00:00:00-05(1 row)

# SET timezone = 'US/Pacific';SET

# SELECT * FROM demo; val ------------------------ 2009-12-31 21:00:00-08(1 row)

As the client changes the timezone, the data is just converted on retrieval to the appropriate value for the given timezone.


Date

Unlike Oracle in PostgreSQL, a date is just a date. Many times the time information is not needed for an application so using a date data type will save space. In PostgreSQL, a date is just 4 bytes.

# CREATE TABLE demo (val date);CREATE TABLE


# SELECT * FROM demo; val ------------ 2010-01-01(1 row)

Time

Similar to the data data type, the time data type is just a time of day value. There is no space saving over a timestamp, they are both 8 bytes, but a time data type gives the flexibility of not needing a date associated with it.

# CREATE TABLE demo (val time);CREATE TABLE


postgres=# SELECT * FROM demo; val ---------- 09:00:00(1 row)

Large ObjectsThe PostgreSQL data type, bytea, differs is how Oracle BLOBs handle binary data,but both data types allow for the storage of objects like images inside of databasetables. The merits of storing that type of data inside of a relation database is


beyond the scope of this paper, but it is possible. For CLOB fields, the PostgreSQL text data type will handle most cases. The Oracle CLOB data type holds up to 4GBof text while the PostgreSQL text data type is limited to 1GB.

Bytea

A bytea is a variable length binary string stored which is accepted in a hex format. It uses two hexadecimal digits per byte to store the data so values can besent to PostgreSQL through standard SQL statements.

# CREATE TABLE demo (val bytea);CREATE TABLE

# INSERT INTO demo VALUES (pg_read_binary_file('pg_logo.png'));INSERT 0 1

# SELECT octet_length(val), substring(val, 0, 5) FROM demo; octet_length | substring --------------+------------ 782449 | \x89504e47(1 row)

Purpose Built TypesIn addition to the core base data types, PostgreSQL has a number of additional data types that are purpose built for a specific type of data. Using these types canmake an application faster as well as easier to manage. Some of them also enable some advanced features that allow for significantly more functionality.

Boolean

The most commonly used of the purpose built type is a boolean. The majority of the data models have some sort of flag to indicate a boolean value. In Oracle, thisis commonly represented as a CHAR(1). A PostgreSQL boolean data type uses thesame one byte as a CHAR(1), but it is significantly easier to work with than Oracle’s. With a CHAR(1), constraints really need to be added to make sure only valid T of F values are entered. Then when looking up values, the case of the value matters. By putting boolean data inside of a generic CHAR data type, a lot of care is required to keep the data valid. The PostgreSQL boolean type eliminatesthe need to worry about the constraints since it is all built in. It even handles casting of standard representations of boolean values.


# CREATE TABLE demo (id int, is_active boolean);CREATE TABLE

# INSERT INTO demo VALUES (1, TRUE), (2, 'on'), (3, 'false'), (4, 'no');INSERT 0 4

# SELECT * FROM demo; id | is_active ----+----------- 1 | t 2 | t 3 | f 4 | f(4 rows)

# SELECT * FROM demo WHERE is_active; id | is_active ----+----------- 1 | t 2 | t(2 rows)

JSON

In the world of today, people want to store their data in JSON. Many web developers feel that its flexible format increases productivity. PostgreSQL has two native JSON data types to store data which adds additional functionality over storing the data in a text or varchar field. Choosing the right JSON data type in PostgreSQL depends on how the data will be used.

Name Description

json A simple extension of a text data type with JSON validation

jsonb A binary representation of the JSON data

When deciding between the two types, json will return the data exactly as it is inputted including white-space while jsonb will optimize the data for querying andthe white-spacing may change a little.


# CREATE TABLE demo (val jsonb);CREATE TABLE

# INSERT INTO demo VALUES ('{ "name":"PostgreSQL", "opensource":true }'), ('{ "name":"Oracle", "opensource":false }');INSERT 0 2

# SELECT * FROM demo; val -------------------------------------------- {"name": "PostgreSQL", "opensource": true} {"name": "Oracle", "opensource": false}(2 rows)

# SELECT * FROM demo WHERE val->>'opensource' = 'true'; val -------------------------------------------- {"name": "PostgreSQL", "opensource": true}(1 row)

The power of the JSON data types are shown when querying the data. There are a number of operators to act on the JSON data as well as the ability to index the full JSON document.

XML

Like Oracle, PostgreSQL has the ability to store XML data in a data type, but it is nothing more than an extension to a text data type. It will validate that the XML stored is well formed, but nothing more. There are a number of functions such as xpath to act on the XML data.

# CREATE TABLE demo (val xml);CREATE TABLE

# INSERT INTO demo VALUES ('<database><name>PostgreSQL</name><opensource>true</opensource></database>');INSERT 0 1

# SELECT * FROM demo; val --------------------------------------------------------------------------- <database><name>PostgreSQL</name><opensource>true</opensource></database>(1 row)


UUID

In Oracle, to handle a UUID, there is a choice between storing the data inefficiently as a VARCHAR2(32) or in the more difficult to handle RAW(16). PostgreSQL has a native UUID data type that stores the data efficiently in 16 bytes while still working with it via SQL the textual representation. This makes the uuid type a good choice for primary keys when they are needed over standardintegers.

# CREATE TABLE demo (val uuid);CREATE TABLE

# INSERT INTO demo VALUES ('6e035bdc-f5e6-11e6-98c8-335d31f42bfd'), ('6ecabdb2-f5e6-11e6-98c9-23ac0ea00caf');INSERT 0 2

# SELECT * FROM demo; val -------------------------------------- 6e035bdc-f5e6-11e6-98c8-335d31f42bfd 6ecabdb2-f5e6-11e6-98c9-23ac0ea00caf(2 rows)

Geometry

Oracle has Oracle Spatial which is a powerful extension for complex geospatial scenarios. There is an PostgreSQL extension, PostGIS, that provides similar if not superior functionality, but PostGIS is beyond the scope of this paper. For simple geometries with only two dimensions where PostGIS is overkill, PostgreSQL has several native geometry types.

Name Size Description

point 16 bytes Point on a plane

line 32 bytes Infinite line

lseg 32 bytes Finite line segment

box 32 bytes Rectangular box

path 16 + 16n bytes Path


polygon 40 + 16n bytes Polygon

circle 24 bytes Circle

There a many functions and operators that can act on the various geometry types.

# CREATE TABLE demo (val circle);CREATE TABLE

# INSERT INTO demo VALUES ('<(2,2),1>'), ('<(10,10),5>');INSERT 0 2

# SELECT area(val) FROM demo; area ------------------ 3.14159265358979 78.5398163397448(2 rows)

Network Addresses

Many applications store network information such as the IP address of users or sensors. In Oracle, this data is typically stored are a VARCHAR2 or a custom Object Type. PostgreSQL has three native types that optimize the network data.

Name Size Description

cidr 7 or 19 bytes IPv4 and IPv6 networks

inet 7 or 19 bytes IPv4 and IPv6 hosts and networks

macaddr 6 bytes MAC addresses

In addition to the space saving on storing the IP address data, the operators on the network address type provide powerful functionality like searching data by subnet.


# CREATE TABLE demo (val inet);CREATE TABLE

# INSERT INTO demo VALUES ('192.168.1.10'), ('192.168.1.12'), ('172.16.1.10');INSERT 0 3

# SELECT * FROM demo; val -------------- 192.168.1.10 192.168.1.12 172.16.1.10(3 rows)

# SELECT * FROM demo WHERE inet '192.168.1.0/24' >> val; val -------------- 192.168.1.10 192.168.1.12(2 rows)

Data Type ConstructsPostgreSQL’s vast number of data types can be used to construct more complex structures depending on needs. The base data types can be combined in a number of ways to more efficiently store data and add functionality that makes accessing the data easier. Some of these constructs are common across other databases like Oracle while some are unique to PostgreSQL.

Composites

A PostgreSQL composite type is a structure similar to an Oracle Object Type, but without the methods associated with the type. A composite allows for related dataelements to be bound together in a common structure which is reusable across a data model.

# CREATE TYPE fullname AS (first varchar, middle varchar, last varchar);CREATE TYPE

# CREATE TABLE demo (val fullname);CREATE TABLE


# INSERT INTO demo VALUES ('(Thomas,Alva,Edison)'), ('(Nikola,,Tesla)');INSERT 0 2

# SELECT (val).first, (val).last FROM demo; first | last --------+-------- Thomas | Edison Nikola | Tesla(2 rows)

Domains

A domain is essentially a data type with an associated constraint. It has no direct equivalent in Oracle, but similar functionality can be achieved with Object Types and constructors. They are extremely useful in defining common business rules that need to be used consistently throughout a data model.

# CREATE DOMAIN customer_number AS varchar(10) CHECK (length(VALUE)=10 AND left(VALUE, 1)='A');CREATE DOMAIN

# CREATE TABLE demo (val customer_number);CREATE TABLE

# INSERT INTO demo VALUES ('A123456789');INSERT 0 1

# INSERT INTO demo VALUES ('B123456789');ERROR: value for domain customer_number violates check constraint "customer_number_check"

Arrays

Oracle has a VARRAY to store a fixed size single dimension collection of data elements. PostgreSQL’s array types expand base data types so multiple dimensions can be used and the array is not fixed length. Using arrays in a relational data model may seem odd, but it is very useful in holding simple data that has a may-to-one relationship with its parent.

# CREATE TABLE demo (val numeric[]);CREATE TABLE


# INSERT INTO demo VALUES ('{2.1,9.8,7.4}');INSERT 0 1

# SELECT * FROM demo; val --------------- {2.1,9.8,7.4}(1 row)

# SELECT unnest(val) FROM demo; unnest -------- 2.1 9.8 7.4(3 rows)

Since arrays are a first class collection in PostgreSQL, there are a number of functions and operators to make access to the data stored in the array easy.

Enums

Oracle does not have an equivalent enumerated (enum) type that is found in PostgreSQL. An enum is extremely useful for representing rarely changing information such as business priority or country code, but stored in the table as an efficient integer. Normally, this type of data is represented in a table with the appropriate foreign keys to ensure data integrity. An enum will guarantee that integrity without the foreign key check.

=# CREATE TYPE priority AS ENUM ('Critical', 'High', 'Medium', 'Low');CREATE TYPE

# CREATE TABLE demo (id int, val priority);CREATE TABLE

# INSERT INTO demo VALUES (1, 'Low'), (2, 'High'), (3, 'Critical'), (4, 'Medium');INSERT 0 4

# SELECT * FROM demo ORDER BY val; id | val ----+---------- 3 | Critical 2 | High 4 | Medium 1 | Low(4 rows)


The other benefit of enum is getting the data as a human readable string without adding another join to the query. The becomes very important in more complex queries where every join compounds the number of potential number of query plans the PostgreSQL optimizer needs to interrogate. Reducing the number of joins in a query will result in faster response times.

Ranges

Most business applications need the ability to look at data in ranges. It is usually around dates which is why most data models have columns holding a start date and an end date. In Oracle, the only option is to have two distinct columns and then add complex logic in a query or stored procedure to correctly find rows that overlap with those ranges.

To return rows in a range using a start date and end date appear to be fairly simple. The logic to get to find the rows that overlap a date range uses a simple WHERE clause with some BETWEEN statements to return the matching rows.

# CREATE TABLE demo (id int, start_dt date, end_dt date);CREATE TABLE

# INSERT INTO demo VALUES (1, '01-01-2010', '01-31-2010');INSERT 0 1# INSERT INTO demo VALUES (2, '02-01-2010', '02-14-2010');INSERT 0 1# INSERT INTO demo VALUES (3, '02-15-2010', '02-28-2010');INSERT 0 1

# SELECT * FROM demo; id | start_dt | end_dt ----+------------+------------ 1 | 2010-01-01 | 2010-01-31 2 | 2010-02-01 | 2010-02-14 3 | 2010-02-15 | 2010-02-28(3 rows)

# SELECT * # FROM demo# WHERE '01-30-2010' BETWEEN start_dt AND end_dt# OR '02-05-2010' BETWEEN start_dt AND end_dt; id | start_dt | end_dt ----+------------+------------ 1 | 2010-01-01 | 2010-01-31 2 | 2010-02-01 | 2010-02-14(2 rows)


This technique falls down when date range is wider than the data stored on the rows. All rows match the date range, but the WHERE clause returns no rows.

# SELECT * # FROM demo# WHERE '12-31-2009' BETWEEN start_dt AND end_dt# OR '01-01-2011' BETWEEN start_dt AND end_dt; id | start_dt | end_dt ----+----------+--------(0 rows)

Getting the logic correct for finding overlapping date ranges can be extremely complex. PostgreSQL range types eliminates this complexity and makes queries simple. There are a number of built-in range types and more can be defined easily.

Name Description

int4range Range of integers

int8range Range of bigints

numrange Range of numerics

tsrange Range of timestamps without time zone

tstzrange Range of timestamps with time zone

daterange Range of date

Using a data range column instead of two distinct columns makes the query to find overlapping rows extremely simple.

# CREATE TABLE demo (id int, range daterange);CREATE TABLE

# INSERT INTO demo VALUES (1, '[01-01-2010, 01-31-2010)');INSERT 0 1# INSERT INTO demo VALUES (2, '[02-01-2010, 02-14-2010)');INSERT 0 1# INSERT INTO demo VALUES (3, '[02-15-2010, 02-28-2010)');INSERT 0 1


# SELECT * FROM demo; id | range ----+------------------------- 1 | [2010-01-01,2010-01-31) 2 | [2010-02-01,2010-02-14) 3 | [2010-02-15,2010-02-28)(3 rows)

# SELECT * FROM demo WHERE daterange('01-30-2010', '02-05-2010') && range; id | range ----+------------------------- 1 | [2010-01-01,2010-01-31) 2 | [2010-02-01,2010-02-14)(2 rows)

# SELECT * FROM demo WHERE daterange('12-31-2009', '01-01-2011') && range; id | range ----+------------------------- 1 | [2010-01-01,2010-01-31) 2 | [2010-02-01,2010-02-14) 3 | [2010-02-15,2010-02-28)(3 rows)

ConclusionThis is just a short introduction to PostgreSQL’s data types and only includes the built-in types. PostgreSQL has many external extensions to add even more types that add unique capabilities like a DNA data type for Bioinformatics. The PostgreSQL documentation is a great resource for a deeper understanding. There are a number of the internal operators and functions that act on the PostgreSQL data types that were just touched on in this paper. Happy data modeling.

About OpenSCGOpenSCG is the leading provider of Oracle to PostgreSQL migration services. We have been focused on heterogeneous database environments (PostgreSQL, SQL Server, Oracle, Sybase, Cassandra) since starting in 2010. Today, with over 120 employees around the world, we help our customers get the most out of the OpenSource database.

We provide services covering all major steps of a typical migration project: assessment, schema conversion, data migration, application conversion, testing, integration, deployment, performance tuning, training, remote DBA, and support.

For more details, visit us at http://www.openscg.com, e-mail us at [email protected], or call +1-732-339-3419.


mailto:[email protected]

http://www.openscg.com/

Documents

PostgreSQL Data Types for Oracle Developers - OpenSCG · PDF fileIntroduction Welcome to the world of PostgreSQL where data types are more than just a passing fancy. They are a passion