© 2011 IBM Corporation September 9, 2010 Storage Provisioning Scott Pickett – WW Informix Technical Sales For questions about this presentation contact:

© 2011 IBM CorporationSeptember 9, 2010

Storage Provisioning

Scott Pickett – WW Informix Technical SalesFor questions about this presentation contact: [email protected]

© 2010 IBM Corporation2

Storage Provisioning

Storage provisioning solves a long time problem for most IDS customers; namely, how do we dynamically and efficiently add additional disk space for IDS storage spaces, not knowing in advance when or which one of the spaces will either run out of space or cross a threshold of a space allocation requirement?

Create a pool of storage, unallocated, within IDS.

Configure a little, in advance.

IDS does the rest. Automatically, if configured to do so.


What is Storage Provisioning (SP)?

Automatic expansion of dbspaces, temporary dbspaces, sbspaces, temporary sbspaces, and blobspaces.

"Out-of-space" errors are significantly reduced.

Chunks are extended and/or created as needed.

IDS can also expand spaces before they are full.

Feature is highly configurable.

Feature will be fully incorporated into OAT.


Setting up Storage Provisioning

Configure various sizes and thresholds.

Make some existing chunks extendable.

Define your storage pool.


Storage Provisioning: The Power of 2

Two available modes:

– Manual

– Automatic

Two available space expansion methods:

– Chunk extension

– Chunk creation

Two available interfaces:

– sysadmin task() / admin() functions (SQL interface)

– OAT (Graphical interface)


What is the Storage Pool?

A list of raw devices, cooked files, and/or directories from which IDS can allocate space as needed.

For example:Path: /dev/rawdevice1Offset: 500000Total Size: 2000000

Path: /ifmx/STORAGE_DIROffset: 0Total Size: 0

storagepool table in the sysadmin database (sysadmin:storagepool)


Storage Pool Facts

There is one storage pool per IDS instance.

A file, device, or directory in the pool is called an entry.

You can add, modify, and delete storage pool entries.

Entries that are used up or found to be faulty are left in the pool, but disabled.

IDS automatically detects and prevents potential chunk overlaps.

Extending a chunk does not use the pool.


Extending a chunk

Chunks are not extendable by default. To make a chunk extendable:

EXECUTE FUNCTION task("modify chunk extendable","<chunk_num>");

Now, you can extend it manually at any time:

EXECUTE FUNCTION task("modify chunk extend","<chunk_num>","<min_size>");

You can also wait for the server to extend it automatically.

Restriction: Only unmirrored dbspace and temporary dbspace chunks can be extended.


New SQL Administration API admin() & task() Commands

storagepool add

storagepool modify

storagepool delete

storagepool purge

modify space sp_sizes

modify space expand

modify chunk extendable [off]

modify chunk extend


New SQL Admin API admin() & task() Commands (cont.)

create <object> from storagepool

drop <object> to storagepool

where <object> is either a space:– dbspace– tempdbspace– blobspace– sbspace– tempsbspace

or a chunk.


Adding a Storage Pool Entry

EXECUTE FUNCTION task("storagepool add", "<path>","<offset>","<total_size>","<chunk_size>","<priority>");

The offset and sizes are in kilobytes by default, though other units can be specified.

The allowed priority values are: – 1 (High), – 2 (Medium),– 3 (Low).

Note: When adding a directory to the storage pool, <offset> and <total_size> must be specified as 0.


Manually Expanding a Space

EXECUTE FUNCTION task("modify space expand", "<space_name>","<min_size>");

The space_name can be any type:– dbspace, – temporary dbspace – sbspace – temporary sbspace– blobspace

IDS might round the min_size up, depending on the space's page size and any chosen pool entry's minimum chunk size.

IDS will expand the space by either extending a chunk or adding a chunk using the storage pool.


Creating a Space from the Storage Pool

EXECUTE FUNCTION task("create <space_type> from storagepool", "<space_name>","<min_size>",optional arguments...);

Valid space_type values are: – dbspace– tempdbspace– sbspace– tempsbspace– blobspace

Optional arguments depend on the space type. See information in the IDS 11.70 Administrator’s Reference for details.


Returning Space to Pool When Dropping Objects

EXECUTE FUNCTION task("drop <space_type> to storagepool", "<space_name>");

Valid space_type values are: – dbspace– tempdbspace– sbspace– tempsbspace – blobspace

Chunk level operations:

EXECUTE FUNCTION task( "drop chunk to storagepool", "<space_name>”, "<path>", "<offset>");

The space must be empty in order to be dropped.


Storage Provisioning Configuration

Instance level

– SP_AUTOEXPAND

– SP_THRESHOLD

– SP_WAITTIME

dbspace level

– Create size

– Extend size

Storage pool entry level

– Chunk size

– Priority


Storage Provisioning Configuration Parameters (1) SP_AUTOEXPAND

– Use this parameter to toggle on or off the automatic expansion of spaces.

– Affects both automatic chunk creations and automatic chunk extensions.

– Does not affect manual space creations or expansions.

Examples:SP_AUTOEXPAND 1 # Auto-expansion enabledSP_AUTOEXPAND 0 # Auto-expansion disabled

Default value: 1 (automatic space expansion is enabled.)


Storage Provisioning Configuration Parameters(2) SP_THRESHOLD

– An amount of free space, expressed as a percentage or an absolute number of kilobytes.

– If the free kilobytes in a space fall below this threshold, the next time the low-space-monitoring task runs, IDS will attempt to expand the space.

– Restriction: This setting applies to all spaces in the instance. You cannot configure this for each dbspace.

– Settings: – 0 disabled

– 1 – 50 - a percentage

– 51 - 999 - invalid

– 1000 - MAXCHUNKSIZE - actual size in KB

Examples:SP_THRESHOLD 10 # Ten percent of totalSP_THRESHOLD 100000 # 100 Mb

Default value: 0 (space expansion is disabled.)


Storage Provisioning Configuration Parameters (3)

SP_WAITTIME

– The number of seconds that a database server thread will wait for space expansion before returning an out-of-space error.

Examples:

– SP_WAITTIME 60 # Wait 1 minute, max

– SP_WAITTIME 0 # Don't wait at all

Default value: 30


Storage Provisioning: Space-Level Configuration

EXECUTE FUNCTION task ("modify space sp_sizes","<space name>", "<create_size>", "<extend_size>");

Create Size

– Minimum size of an automatically created chunk in this space.

Extend Size

– Minimum amount by which a chunk in this dbspace can be automatically extended.

Neither setting affects manual operations.

Express as a percentage or a number of kilobytes.– Default values: 10% and 10000k, respectively.


Storage Provisioning: Entry-Level Configuration

EXECUTE FUNCTION task("storagepool add", "<path>", "<offset>","<total_size>", "<chunk_size>", "<priority>");

Chunk Size

– Minimum size of a chunk created from this entry.

Priority

– 1 (High), 2 (Med), or 3 (Low)

– Storage pool entries will be evaluated, though not necessarily used, in priority order.


Storage Provisioning: Modifying/Deleting Pool Entries

EXECUTE FUNCTION task("storagepool modify","<entry_id>","<new_total_size>",

"<new_chunk_size>","<new_priority>");

EXECUTE FUNCTION task("storagepool delete","<entry_id>");

Note: You can find the correct entry_id value in the storagepool table in the sysadmin database (sysadmin:storagepool). You may have to know the chunk path and other related data beforehand ......


Manually Creating a Chunk from the Storage Pool (1)

• EXECUTE FUNCTION task("storagepool add“,”<path>","<offset>“,"<total_size>“,"<chunk_size “,"<priority>");

• execute function task ("storagepool add“, “/data/IBM/informix/demo/demo_on/chunk1“,"0“,"30 MB“,”30 MB“,"1");

There is no onstat option to see chunks allocated to the storage pool that are not allocated to a storage space. OAT must be used or “select * from sysadmin:storagepool where path matches ‘*chunk1*’”.


Manually Creating a Chunk from the Storage Pool (2)

EXECUTE FUNCTION task("create chunk from storagepool",

"<space_name>","<min_size>");

execute function task(“create chunk from storagepool”, ”rootdbs”,”30 MB”);

This command manually uses a chunk pre-created in the storage pool and adds it to the dbspace requested.


Manually Creating a Chunk from the Storage Pool (3) To make the chunk extensible:

execute function task("modify chunk extendable","5");

(expression) Chunk 5 is now extendable.


Error Reporting

execute function task("modify space expand", "tempdbs","25 MB");


Chunk Modifications (1)

execute function task("modify chunk extendable”,”3"); (expression) Chunk 3 is now extendable.

Now that the temp chunk is extendable, lets do the others …. or can we …..

execute function task("modify chunk extendable”,”2"); (expression) FAILED: Storage Provisioning - BLOBspace, Smart BLOBspace, and

mirrored chunks cannot be extended.



• execute function task("modify chunk extendable”,”1");• (expression) Chunk 1 is now extendable.

Too much unused space in the rootdbs. Return a chunk to the storage pool.

• execute function task ("drop chunk to storagepool","rootdbs",• "/data/IBM/informix/demo/storage_pool/chunk1","0");• (expression) Chunk successfully dropped.



The chunk is now restored to the storage pool.


Create a Storage Space from the Pool

execute function task ("create dbspace from storagepool","data1",“30 MB"); (expression) Space 'data1' added.

The dbspace data1 was created from the pool. Above we see that the rootdbs first chunk is 500000 pages. We will now load enough data into a table there to see it fill up and then expand. The space will

not be manually expanded. SP_AUTOEXPAND is 1 here.


Automatic Storage Expansion

Below we see the results after the space has filled up. The first chunk has gone from 500000 pages in size to 585000 pages in size with minimal pages free. The chunk expanded as expected.


Questions?

© 2011 IBM CorporationSeptember 9, 2010

Open Admin Tool (OAT)Storage Provisioning


Streamlined Storage Administration

The Space Administration > Storage pages:

Combine the information displayed in the Space Administration > Dbspaces and Chunks pages.

Display information about tables, extents, and the new storage pool.

Provide a central location for you to administer storage space with OAT.


Storage

The storage view displays summary information for server level spaces and usage:– Expand a panel for more information and actions.

– Use the tree to drill down on a particular space.


Storage > Spaces

The Spaces page displays information about the storage spaces on the server including:– An indicator of whether the space is expandable.

– The create and extend sizes that are used when the space is expanded.

You can create a new space or expand, modify, or drop an existing space.


Storage > Actions > Create Space

You can create a space and configure it as expandable:

The option to show the SQL is provided for each storage action.


Storage > Spaces > Expand Space

You can expand a space on demand:– Basic option: Expand the space by the amount configured as

the extend size. – Advanced option: Specify the amount to expand the space by

or add a chunk.


Storage > Server View > Chunks

The Chunks page displays detailed information about the chunks on the server.

You can add, drop or modify a chunk to make it extensible, or extend a chunk.


Storage > server_name> Storage Pool

Configure the database server for automatic storage space expansion:– The threshold sets the minimum space allowed before the space is expanded.– The wait time sets how long threads wait for a new space to be added.– You must the click “Automatic Space Expansion” box for it to be automatic.

Add, modify or delete storage pool objects.


Storage > server_name > Storage Pool

You can add a storage pool entry within OAT:


Storage > Information The information view graphically/textually displays the

summary used and free status of the database server by space type.

A “Optimization Policies” tab on the menu bar, controls the options for compress, repack, shrink and de-fragmentation ...


Storage > Information > Organization Policies (1)


Storage > Information > Organization Policies (2)

This is a database server wide set of policies.

By default the policies are disabled:– Can be enabled once configured.

If policy is enabled, options exist to threshold tune:– Compression - by number of rows in tables/fragments.– Repack - by % of noncontiguous storage.– Shrink - by % of unused space in tables/fragments.– Extents - in excess of a total number of table(s)/index(es) extents

A scheduler allows immediate scheduling of policy execution. You can also see the SQL generated to run this at the bottom

the screen by clicking the “Show SQL” button: – 'Long' SQL statements do not show in entirety. Arrow button down to

see all.


Storage > Space View

Information about the storage space selected in the tree including:– Summary graphic and text information and usage.– Tables and Indexes.– Chunk info.


Storage > space_name > Tables and Indexes View (1) This view is by dbspace (not shown), after clicking the expand window icon in the upper right hand window corner. Used / unused space is shown in red / green; full page usage is shown in purple, mostly used pages in red, partially used pages in green, and unused pages in blue. Tables/indexes with poor extent utilization are shown.


Storage > space_name > Tables and Indexes View (2) This view is obtained by clicking the window expand icon. The 'Actions'

tab on ‘Tables and Indexes’ allows 'Optimize Space' and 'Compression Estimates' for any object selected in the left hand column check box.


Help in Context

The cursor can be moved over any field to display more detailed information about the operation requested. A yellowish pop up box appears with context appropriate information.


Performance Analysis > onstat Utility

onstat -d has been enhanced with new flags column options:

– Dbspaces: In position 5, an 'A' marks the space as 'A'uto Expandable;

null means it is not auto expandable.– Chunks: In position 5, an 'E' marks the chunks as 'E'xpandable; a dash '-' means it not expandable.


Questions?


Where to get Open Admin Tool (OAT)

http://www.openadmintool.org

Its free !!!!


Logo

Documents

© 2011 IBM Corporation September 9, 2010 Storage Provisioning Scott Pickett – WW Informix Technical Sales For questions about this presentation contact: