Data Pump Help

One of the most significant characteristics of an export operation is its mode, because the mode largely

determines what is exported. Export provides different modes for unloading different portions of the database.

The mode is specified on the command line, using the appropriate parameter. The available modes are as

follows:

Full Export Mode

Schema Mode

Table Mode

Tablespace Mode

Transportable Tablespace Mode

Full Export Mode A full export is specified using the FULL parameter. In a full database export, the entire database is

unloaded. This mode requires that you have the EXP_FULL_DATABASE role.

Schema Mode A schema export is specified using the SCHEMAS parameter. This is the default export mode. If you have

the EXP_FULL_DATABASE role, then you can specify a list of schemas and optionally include the schema

definitions themselves, as well as system privilege grants to those schemas. If you do not have the

EXP_FULL_DATABASE role, you can export only your own schema.

Table Mode A table export is specified using the TABLES parameter. In table mode, only a specified set of tables,

partitions, and their dependent objects are unloaded. You must have the EXP_FULL_DATABASE role to

specify tables that are not in your own schema, and only one schema can be specified. Note that type

definitions for columns are not exported in table mode. It is expected that the type definitions already

exist in the target instance at import time. Also, as in schema exports, cross-schema references are not

exported.

Tablespace Mode A tablespace export is specified using the TABLESPACES parameter. In tablespace mode, only the tables

contained in a specified set of tablespaces are unloaded. If a table is unloaded, its dependent objects are

also unloaded. Both object metadata and data are unloaded. In tablespace mode, if any part of a table

resides in the specified set, then that table and all of its dependent objects are exported. You must have

the EXP_FULL_DATABASE role to use tablespace mode.

Transportable Tablespace Mode A transportable tablespace export is specified using the TRANSPORT_TABLESPACES parameter. In

transportable tablespace mode, only the metadata for the tables (and their dependent objects) within a

specified set of tablespaces are unloaded. This allows the tablespace datafiles to then be copied to

another Oracle database and incorporated using transportable tablespace import. This mode requires that

you have the EXP_FULL_DATABASE role.

In this sample control file, the numbers that appear to the left would not appear in a real control

file. They are keyed in this sample to the explanatory notes in the following list:

1. This is how comments are entered in a control file. See Comments in the Control File.

2. The LOAD DATA statement tells SQL*Loader that this is the beginning of a new data load. See Appendix A for syntax information.

3. The INFILE clause specifies the name of a datafile containing data that you want to load. See Specifying Datafiles.

4. The BADFILE parameter specifies the name of a file into which rejected records are placed. See Specifying the Bad File.

5. The DISCARDFILE parameter specifies the name of a file into which discarded records are placed. See Specifying the Discard File.

6. The APPEND parameter is one of the options you can use when loading data into a table that is not empty. See Loading Data into Nonempty Tables.

To load data into a table that is empty, you would use the INSERT parameter. See Loading

Data into Empty Tables.

7. The INTO TABLE clause allows you to identify tables, fields, and datatypes. It defines the relationship between records in the datafile and tables in the database. See Specifying Table

Names.

8. The WHEN clause specifies one or more field conditions. SQL*Loader decides whether or not to load the data based on these field conditions. See Loading Records Based on a Condition.

9. The TRAILING NULLCOLS clause tells SQL*Loader to treat any relatively positioned columns that are not present in the record as null columns. See Handling Short Records with Missing

Data.

10. The remainder of the control file contains the field list, which provides information about column formats in the table being loaded. See Chapter 6 for information about that section of

the control file.

Data Pump has three distinct components. The uniqueness of Data Pump is that all processing is done through the database server and through DIRECTORY

objects. To do this there are two internal packages to Oracle and an optional replacement for the original export and

import command line utilities (exp & imp).

expdp & impdp

These to command line utilities are very close to the old standby export & import (exp & imp) utilities. They are not

stand-alone utilities in the sense that they use the DBMS_DATAPUMP PL/SQL package to execute the export and import

functions. They accept a variety of command line options that, like exp & imp, allow you to pick and choose the

objects to be exported and imported.

DBMS_DATAPUMP

The Data Pump API and can be used independently of expdp & impdp. Is the package accessed to move data and / or

metadata between databases.

DBMS_METADATA

The meta-data API in Oracle and can also be used independently of expdp & impdp. If you remember this is the

package we were using in the last two articles for extracting meta-data. I am very interested in how it interfaces with

Data Pump.

Advantages and Disadvantages of Datapump

Advantages:

===> Better control on the job running it provides features like start, stop, restart

===> Improved performance because of It is server side technology with parallel

streams option

===> Using the parallel streams option, data pump can backup large volume of data

quickly

===> Data pump is 15-50% faster than the conventional export import.

===> It has the ability to estimate the job times

===> Failed jobs can be restarted

===> Using exclude/include option we can perform fine-grained object selection

===> Backup jobs can be monitored

===> It has the remapping capabilities

===> It supports the export/import operations over the network. The NETWORK_LINK

parameter initiate the export using a

database link

===> Using Query parameter DBA can extract the data from tables like SELECT

Content parameter gives the flexibility for what to import/export. For example

Metadata only, data or both

===> It supports full range of data types

===> It supports cross platform compatibility

===> No need to specify the buffer size like in normal exp/imp

===> It has its own performace tuning features

===> V$session_longops view can be used for the time estimation for the data pump

jobs

===> It supports interactive mode that allows the dba to monitor or interact with

ongoing jobs

===> Dumps will be in compressed

===> Data can be encrypted

===> XML schemas and XML type is supported by the Data Pump

Disadvantages:

===> Export cannot be taken to tape

===> Import will work only with Oracle 10g or above

===> Cannot use with Unix pipes

Data Pump Features

1.Writes either

Direct Path unloads External tables (part of cluster, has LOB, etc) 2.Command line interface

3.Writing to external tables

4.DBMS_DATAPUMP Data Pump API 5.DBMS_METADATA Metadata API 6.Checkpoint / Job Restart

Job progress recorded in Master Table - All stopped Data Pump jobs can be restarted without loss

of data as long as the master table and dump file set remain undisturbed while the job is stopped. It

doesnt matter if the job was stopped voluntarily by a user of if the stoppage was involuntary due to a

crash,power outage, etc.

May be stopped and restarted later

Abnormally terminated job is also restartable

Current objects can be skipped on restart if problematic

7.Better Job Monitoring and Control

Can detach from and attach to running jobs from any location - Multiple clients can attach to a job

to see what is going on. Clients may also detach from an executing job without affecting it.

Initial job space estimate and overall percent done - At Export time, the approximate size of the

job is estimated before it gets underway. The default method for determining this is to estimate the size of

a partition by counting the number of blocks currently allocated to it. If tables have been

analyzed, statisticscan also be used which should provide a more accurate estimate. The user gets an

estimate of how much dump file space will be consumed for the operation.

Job state and description - Once the Export begins, the user can get a status of the job by seeing a

percentage of how far along he is in the job. He can then extrapolate the time required to get the job

completed.

Per-worker status showing current object and percent done

Enterprise Manager interface available - The jobs can be monitored from any location

8. Interactive Mode for expdp and impdp clients

PARALLEL: add or remove workers

ADD_FILE: add dump files and wildcard specs

STATUS: get detailed per-worker status

STOP_JOB {=IMMEDIATE}: stop the job, leaving it restartable, immediate doesnt wait for workers

to finish current work items.

START_JOB: Restart a previously stopped job, can change reporting interval

KILL_JOB: stop job and delete all its resources, leaving it unrestartable, the master table and dump

files are deleted

CONTINUE: leave interactive mode, continue logging

EXIT: exit client, leave job running

All modes of operation are supported: full, schema, table, tablespace, and transportable tablespace.

9.Dumpfile Set Management

Directory based: e.g., DMPDIR:export01.dmp where DMPDIR is external directory

Can specify maximum size of each dumpfile

Can dynamically add dumpfiles to jobs - If a job ran out of space, can use ADD_FILE command and

specify a FILESIZE value

Wildcard file specs supported - Wildcard file support makes it easy to spread the I/O load over

multiple spindles:

e.g.: Dumpfile=dmp1dir:full1%u.dmp, dmp2dir:full2%u.dmp

Dump file compression of metadata - Metadata is compressed before being written to the dumpfile

set COMPRESSION=METADATA_ONLY

In Oracle Database 11g, this compression capability has been extended so that we can

now compress table data on export. Data Pump compression is an inline operation, so the reduced

dumpfile size means a significant savings in disk space.

Automatically uncompressed during Import. Data Pump compression is fully inline on the import

side as well, so there is no need to uncompress a dumpfile before importing it.

Dumpfile set coherency automatically maintained

Data Pump supplies encryption options for more flexible and robust security.

10. Network Mode

Data Pump Export and Import both support a network mode in which the jobs source is a remote Oracle instance. This is an overlap of unloading the data, using Export, and loading the data, using Import, so

those processes dont have to be serialized. A database link is used for the network. We dont have to worry about allocating file space because there are no intermediate dump files.

Network Export

Unload a remote database to a local dump file set Allows export of read-only databases for archiving

Network Import

Overlap execution of extract and load No intermediate dump files

Because Data Pump maintains a Master Control Table and must perform database writes, Data Pump cant directly Export a Read-only database. Network mode allows the user to export Read-Only databases:

The Data Pump Export job runs locally on a read/write instance and extracts the data and metadata from

the remote read-only instance. Both networkexport and import use database links to communicate with

the remote source.

First level parallelism is supported for both network export and import. I/O servers do not operate

remotely, so second level, intra-partition parallelism is not supported in network operations.

11.Fine-Grained Object Selection

All object types are supported - With the new EXCLUDE and INCLUDE parameters, a Data Pump job

can include or exclude any type of object and any subset of objects within a type.

Exclude parameter: specified object types are excluded from the operation

Include parameter: only the specified object types are included

Both take an optional name filter for even finer granularity:

INCLUDE/ PACKAGE: LIKE PAYROLL%

EXCLUDE TABLE: in (FOO,BAR,)

e.g.:

EXCLUDE=function

EXCLUDE=procedure

EXCLUDE=package:like PAYROLL%

Would exclude all functions, procedures, and packages with names starting with PAYROLL from the job.

Using INCLUDE instead of EXCLUDE above, would include the functions, procedures, and packages with

names startingwith PAYROLL.

12.DDL Transformations

Easy with XML, because object metadata is stored as XML in the dump file set,it is easy to apply

transformations when DDL is being formed (via XSL-T) during import.

REMAP_SCHEMA -> REMAP_SCHEMA provides the old FROMUSER / TOUSER capability to change

object ownership.

REMAP_TABLESPACE -> REMAP_TABLESPACE allows objects to be moved from one tablespace to

another. This changes the tablespace definition as well

REMAP_DATAFILE -> REMAP_DATAFILE is useful when moving databases across platforms that

have different file system semantics.

Segment and storage attributes can be suppressed -> The TRANSFORM parameter can also be

used so that storage clauses are not generated in the DDL. This is useful if the storage characteristics of

the target instance are very different from those of the source.

Data Pump Benefits (advantages over normal export & import)

o Restartable

o Improved control

o Files will created on server, not on client side

o Parallel execution

o Online writing course

o A Jobs

o Web Designers

o Locations

o Intermediate

o Packages

o Automated performance tuning

o Simplified monitoring

o Improved object filtering

o Dump will be compressed

o Data can be encrypted (in Oracle 11g or later)

o Remap of data during export or import (in 11g or later)

o We can export one or more partitions of a table without having to move the entire table (in 11g or

later)

o XML schemas and XMLType columns are supported for both export and import (in 11g or later)

o Using the Direct Path method of unloading or loading data, a single stream of Data Pump export

(unload) is approximately 2 times faster than original Export, because the Direct Path API has been

modified to be even more efficient. Depending on the level of parallelism, the level of improvement can be

much more.

o Original Import uses only conventional mode inserts, so a single stream of Data Pump Import is

10-45 times faster than normal Import. As with Export, the jobs single stream can be changed to

parallel streams for even more improvement.

o With Data Pump, it is much easier for the DBA to manage and monitor jobs. During a long-running

job, the DBA can monitor a job from multiple locations and know how far along it is, how much there is

left to go, what objects are being worked on, etc. The DBA can also affect the jobs operation, i.e. abort it,

adjust its resource consumption, and stop it for later restart.

o Since the jobs are completed much more quickly than before, production systems have less

downtime.

o Data Pump is publicly available as a PL/SQL package (DBMS_DATAPUMP), so customers can write

their own data movement utilities if so desired. The metadata capabilities of the Data Pump are also

available as a separate PL/SQL package, DBMS_METADATA.

o While importing, if destination schema is not existed, Data Pump will create the user and import

the objects.

Data Pump requires no special tuning. Data Pump runs optimally out of the box. Original Export and (especially) Import require careful tuning to achieve optimum results. There are no Data

Pump performance tuning parameters other than the ability to dynamically adjust the degree of

parallelism.

We can dynamically throttle the number of threads of execution throughout the lifetime of the job. There

is an interactive command mode where we can adjust the level of parallelism. For example, we can start

up a job during the day with a PARALLEL=2, and then increase it at night to a higher level.

All the Oracle database data types are supported via Data Pumps two data movement mechanisms, Direct Path and External Tables.

With Data Pump, there is much more flexibility in selecting objects for unload and load operations. We can

now unload any subset of database objects (such as functions, packages, and procedures) and reload

them on the target platform. Almost all database object types can be excluded or included in an operation

using the new Exclude and Include parameters.

We can either use the Command line interface or the Oracle Enterprise Manager web-based GUI interface.

Data Pump handles all the necessary compatibility issues between hardware platforms and operating

systems.

Oracle Data Pump supports Oracle Apps 11i.

We can use the ESTIMATE ONLY command to see how much disk space is required for the jobs dump file set before we start the operation.

Jobs can be monitored from any location is going on. Clients may also detach from an executing job

without affecting it.

Every Data Pump job creates a Master Table in which the entire record of the job is maintained. The

Master Table is the directory to the job, so if a job is stopped for any reason, it can be restarted at a later

point in time, without losing any data. Whenever Data Pump export or import is running, Oracle will create

a table with the JOB_NAME and will be deleted once the job is done. From this table, Oracle will find out

how much job has been completed and from where to continue etc.

With Data Pump, it is now possible to change the definition of some objects as they are created at import

time. For example, we can remap the source datafile name to the target datafile name in all DDL

statements where the source datafile is referenced. This is really useful if we are moving across platforms

with different file system syntax.

Data Pump supports the Flashback infrastructure, so we can perform an export and get a dumpfile set

that is consistent with a specified point in time or SCN.

Data Pump Vs SQL*Loader We can use SQL*Loader to load data from external files into tables of an Oracle database. Many

customers use SQL*Loader on a daily basis to load files (e.g. financial feeds) into their databases. Data

Pump Export and Import may be used less frequently, but for very important tasks, such as migrating

between platforms, moving data between development, test, and production databases, logical database

backup, and for application deployment throughout a corporation.

Data Pump Vs Transportable Tablespaces We can use Transportable Tablespaces when we want to move an entire tablespace of data from one

Oracle database to another. Transportable Tablespaces allows Oracle data files to be unplugged from a

database, moved or copied to another location, and then plugged into another database. Moving data

using Transportable Tablespaces can be much faster than performing either an export or import of the

same data, because transporting a tablespace only requires the copying of datafiles and integrating the

tablespace dictionary information. Even when transporting a tablespace, Data Pump Export and Import

are still used to handle the extraction and recreation of the metadata for that tablespace.

The various background processes related to Data Pump are:

Data Pump Master Process (DMnn) : Creates and deletes the master table at the time of export and

import. Master table contains the job state and object information.

Coordinates the Data Pump job tasks performed by Data Pump worker processes and handles client

interactions. The Data Pump master (control) process is started during job creation and coordinates all

tasks performed by the Data Pump job. It handles all client interactions and communication, establishes

all job contexts, and coordinates all worker process activities on behalf of the job. Creates the Worker

Process.

Data Pump Worker Process (DWnn) : It performs the actual heavy duty work of loading and unloading of

data. It maintains the information in master table. The Data Pump worker process is responsible for

performing tasks that are assigned by the Data Pump master process, such as the loading and unloading

of metadata and data.

Shadow Process : When client logs in to an Oracle Server the database creates and Oracle process to

service Data Pump API.

Client Process : The client process calls the Data pump API.

Data Pump Disadvantages Cant use UNIX pipes Can't run as SYS (/ as sysdba)

How Data Pump accesses loading and unloading of Data:

Oracle has provided direct path to unload or export operations since Oracle 7.3. This method has been very

useful for DBAs that want a quick export of the database and this process has been further enhanced in the

Data Pump technology. Oracle uses the direct path method for loading (impdp) and unloading (expdp) when the

structure of the tables allows it. If the table is part of a cluster, or it has a global index on a partitioned table,

then Data Pump accesses the data in a different method called External Table. Both the direct path load and

external table method support the same external data representation, so we can load the data that was unloaded with External Table method and vice versa.

Getting Started...

As stated earlier, Data Pump is a server-based utility, rather than client-based; dump files, log files, and SQL

files are accessed relative to server-based directory paths. Data Pump requires you to specify directory paths as directory objects. A directory object maps a name to a directory path on the file system.

1. The following SQL statements creates a user, a directory object named dpump_dir1 and grants the permissions to the user.

SQLPLUS system/manager@TDB10G as sysdba

SQL> create user dpuser identified by dpuser;

SQL> grant connect, resource to dpuser;

SQL> CREATE DIRECTORY dpump_dir1 AS '/opt/app/oracle';

SQL> grant read, write on directory dpump_dir1 to dpuser

2. Let us see how the INCLUDE and EXCLUDE parameters can be used to limit the load and unload of particular

objects. When the INCLUDE parameter is used, only the objects specified by it will be included in the export.

When the EXCLUDE parameter is used, all objects except those specified by it will be included in the export: Assume we have EMP,EMP_DETAILS and DEPT tables owned by dpuser.

$ expdp dpuser/dpuser@TDB10G schemas=dpuser include= TABLE:\"IN (\'EMP\', \'DEPT\')\"

directory=dpump_dir1 dumpfile=dpuser.dmp log=dpuser.log

$expdp dpuser/dpuser@TDB10G schemas=dpuser exclude=TABLE:\"= \'EMP_DETAILS\'\"

directory=dpump_dir1 dumpfile=dpuser2.dmp logfile=dpuser.log

As stated earlier, Data pump performance can be significantly improved by using thePARALLEL parameter. This

should be used in conjunction with the "%U" wildcard in theDUMPFILE parameter to allow multiple dumpfiles to be created or read:

$expdp dpuser/dpuser@TDB10G schemas=dpuser

directory=dpump_dir1 parallel=4 dumpfile=dpuser_%U.dmp logfile=dpuser.log

Data Pump API:

The Data Pump API, DBMS_DATAPUMP, provides a high-speed mechanism to move the data from one database

to another. Infact, the Data Pump Export and Data Pump Import utilities are based on the Data Pump API. The

structure used in the client interface of this API is a job handle. Job handle can be created using the OPEN or

ATTACH function of the DBMS_DATAPUMP package. Other DBA sessions can attach to a job to monitor and control its progress so that remote DBA can monitor the job that was scheduled by an on-site DBA.

The following steps list the basic activities involved in using Data Pump API.

1. Execute DBMS_DATAPUMP.OPEN procedure to create job.

2. Define parameters for the job like adding file and filters etc.

3. Start the job.

4. Optionally monitor the job until it completes.

5. Optionally detach from job and attach at later time.

6. Optionally, stop the job

7. Restart the job that was stopped.

Example of the above steps:

Declare

P_handle number; --- -- Data Pump job handle

P_last_job_state varchar2(45); ---- -- To keep track of job state

P_job_state varchar2(45);

P_status ku$_Status ----- -- The status object returned by get_status

BEGIN

P_handle:=DBMS_DATAPUMP.OPEN ('EXPORT','SCHEMA', NULL,'EXAMPLE','LATEST');

-- Specify a single dump file for the job (using the handle just returned)

-- and a directory object, which must already be defined and accessible

-- to the user running this procedure

DBMS_DATAPUMP.ADD_FILE (p_handle,'example.dmp','DMPDIR');

-- A metadata filter is used to specify the schema that will be exported.

DBMS_DATAPUMP.METADATA_FILTER (p_handle,'SCHEMA_EXPR','IN (''dpuser'')');

-- Start the job. An exception will be generated if something is not set up

-- Properly.

DBMS_DATAPUMP.start_job (p_handle);

----The export job should now be running.

The status of the job can be checked by writing a separate procedure and capturing the errors and status until it

is completed. Overall job status can also be obtained by querying SELECT * from dba_datapump_jobs.

Conclusion:

Oracle Data Pump is a great tool for the fast movement of data between the databases and much of this

performance improvement is derived from the use of parameter parallelism. Even when the Transportable Tablespace feature of Oracle is used to move self-contained data between the databases, Data Pump is still

required for handling the extraction and recreation of the metadata for that tablespace. Whenever possible,

Data Pump performance is further maximized by using Direct-Path driver. Otherwise, Data Pump accesses the

data using an External Table access driver.Data Pump provides flexibility, with the implementation of

parameters such as INCLUDE, EXCLUDE, QUERY, and TRANSFORM that gives the DBA more control of data and

objects being loaded and unloaded. With all of these features, Data Pump is a welcome addition to DBA tools in a world that constantly redefines the size of the large database.

How Does Data Pump Access Data?

Data Pump supports two access methods to load and unload table row data: direct path and external

tables. Because both methods support the same external data representation, data that is unloaded

with one method can be loaded using the other method. Data Pump automatically chooses the

fastest method appropriate for each table.

Data Pump also uses functionality provided in the DBMS_METADATA PL/SQL package to handle all

operations involving metadata, including complete extraction, transformation, and re-creation of all

database object definitions.

Direct Path Loads and Unloads

The Oracle database has provided direct path unload capability for export operations since Oracle

release 7.3 and a direct path API for OCI since Oracle8i. Data Pump technology enhances direct

path technology in the following ways:

Support of a direct path, proprietary format unload.

Improved performance through elimination of unnecessary conversions. This is possible

because the direct path internal stream format is used as the format stored in the Data Pump

dump files.

Support of additional datatypes and transformations.

Data Pump uses direct path load and unload when the structure of a table allows it.

In the following circumstances, Data Pump cannot use direct path loading:

A global index on multipartition tables exists during a single-partition load. This includes

object tables that are partitioned.

A domain index exists for a LOB column.

A table is in a cluster.

A table has an active trigger.

A table has fine-grained access control enabled in insert mode.

A table contains BFILE columns or columns of opaque types.

A referential integrity constraint is present.

A table contains VARRAY columns with an embedded opaque type.

If any of these conditions exist for a table, Data Pump uses external tables rather than direct path to

move the data for that table.

External Tables

The Oracle database has provided an external tables capability since Oracle9i that allows reading of

data sources external to the database. As of Oracle Database 10g, the external tables feature also

supports writing database data to destinations external to the database. Data Pump provides an

external tables access driver(ORACLE_DATAPUMP) that reads and writes files. The format of the files is

the same format used with the direct path method. This allows for high-speed loading and

unloading of database tables. Data Pump uses external tables as the data access mechanism in the

following situations:

Loading and unloading very large tables and partitions in situations where parallel SQL can

be used to advantage

Loading tables with global or domain indexes defined on them, including partitioned object

tables

Loading tables with active triggers or clustered tables

Loading and unloading tables with encrypted columns

Loading tables with fine-grained access control enabled for inserts

Loading tables that are partitioned differently at load time and unload time

Query Parameter in Datapump (EXPDP/IMPDP) utility

Sometimes we need to export or imprt only some data from a huge table. This can be done by using

query parameter in Datapump utility. Query parameter can be used in both EXPDP as well as IMPDP

utility.

1. EXPDP :-

Query parameter can be used in two ways :-

a. Using parameter file :-

bash-3.2$

bash-3.2$ cat test.par

TABLES=AMIT

directory=datapump

DUMPFILE=expdp_test.dmp

logfile=expdp_test.log

query=amit:"where DATE > '01-DEC-2013'"

bash-3.2$

bash-3.2$ expdp user/user parfile=test.par

Export: Release 11.2.0.1.0 - Production on Fri Dec 13 12:20:47 2013

Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

Starting "USER"."SYS_EXPORT_TABLE_01": user/****** parfile=test.par

Estimate in progress using BLOCKS method...

Processing object type TABLE_EXPORT/TABLE/TABLE_DATA

Total estimation using BLOCKS method: 384 KB

Processing object type TABLE_EXPORT/TABLE/TABLE

Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX

Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

. . exported "USER"."AMIT" 30.67 KB 75 rows

Master table "USER"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded

******************************************************************************

Dump file set for USER.SYS_EXPORT_TABLE_01 is:

/disk1/datapump/expdp_etst.dmp

Job "USER"."SYS_EXPORT_TABLE_01" successfully completed at 12:20:53

b. In Command Line :-

expdp user/user directory=datapump dumpfile=expdp_taa_01.dmp logfile=expdp_taa_01.log

query=amit:\"where DATE \> \'01-DEC-2013\'\" tables=AMIT

Export: Release 11.2.0.1.0 - Production on Fri Dec 13 12:24:50 2013




Starting "USER"."SYS_EXPORT_TABLE_01": user/******** directory=datapump dumpfile=expdp_taa_01.dmp

logfile=expdp_taa_01.log query=amit:"where DATE > '01-DEC-2013'" tables=AMIT

Estimate in progress using BLOCKS method...

Processing object type TABLE_EXPORT/TABLE/TABLE_DATA

Total estimation using BLOCKS method: 384 KB

Processing object type TABLE_EXPORT/TABLE/TABLE

Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX

Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

. . exported "USER"."AMIT" 30.67 KB 75 rows

Master table "USER"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded

******************************************************************************

Dump file set for USER.SYS_EXPORT_TABLE_01 is:

/disk1/datapump/expdp_taa_01.dmp

Job "USER"."SYS_EXPORT_TABLE_01" successfully completed at 12:24:56

2. IMPDP :-

a. Using par file :-

bash-3.2$ cat test.par

schemas=DATA

include=TABLE:"IN('TEST')"

directory=datapump

DUMPFILE=expdp_fullDB_12_Dec_13.dmp

logfile=expdp_etst.log

query=test:"where DATE > '01-DEC-2013'"

remap_schema=DATA:AMIT

bash-3.2$ impdp amit/amit parfile=test.par

Import: Release 11.2.0.1.0 - Production on Fri Dec 13 12:44:22 2013




Master table "AMIT"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded

Starting "AMIT"."SYS_IMPORT_SCHEMA_01": amit/******** parfile=test.par

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA

. . imported "AMIT"."TEST" 51.17 KB 75 out of 1614 rows

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/INDEX

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/STATISTICS/TABLE_STATISTICS

Job "AMIT"."SYS_IMPORT_SCHEMA_01" successfully completed at 12:44:44

b. In Command line :-

bash-3.2$ impdp amit/amit directory=datapump dumpfile=expdp_fullDB_12_Dec_13.dmp logfile=test.log

query=TEST:\"where DATE \> \'01-DEC-2013\'\" schemas= DATA include=TABLE:\"IN$\'TEST\'$\"

remap_schema=DATA:AMIT

Import: Release 11.2.0.1.0 - Production on Fri Dec 13 12:38:29 2013




Master table "AMIT"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded

Starting "AMIT"."SYS_IMPORT_SCHEMA_01": amit/******** directory=datapump

dumpfile=expdp_fullDB_12_Dec_13.dmp logfile=test.log query=test:"where DATE > '01-DEC-2013'"

schemas=DATA include=TABLE:"IN('TEST')" remap_schema=DATA:AMIT

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA

. . imported "AMIT"."TEST" 51.17 KB 75 out of 1614 rows

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/INDEX

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/STATISTICS/TABLE_STATISTICS

Job "AMIT"."SYS_IMPORT_SCHEMA_01" successfully completed at 12:38:55

The Bad File

The bad file contains records that were rejected, either by SQL*Loader or by the Oracle database. If you do not specify a

bad file and there are rejected records, then SQL*Loader automatically creates one. It will have the same name as the

data file, with a.bad extension. Some of the possible reasons for rejection are discussed in the next sections.

The Discard File

As SQL*Loader executes, it may create a file called the discard file. This file is created only when it is needed, and only if

you have specified that a discard file should be enabled. The discard file contains records that were filtered out of the

load because they did not match any record-selection criteria specified in the control file.

The discard file therefore contains records that were not inserted into any table in the database. You can specify the

maximum number of such records that the discard file can accept. Data written to any database table is not written to

the discard file.

What Is a Control File?

Every Oracle Database has a control file, which is a small binary file that records the physical structure of the

database. The control file includes:

The database name

Names and locations of associated datafiles and redo log files

The timestamp of the database creation

The current log sequence number

Checkpoint information

The control file must be available for writing by the Oracle Database server whenever the database is open.

Without the control file, the database cannot be mounted and recovery is difficult.

The control file of an Oracle Database is created at the same time as the database. By default, at least one copy

of the control file is created during database creation. On some operating systems the default is to create

multiple copies. You should create two or more copies of the control file during database creation. You can also

create control files later, if you lose control files or want to change particular settings in the control files.

SQL*Loader

Maximazing SQL*Loader Performance

Use Direct Path Loads - The conventional path loader essentially loads the data by using standard insert

statements. The direct path loader (direct=true) loads directly into the Oracle data files and creates blocks in

Oracle database block format. There are certain cases, however, in which direct path loads cannot be used

(clustered tables). To prepare the database for direct path loads, the script

$ORACLE_HOME/rdbms/admin/catldr.sql.sql must be executed.

Disable Indexes and Constraints. For conventional data loads only, the disabling of indexes and

constraints can greatly enhance the performance.

Use a Larger Bind Array. For conventional data loads only, larger bind arrays limit the number of calls to

the database and increase performance. The size of the bind array is specified using the bindsize

parameter. The bind array's size is equivalent to the number of rows it contains (rows=) times the maximum

length of each row.

Use ROWS=n to Commit Less Frequently. For conventional data loads only, the rows parameter

specifies the number of rows per commit. Issuing fewer commits will enhance performance.

Use Parallel Loads. Available with direct path data loads only, this option allows multiple SQL*Loader

jobs to execute concurrently.

$ sqlldr control=first.ctl parallel=true direct=true

$ sqlldr control=second.ctl parallel=true direct=true

Use Fixed Width Data. Fixed width data format saves Oracle some processing when parsing the data. The

savings can be tremendous.

Disable Archiving During Load. While this may not be feasible in certain environments, disabling

database archiving can increase performance considerably.

Use unrecoverable. The unrecoverable option (unrecoverable load data) disables the writing of the data to

the redo logs. This option is available for direct path loads only.

Using the table table_with_one_million_rows, the following benchmark tests were performed with the various

SQL*Loader options. The table was truncated after each test.

SQL*Loader Option Elapsed Time (Seconds) Time Reduction

direct=false

rows=64

135 -

direct=false

bindsize=512000

rows=10000

92 32%

direct=false

bindsize=512000

rows=10000

database in noarchivelog

85 37%

direct=true 47 65%

SQL*Loader Option Elapsed Time (Seconds) Time Reduction

direct=true

unrecoverable

41 70%

direct=true

unrecoverable

fixed width data

41 70%

The results above indicate that conventional path loads take the longest. However, the bindsize and rows

parameters can aid the performance under these loads. The test involving the conventional load didnt come close to the performance of the direct path load with the unrecoverable option specified.

It is also worth noting that the fastest import time achieved for this table (earlier) was 67 seconds, compared to

41 for SQL*Loader direct path a 39% reduction in execution time. This proves that SQL*Loader can load the same data faster than import. These tests did not compensate for indexes. All database load operations will

execute faster when indexes are disabled.

SQL*Loader Control File The control file is a text file written in a language that SQL*Loader understands. The control file describes the

task that the SQL*Loader is to carry out. The control file tells SQL*Loader where to find the data, how to parse

and interpret the data, where to insert the data, and more. See Chapter 4, "SQL*Loader Case Studies" for

example control files.

Although not precisely defined, a control file can be said to have three sections:

1. The first section contains session-wide information, for example: o global options such as bindsize, rows, records to skip, etc.

o INFILE clauses to specify where the input data is located

o data character set specification

2. The second section consists of one or more "INTO TABLE" blocks. Each of these blocks contains information about the table into which the data is to be loadedsuch as the table name and the columns of

the table.

3. The third section is optional and, if present, contains input data.

Documents

Data Pump Help