Advanced Queuing & E-Business Suite& E-Business Suite

By Warwick Sands

Senior Developer at DHL Logistics&

Jurijs VelikanovsLead Oracle Apps DBA

Rolta TUSC Asia Pacific Ltd

Agenda for Meet Up #7

• Introduction

• AQ Concepts

• AQ in Development

• AQ & e-Business Suite (DBA)• AQ & e-Business Suite (DBA)

Warwick Sands

• I am 58 years old

• Father of a son and a daughter

– No grandchildren

• Started

• Clothing manufacturer until late ‘70s

• Z80 Assembly language programmer (early ‘80s)• Z80 Assembly language programmer (early ‘80s)

– Wrote extensions to Basic and DOS

• Finally ended up in the Oracle environment in 1989

• Currently working at DHL Logistics – since 2005

– Speaker • Uniface Trainer 1996 – 1998• AusOUG (2002/2003/2004)

Jurijs Velikanovs

• I am 30 years old

• Farther of two sons

• Originally from Latvia

• Started

• as Oracle DBA in 1997 (11+)

• as Apps DBA in 2001 (7+)

or Yury Velikanov

• as Apps DBA in 2001 (7+)

• OCP 7/8/8i/9i/10g OCM 9i/10G• Oracle 11i Applications Administrator Certified Professional • Oracle 11i Applications Workflow Certified Experts

– Speaker • UKOUG (2005/2006/2007/2008)• US Hotsos (2005/2006)• Australia InSynch (2009)

What is Advanced Queuing? What for?

• Typical problems to solve using AQ module

– Good examples• Multiple tasks distribution across several workers/processes (billing system)• Many to Many Producers/Consumers messages processing• Deferred actions processing (EBS)

– Questionable examples• Auditing• High volumes data processing• C

How important AQ for EBS?

09:39:44 SYSTEM:OAPROD> SELECT owner, COUNT (*)

09:39:45 SYSTEM:OAPROD> FROM dba_queues09:39:45 SYSTEM:OAPROD> GROUP BY CUBE (owner)

09:39:45 SYSTEM:OAPROD> ORDER BY COUNT (*);

OWNER COUNT(*)

------------------------------ ----------

ODM 2

IBU 2

PO 2

AMV 2

OKC 3

CCT 4CCT 4

CS 4

AR 4

IEO 4

IEM 4

ASO 5

JTF 6

SYSTEM 8

XLA 9

XNP 12

SYS 15

XDP 15

APPS 20

APPLSYS 56

177

AQ usage in Apps DBA area

• Execution of DIFFERED activities

– Asynchronous workflow processing

• Order processing

• Order approvals

– Event processing (Business Event System)

• Data transfer (EBS internally & externally)

• Web servers cache invalidation

• Privileges management

– EBS Interfaces

• Email-s sending (WF Notification Mailer)

What is Advanced Queuing?

• Advanced Queuing is producer/consumer messaging interface– Producer/Enqueue => queue => Consumer/Dequeue– persistent messaging solution (could be buffered starting from 10GR2)

• An Oracle module based on– Set of DB objects (TABLES/IOT/INDEXES/VIEWS)

– PL/SQL API (DBMS_AQ/DBMS_AQADM)

– Oracle Background processes (aq_tm_processes)

• AQ background processes (ora_qmnc/ora_q00x)

– Managing time related functionality (delays, expirations, purging)

Short AQ demonstration

• Goals– On a simple example show AQ concepts

– Give you a simple code to play with– Give you a simple code to play with

– Show main underlying DB objects

Let create a Queue and Test it !

• Create a queue table

– Create type to be used as USER_DATA/PAYLOAD_TYPE in the queue table. There are two options:

• RAW

• User type (of any complexity)CREATE OR REPLACE TYPE meetup_msgtype AS OBJECT (

text_value VARCHAR2 (100),

number_value number,

date_value date

);

/

desc meetup_msgtype

– Create queue table (using AQ admin API => DBMS_AQADMIN)BEGIN

DBMS_AQADM.create_queue_table (queue_table => 'meetup_qtab',

queue_payload_type => 'meetup_msgtype'

);

END;

/

desc meetup_qtab

Let create a Queue and Test it !

• Create queueBEGIN

DBMS_AQADM.create_queue (queue_name => 'meetup_msgqueue01',

queue_table => 'meetup_qtab'

);

END;

/

• Start the queue• Start the queueBEGIN

DBMS_AQADM.start_queue (queue_name => 'meetup_msgqueue01');

END;

/

Let create a Queue and Test it !

• Now we are ready to enqueue a message DECLARE

enqueue_options DBMS_AQ.enqueue_options_t;

message_properties DBMS_AQ.message_properties_t;

message_handle RAW (16);

MESSAGE meetup_msgtype;

BEGIN

MESSAGE := meetup_msgtype ('IT IS A TESTING MESSAGE', 1, SYSDATE);

DBMS_AQ.enqueue (queue_name => 'meetup_msgqueue01',

enqueue_options => enqueue_options,

message_properties => message_properties,

payload => MESSAGE,

msgid => message_handle

);

COMMIT;

END;

/

select * from MEETUP_QTAB;

select * from AQ$MEETUP_QTAB;

Main underlying DB objects

• There are several DB objects created to support AQcolumn object_name format a30

column object_type format a30

select object_name, object_type from dba_objects where object_name like '%MEETUP%'

order by 2, 1;

OBJECT_NAME OBJECT_TYPEOBJECT_NAME OBJECT_TYPE

------------------------------ ------------------------------

AQ$_MEETUP_QTAB_I INDEX – index used by AQ backgraund processes

AQ$_MEETUP_QTAB_T INDEX -- time_manager_info

AQ$_MEETUP_QTAB_E QUEUE -- error queue in case of dequeueing fails

MEETUP_MSGQUEUE01 QUEUE -- a main queue we are working with

MEETUP_QTAB TABLE -- a table where all data is stored

MEETUP_MSGTYPE TYPE

AQ$MEETUP_QTAB VIEW –- a friendly view created for us

AQ$_MEETUP_QTAB_F VIEW

8 rows selected.

-- SYS_XXXXXXXXX INDEX -- msg_id

Main underlying DB objects

• There are several indexes created to support AQset lines 300

column index_name format a30

column column_name format a30

SELECT index_name, column_name, column_position FROM dba_ind_columns a

WHERE table_name = 'MEETUP_QTAB' and table_owner = 'SYSTEM'

ORDER BY index_owner, table_name, index_name, column_position;ORDER BY index_owner, table_name, index_name, column_position;

INDEX_NAME COLUMN_NAME COLUMN_POSITION

------------------------------ ------------------------------ ---------------

AQ$_MEETUP_QTAB_I Q_NAME 1

AQ$_MEETUP_QTAB_I STATE 2

AQ$_MEETUP_QTAB_I ENQ_TIME 3

AQ$_MEETUP_QTAB_I STEP_NO 4

AQ$_MEETUP_QTAB_I CHAIN_NO 5

AQ$_MEETUP_QTAB_I LOCAL_ORDER_NO 6

AQ$_MEETUP_QTAB_T TIME_MANAGER_INFO 1

SYS_C006549 MSGID 1

8 rows selected.

A quick look on a queue table

• select * from AQ$MEETUP_QTAB;01:00:25 APPS:ENV01> desc AQ$MEETUP_QTABName Type

--------------------------------------

(1) QUEUE VARCHAR2(30) -- Queue name. Multiple queues can be created on the same table (if same msg type)(2) MSG_ID NOT NULL RAW(16) -- Each MSG have unique identifier (!!HEXTORAW

('286A342F44A92260E0440016357312E4')!!)(3) CORR_ID VARCHAR2(128)-- Application/user provided meaningful property for a message (non unique)

MSG_PRIORITY NUMBER --

(4) MSG_STATE VARCHAR2(9) -- State of this message. Ready – ready to be enqueued (WAITING, PROCESSED, EXPIRED)(5) DELAY DATE -- Date starting from message have to be dequeued (if during enqueue delay was set)(6) EXPIRATION NUMBER -- number of seconds in which the message will expire after being READY(7) ENQ_TIME DATE0 -- A time message has been enqueued (a record inserted in queue table)(7) ENQ_TIME DATE0 -- A time message has been enqueued (a record inserted in queue table)

ENQ_USER_ID NUMBER -- user_id from DBA_USERS

ENQ_TXN_ID VARCHAR2(30) -- enqueue transaction id

(8) DEQ_TIME DATE -- Dequeue timeDEQ_USER_ID NUMBER

DEQ_TXN_ID VARCHAR2(30)

(9) RETRY_COUNT NUMBER -- number of dequeue retriesEXCEPTION_QUEUE_OWNER VARCHAR2(30) -- where to put exception message if dequeue failed several times

(10) EXCEPTION_QUEUE VARCHAR2(30) -- (if not default error queue is used)(11) USER_DATA MEETUP_MSGTYPE-- a message itself, type of this column is set during creation of the queue

ORIGINAL_QUEUE_NAME VARCHAR2(30)

ORIGINAL_QUEUE_OWNER VARCHAR2(30)

EXPIRATION_REASON VARCHAR2(18)

SENDER_NAME VARCHAR2(30)

SENDER_ADDRESS VARCHAR2(1024

SENDER_PROTOCOL NUMBER

ORIGINAL_MSGID RAW(16)

• select * from MEETUP_QTAB; -- a table has almost the same fields (exception TIME_MANAGER_INFO)

(12)

How to dequeue message?

• To dequeue use the following PL/SQLDECLARE

dequeue_options DBMS_AQ.dequeue_options_t;

message_properties DBMS_AQ.message_properties_t;

message_handle RAW (16);

MESSAGE meetup_msgtype;

BEGIN

DBMS_AQ.dequeue (queue_name => 'meetup_msgqueue01',

dequeue_options => dequeue_options,

message_properties => message_properties,

payload => MESSAGE,

msgid => message_handle

); -- At that point session communicate with background processes reading the message. If there is no message in the queue. Session will wait until next one will be enqueued.

DBMS_OUTPUT.put_line ( 'Message: '

|| MESSAGE.text_value

|| ' ... '

|| to_char(MESSAGE.date_value, 'YYYY.MM.DD HH24:MI:SS')

);

COMMIT;

END;

/

• Use undocumented in 9i/10G statement (officially supported starting from 11g):SELECT ... FROM "MEETUP_QTAB" FOR UPDATE SKIP LOCKED

• Let check the queue tableselect * from AQ$meetup_QTAB;

Dequeue process main options

• dequeue_options.WAIT := 5;

– Default is to wait infinitely

– 0 wouldn’t wait (used sometime, dequeue/dbms_lock/dequeue)

• dequeue_options.DEQUEUE_MODE := DBMS_AQ.REMOVE/BROWSE

– Default is REMOVE

– BROWSE is read/only check (good for monitoring)

• dequeue_options.NAVIGATION := DBMS_AQ.FIRST_MESSAGE/NEXT_MESSAGE

– Default is NEXT_MESSAGE – if dequeue call fail move to next message

– FIRST_MESSAGE – always try to dequeue first message until success or message moved to exceptions queue

Queues’ main options

select * from DBA_QUEUES WHERE QUEUE_TABLE='MEETUP_QTAB';– QUEUE_TABLE – one queue table can contain multiple same type based

queues– MAX_RETRIES – how many times to try to dequeue before fail– RETRY_DELAY – in seconds min retry interval. Message is not allowed to

dequeue before that time– !!! RETENTION !!! – time how long to keep messages in the queue after dequeue

• All parameters can be set at a queue definition timePROCEDURE CREATE_QUEUEArgument Name Type In/Out Default

------------------------------ ----------------------- ------ -------

QUEUE_NAME VARCHAR2 IN

QUEUE_TABLE VARCHAR2 IN

QUEUE_TYPE BINARY_INTEGER IN DEFAULT

MAX_RETRIES NUMBER IN DEFAULT

RETRY_DELAY NUMBER IN DEFAULTRETENTION_TIME NUMBER IN DEFAULTDEPENDENCY_TRACKING BOOLEAN IN DEFAULT

COMMENT VARCHAR2 IN DEFAULT

AUTO_COMMIT BOOLEAN IN DEFAULT

Retention time

• Some times it is only way to keep queues history for troubleshooting purporses. If the RETENTION is set oracle keep messages in the queue for the time specified by RETENTION in secounds.

• By default (RETENTION = 0) a message is removed immediately after succesful dequeue call

• RETENTION != 0 gives us access to important performance data• RETENTION != 0 gives us access to important performance data– deq_time - enq_time = Time message has been in the queue waiting to

be consumed

• Setting to some higher value then 0 have impact on the space utilized by queue table and some SQL-s performance– If keep retention interval reasonably small then impact is minimal

exec dbms_aqadm.alter_queue ( queue_name => 'meetup_msgqueue01' , retention_time => 3600 );

Message Properties

– message_properties.expiration := 60;• If the message is not dequeued during time interval set then it is

transferred into an exception queue• MSG_STATE set to EXPIRED (3); ORIGINAL_QUEUE_NAME /

EXPIRATION_REASON (TIME_EXPIRATION)

– message_properties.delay := 60;– message_properties.delay := 60;• status is set to WAITING straight after enqueue• A Background Time-Manager Daemon wakes up periodically, scans an

internal index (AQ$_meetup_QTAB_T / time_manager_info) for all WAIT state messages, and marks messages as READY if their delay time has passed

• DEQ_TIME – nvl(DELAY, ENQ_TIME) = processing time

Queue table main properties

SELECT * FROM DBA_QUEUE_TABLES WHERE QUEUE_TABLE='MEETUP_QTAB';

– TYPE (OBJECT/RAW)

– SORT_ORDER (PRIORITY, ENQUEUE_TIME/ENQUEUE_TIME)

• Depending on if you would like to separate massages by priority of us FIFO only

• Depending on if you would like to separate massages by priority of us FIFO only

– RECIPIENTS (SINGLE/MULTIPLE)

• Have a significant impact on how messages are dequeued and structures used to handle is (see next few slides)

CONSUMERS – (SINGLE/MULTIPLE)

• To create Multiple Consumers table we run the following SQLBEGIN

DBMS_AQADM.create_queue_table (queue_table => 'meetup_qtab02',

queue_payload_type => 'meetup_msgtype',

multiple_consumers => TRUE

);

END;

/

BEGIN

DBMS_AQADM.create_queue (queue_name => 'meetup_msgqueue02',

queue_table => 'meetup_qtab02'

);

END;

/

BEGIN

DBMS_AQADM.start_queue (queue_name => 'meetup_msgqueue02');

END;

/

• To change consumer type table has to be recreated

CONSUMERS – (SINGLE/MULTIPLE)

• There are more objects created to support MULTIPLE CONSUMERS AQcolumn object_name format a30

column object_type format a30

select object_name, object_type from dba_objects where object_name like '%meetup%02%' order by 2,1;

OBJECT_NAME OBJECT_TYPE

------------------------------ --------------------

AQ$_MEETUP_QTAB02_V EVALUATION CONTEXT

AQ$_MEETUP_QTAB02_E QUEUE

MEETUP_MSGQUEUE02 QUEUE

AQ$_MEETUP_QTAB02_N SEQUENCE

(1) AQ$_MEETUP_QTAB02_H TABLE –- (IOT) STORES DEQ_TIME FOR EACH CONSUMER.

(2) AQ$_MEETUP_QTAB02_I TABLE -- (IOT) TRACK DOWN WHICH CONSUMERS DIDN’T CONSUME A MESSAGE.

AQ$_MEETUP_QTAB02_NR TABLE –- (IOT)

(3) AQ$_MEETUP_QTAB02_S TABLE -- STORES INFORMATION ON ALL REGISTERED CONSUMERS

AQ$_MEETUP_QTAB02_T TABLE –- (IOT)

MEETUP_QTAB02 TABLE

AQ$MEETUP_QTAB02 VIEW

AQ$MEETUP_QTAB02_R VIEW

AQ$MEETUP_QTAB02_S VIEW

14 rows selected. (queue table type)

CONSUMERS – (SINGLE/MULTIPLE)

• Create subscriber listDECLARE

subscriber SYS.aq$_agent := SYS.aq$_agent ('RED', NULL, NULL);BEGIN

DBMS_AQADM.add_subscriber (queue_name => 'meetup_msgqueue02',subscriber => subscriber);

END;/

DECLARE

subscriber SYS.aq$_agent := SYS.aq$_agent ('GREEN', NULL, NULL);

BEGINDBMS_AQADM.add_subscriber (queue_name => 'meetup_msgqueue02',

subscriber => subscriber);

END;/DECLARE

subscriber SYS.aq$_agent := SYS.aq$_agent (‘BLUE', NULL, NULL);

BEGINDBMS_AQADM.add_subscriber (queue_name => 'meetup_msgqueue02',

subscriber => subscriber);

END;/

• The whole multi consumers idea is to allow dequeue the same message by several customers– Message is not removed from the queue until all consumers dequeue it

CONSUMERS – (SINGLE/MULTIPLE)

• To dequeue a message from multiple consumer queue use the following SQLDECLARE

dequeue_options DBMS_AQ.dequeue_options_t;message_properties DBMS_AQ.message_properties_t;message_handle RAW (16);MESSAGE meetup_msgtype;

BEGINdequeue_options.consumer_name := 'RED'; -- GREEN/BLUEDBMS_AQ.dequeue (queue_name => 'meetup_msgqueue02',

dequeue_options => dequeue_options,message_properties => message_properties,payload => MESSAGE,payload => MESSAGE,msgid => message_handle);

DBMS_OUTPUT.put_line ( 'Message: '|| MESSAGE.text_value|| ' ... '|| to_char(MESSAGE.date_value, 'YYYY.MM.DD HH24:MI:SS'));

COMMIT;END;/

• To support multiple consumers additional data structures are necessary

AQ Error handling (RETRY_COUNT)

RETRY_COUNT

1. if DBMS_AQ.dequeue call rolled back RETRY_COUNT is increased by one

2. if RETRY_COUNT reach MAX_RETRIES value (5) it moved to exception queue

3. optionally RETRY_DELAY could be set on queue level

EXPIRATION TIME

1. If EXPIRATION TIME was set during enqueue call a message moved to exception queue if it isn’t dequeued within timeframe set

Exception queues

• Exception queue is message level parameter (message_properties.exception_queue:=‘<qname>';)

• If exception queue is not explicitly set then default exception queue AQ$_<qtab>_E is used

• You specifically set a queue as exception queue defining it(queue_type=>DBMS_AQADM.EXCEPTION_QUEUE)

• It can’t have max_retries,retry_delay,retention_time paremeters

• You can’t explicitly enqueue in that queue

AQ Exceptions handling

• In order to start dequeueing from exception queue you have to start it (it isn’t started by default)

begin

DBMS_AQADM.start_queue(queue_name=>'AQ$_meetup_QTAB01_E',enqueue=>FALSE, dequeue=>TRUE);

end;

/

DECLARE

dequeue_options DBMS_AQ.dequeue_options_t;

message_properties DBMS_AQ.message_properties_t;message_properties DBMS_AQ.message_properties_t;

message_handle RAW (16);

MESSAGE meetup_msgtype;

BEGIN

DBMS_AQ.dequeue (queue_name => 'AQ$_meetup_QTAB01_E', dequeue_options => dequeue_options,

message_properties => message_properties, payload => MESSAGE, msgid => message_handle);

DBMS_OUTPUT.put_line ( 'Message: '

|| MESSAGE.text_value

|| ' ... '

|| to_char(MESSAGE.date_value, 'YYYY.MM.DD HH24:MI:SS')

);

COMMIT;

END;

/

Drop & Disable syntaxes

• DBMS_AQADM.stop_queue (queue_name => '<qname>');

• DBMS_AQADM.drop_queue (queue_name => '<qname>');

– You can’t drop active queueORA-24011: cannot drop QUEUE, APPS.meetup_MSGQUEUE03 should be stopped first

• DBMS_AQADM.drop_queue_table (queue_table => 'qtab');

– Can’t run with any active queue (unless force is specified)

– If force => true all queues forced to be closed and tables is dropped

BEGIN

DBMS_AQADM.DROP_QUEUE_TABLE(QUEUE_TABLE=>'MEETUP_QTAB',FORCE=>TRUE);

END;

/

Drop & Disable syntaxes

• To remove a subscriberDECLARE

subscriber SYS.aq$_agent := SYS.aq$_agent ('RED', NULL, NULL);

BEGIN

DBMS_AQADM.remove_subscriber (queue_name => 'meetup_msgqueue03',

subscriber => subscribersubscriber => subscriber

);

END;

/

– !!! Performance !!! Be careful, as this operation might take a while as all corresponding records from AQ$_<qtab>_I and queue itself will need to be removed.

– Typical problems then somebody creates unused subscriber. Messages in <qtab> and AQ$_<qtab>_I stack. Tables growing very big.

– You might check if there any old undequeued messages in AQ$_<qtab>_I table corresponding to any particular subscriber

AQ Maintenance tasks

• Control size of the queue tablesSELECT a.owner,

a.queue_table,

TRUNC (b.BYTES / 1024 / 1024) mb

FROM dba_queue_tables a, dba_segments b

WHERE a.owner = b.owner AND a.queue_table = b.segment_name

ORDER BY 3 DESC;

OWNER TABLE_NAME SIZE_MB

XXIO XXIO_I00_O_JMS_Q_TABLE 3013

APPLSYS WF_NOTIFICATION_OUT 304

APPLSYS WF_DEFERRED 96

APPLSYS WF_DEFERRED_TABLE_M 25

...

• Keep AQ tables as small as reasonable Subject: How to Move Queue Tables without using the Export or Datapump Utilities

Doc ID: 304522.1

• Control if all subscribers dequeue messages from multi-consumers tables

Development using AQ

Warwick SandsSenior Developer at DHL Logistics

Inbound Payload

• TYPE inbound_payload_TYPE IS

RECORD

(guid Varchar2(10), -- original global iddocType Varchar2(10),

storerKey Varchar2(30), -- customer identifierco Varchar2(3),

whse Varchar2(3),

div Varchar2(3),div Varchar2(3),

subCo Varchar2(5),

action Varchar2(15),

pending_guid Varchar2(10),

-- cannot proceed until this guid has been processedenq_datim timestamp) ; -- enqueue time

Oracle EBS to local

EBS Payload

• TYPE ods_payload_TYPE IS

RECORD

(docType Varchar2(30),

payload blob) ;

Translation and ping

Local Payload

• TYPE mercator_payload_TYPE IS

RECORD

(guid Varchar2(10),

guid_status Varchar2(10),

docType Varchar2(30),

logger_status Varchar2(15),

enq_date timestamp,

document_txt CLOB) ;document_txt CLOB) ;

Local to Mercator

Admin package has

• Queue create

• Queue_start

• Queue_stop

• Queue_drop• Queue_drop

• Queue_name

• Queue_control

Queue Names

• Don’t try to remember Queue Names• FUNCTION queue_name(i_direction IN Varchar2,

i_Schema IN Boolean := TRUE)

RETURN Varchar2

IS

BEGIN

RETURN CASE WHEN i_Schema THENRETURN CASE WHEN i_Schema THEN

k_schemaName || '.' END

|| CASE LOWER(i_direction)

WHEN 'in' THEN k_from_q

WHEN 'out' THEN k_to_q

WHEN 'excpn' THEN

k_excpQprefix || k_qTab

|| k_excpQsuffix

ELSE k_qTab

END ;

END queue_name ;

Operational package has

• Enqueue

• Dequeue

• Excpn_clean

• Inform• Inform

• Dequeue_start

• Dequeue_stop

Inform

• Mercator AQ adaptor is poorly written

– Throughput drops as a function of time

– Triggering method changed to file based.based.

– If items have been enqueued then file is touched

– This is performed at the end of the enqueue process which runs every minute

Inform fragment

• -- ODS

v_qNm := UPPER(brkr_ods_admin.queue_name('in', FALSE));

SELECT COUNT(*)

INTO v_cnt

FROM brkr_ods_qtab

WHERE q_name = v_qNm

AND STATE = 0 ;

IF v_cnt > 0 THEN

fHandle := com_file.open('brokerAQ:brkr_ods.txt‘,'A‘);

com_file.writeLine

(fHandle, TO_CHAR(sysdate, 'dd-mon-yyyy hh24:mi:ss')

|| '; Count=' || LTRIM(TO_CHAR(v_cnt))

|| com_constant.k_eol || i_msg ) ;

com_file.close(fHandle) ;

END IF ;

AQ in EBS

By Jurijs VelikanovsLead Oracle Apps DBA

Rolta TUSC Asia Pacific Ltd

How important AQ for EBS?

09:39:44 SYSTEM:OAPROD> SELECT owner, COUNT (*)

09:39:45 SYSTEM:OAPROD> FROM dba_queues09:39:45 SYSTEM:OAPROD> GROUP BY CUBE (owner)

09:39:45 SYSTEM:OAPROD> ORDER BY COUNT (*);

OWNER COUNT(*)

------------------------------ ----------

ODM 2

IBU 2

PO 2

AMV 2

OKC 3

CCT 4CCT 4

CS 4

AR 4

IEO 4

IEM 4

ASO 5

JTF 6

SYSTEM 8

XLA 9

XNP 12

SYS 15

XDP 15

APPS 20

APPLSYS 56

177

AQ in Apps DBA area (ATG)

NAME QUEUE_TABLE MAX_RETRIES RETRY_DELAY ENQUEUE DEQUEUE RETENTION USER_COMMENT

ECX_INBOUND ECX_INQUEUE 5 120 YES YES 0 Engine Queue for Transaction

ECX_IN_OAG_Q ECX_IN_OAG_Q_TABLE 5 120 YES YES 0 Engine Queue for Transaction

ECX_OUTBOUND ECX_OUTQUEUE 5 120 YES YES 0 Engine Queue for Transaction

FND_CP_GSM_IPC_AQ FND_CP_GSM_IPC_AQTBL 5 0 YES YES 0

FND_CP_GSM_OPP_AQ FND_CP_GSM_OPP_AQTBL 5 0 YES YES 0

FND_CP_TM_AQ FND_CP_TM_AQTBL 5 0 YES YES 0

FND_CP_TM_RET_AQ FND_CP_TM_RET_AQTBL 5 0 YES YES 0

WF_CONTROL WF_CONTROL 5 0 YES YES 0 Workflow JMS Topics

WF_DEFERRED WF_DEFERRED 5 3600 YES YES 86400 compile

WF_DEFERRED_QUEUE_M WF_DEFERRED_TABLE_M 5 0 YES YES 0 Workflow Deferred Queue

WF_ERROR WF_ERROR 5 3600 YES YES 0 Workflow event system default queue

WF_IN WF_IN 5 0 YES YES 604800 Workflow event system default queue

WF_INBOUND_QUEUE WF_INBOUND_TABLE 5 0 YES YES 0 Workflow Inbound QueueWF_INBOUND_QUEUE WF_INBOUND_TABLE 5 0 YES YES 0 Workflow Inbound Queue

WF_JAVA_DEFERRED WF_JAVA_DEFERRED 5 3600 YES YES 86400 Workflow Java Deferred Queue

WF_JAVA_ERROR WF_JAVA_ERROR 5 3600 YES YES 0 Workflow Java Error Queue

WF_JMS_IN WF_JMS_IN 5 0 YES YES 0 Workflow JMS Topics

WF_JMS_JMS_OUT WF_JMS_JMS_OUT 5 0 YES YES 0 JMS deposit JMS Topics

WF_JMS_OUT WF_JMS_OUT 5 0 YES YES 0 Workflow JMS Topics

WF_NOTIFICATION_IN WF_NOTIFICATION_IN 5 3600 YES YES 86400 Workflow JMS Topics

WF_NOTIFICATION_OUT WF_NOTIFICATION_OUT 5 3600 YES YES 86400 Workflow JMS Topics

WF_OUT WF_OUT 5 0 YES YES 604800 Workflow event system default queue

WF_OUTBOUND_QUEUE WF_OUTBOUND_TABLE 5 0 YES YES 0 Workflow Outbound Queue

WF_REPLAY_IN WF_REPLAY_IN 5 0 YES YES 604800 Workflow event system default queue

WF_REPLAY_OUT WF_REPLAY_OUT 5 0 YES YES 604800 Workflow event system default queue

WF_SMTP_O_1_QUEUE WF_SMTP_O_1_TABLE 0 0 YES YES 0

[WORKFLOW_USER_QUEUE] -

APPLSYS.WF_SMTP_O_1_QUEUE

WF_WS_JMS_IN WF_WS_JMS_IN 5 0 YES YES 0 WebServices Inbound JMS Topics

WF_WS_JMS_OUT WF_WS_JMS_OUT 5 0 YES YES 0 WebServices Outbound JMS Topics

WF_WS_SAMPLE WF_WS_SAMPLE 5 0 YES YES 0 WebServices Sample Topics

AQ usage in Apps DBA area

• Execution of DIFFERED activities

– Asynchronous workflow processing

• Order processing

• Order approvals

– Event processing (Business Event System)

• Data transfer (EBS internally & externally)

• Web servers cache invalidation

• Privileges management

– EBS Interfaces

• Email-s sending (WF Notification Mailer)

Deferred Events Processing

• Deferred Events depending on what type of subscriptions exist on those are enqueued in two different queues

– Events with PL/SQL actions only are placed in

• WF_DEFERRED queue

• Processed by - Workflow Deferred Agent• Processed by - Workflow Deferred Agent

– Events with Java and PL/SQL activities are placed in

• WF_JAVA_DEFERRED queue

• Processed by - Workflow Java Deferred Agent

– Those queues are different from WF Processes

• WF_DEFERRE_TABLE_M

• Workflow Background Engine

WF Notification Mailer example

WF_DEFERRED

2. Deferred

WF_NOTIFICATION_OUT

3. Notification 1.2 Differs processingGenerates an event

2.1 Reads event and starts subscription processing

2.3 Generates data for outbound interface (email text)

3.1 Reads the message

WF_NOTIFICATION_ATTRIBUTES

1. EBS user sends an email

2. Deferred Agent Listener

3. Notification Mailer

WF_NOTIFICATIONS

WF_NOTIFICATION_ATTRIBUTESWF_NOTIFICATION_ATTRIBUTES

1.1 Provides an application data

Generates an event

2.2 Reads the application data

3.2 Sends an email via SMTP

3.3 Updates a status

Troubleshooting Hints ...

• Set RETENTION to reasonable value (1-24h)exec dbms_aqadm.alter_queue ( queue_name => '<queue

name>' , retention_time => 3600 );

• How to get a full trace for a particular message?

WF_DEFERRED WF_NOTIFICATION_OUT

WF_NOTIFICATIONS

A full trace for AQ message?

SELECT

n.notification_id

C

,de.def_enq_time

C

,ou.out_enq_time

C

FROM applsys.wf_notifications n

, (SELECT d.enq_time def_enq_time

,d.deq_time def_deq_time

,TO_NUMBER (SELECT VALUE

Microsoft Office

Excel Worksheet

FROM TABLE (d.user_data.parameter_list)

WHERE NAME = 'NOTIFICATION_ID')

d_notification_id

,d.user_data def_user_data

,msg_state def_state

,d.*

FROM applsys.aq$wf_deferred d

WHERE d.corr_id IS NOT NULL

AND d.corr_id = 'APPS:oracle.apps.wf.notification.send') de

WHERE 1 = 1

AND n.notification_id=4687832

AND n.notification_id = de.d_notification_id(+)

AND n.notification_id = ou.o_notification_id(+)

ORDER BY n.begin_date DESC

/

Troubleshooting Hints ...

• Check on regular basis if there are a lot of un-dequeuedmessages in AQ tables

– ( DEQ_TIME is null and TIME_MANAGER_INFO < SYSDATE-5/(24*60) )

• Build a monitoring to alert if there is a stack message in queuesqueues

– dequeue_options.DEQUEUE_MODE := DBMS_AQ.BROWSE

• Check and address any messages in exception queues– select count(*) from AQ$_WF_DEFERRED_E;

www.sydneyoracle.com.au

I. Oracle E-Business Suite TechnologySteven Chan, Director, Applications Technology Integrationhttp://blogs.oracle.com/schan/

II. Mailing list dedicated to Oracle Apps DBA questions

Warwick Sands &Yury Velikanov

http://www.freelists.org/list/ora-apps-dbaora-apps-dba-request@freelists.org with 'subscribe' in the Subject field