Job Control and and Parallel Processing-En

Job control and Parallel processing

Thomas BollmeierIBU Financial Services, NAD, Tools & Technology

SAP AG 2001, Title of Presentation, Speaker Name 2

Job control : Requirements

Control of the start of processes depending on the technical and business processing status of other processes

Example : Mass processing of account extract only after successful processing of account balancing.

Synchronisations of parallel sub processes That is follow up processes should start only after end of all parallel

sub processes

=> Use of a job control


Job control : Performance features

Processes are specialisations of an abstract process class • Advantage : can be expanded easily • Implemented process classes:

• Report processes (Attributes: Report name, Variant, Mass run-ID); use in the daily end processing

• Processes in the parallel processing

Definition of flow plans in the Customizing• Predecessor-Successor –Relationships between Report processes

Flexible generation of flows through customer reports• BAPI JobControlFS.createNetwork

Start of Reportprocesses through external Job control possible• BAPI JobControlFS.startProcess

• Possibility of the status query through BAPI JobControlFS.getDetail planned


Job control : Design

Assigned DB-Tables:

Process flow pattern : TBANK_RFL_TEMPL TBANK_RFL_TREP TBANK_RFL_TREL

Process flow: BANK_PROCFL_FLOW BANK_PROCFL_REL

Process: BANK_PROCFL_PROC BANK_PROCFL_LOG

Report process: BANK_REPORTS BANK_REP_PARAMS


Jobsteuerung: Prozessablauf I

BANK_API_PROCFL_START

BANK_OBJ_PROCFL_GET_PROC(procid)

[Start check negative]

BANK_OBJ_PROCFL_LOCK_PROC

[Lock can not be set ]

BANK_OBJ_PROCFL_WAIT_FOR_PRED

BANK_OBJ_PROCFL_REL_PROC

[at least one predecessor aborted]

BANK_OBJ_PROCFL_GET_MODE

BANK_OBJ_PROCFL_START_ASYNC BANK_OBJ_PROCFL_START_SYNC

[asynchronous processing][else]

Start Checks

Following checks are done :

1.) Check for restart . If thestart counter is(START_COUNT) > 0 , thenthe codeXREST_ALLOWED must beset for the process.

2.) Check whether externalstart is expected. (code.:XSTART_EXTERN).

3.) Status check. The statusmust have the value 'not yetstarted' .


Jobsteuerung: Prozessablauf II

:cl_procflow :R/3-Job control

BANK_OBJ_PROCFL_START_ASYNC

BANK_OBJ_PROCFL_GET_JOBDATA

JOB_OPEN( startjob )

JOB_OPEN( end job )

SUBMIT end report

JOB_CLOSE( end job )

SUBMIT startreport

JOB_CLOSE( startjob )

endjob is planned assuccessor of startjob . Itruns even during aborting ofstartjob and ensures such adefined process end even incase of aborting.

startreport

:cl_procflow

BANK_OBJ_PROCFL_START_SYNC

end report

:cl_procflow

BANK_OBJ_PROCFL_END

The ReportRBANK_PROC_START iscalled. As Parameter P_IDthe ID of the process to bestarted is transferred.

Der ReportRBANK_PROC_END startsthe flow at the process endIts Parameter P_ID duringcalling is supplied with theID of the process to beclosed.


Jobsteuerung: Prozessablauf III

:cl_procflow :cl_ooman

Method name

BANK_OBJ_PROCFL_START_SYNC

BANK_DB_PROCFL_SET_STATUS (aktiv)

BANK_API_OOMAN_GET_METHOD

Reading of the method ofthe process class , whichimplements the methodAT_START of the interfaceif_proc

Process:Process class

Process::AT_START

:DB-adminis-tration

COMMIT

BANK_OBJ_PROCFL_GET_CHILDS


BANK_API_PROCFL_START

* [for all child processes]

BANK_OBJ_PROCFL_END

BANK_OBJ_PROCFL_GET_MODE

[in case synchronous mode]

BANK_OBJ_PROCFL_EXPORT_ID


Jobsteuerung: Prozessablauf IV

BANK_OBJ_PROCFL_END

BANK_DB_PROCFL_GET_STATUS

closed, aborted]

BANK_OBJ_PROCFL_LOCK_PRC

[Locking not possible ]

BANK_OBJ_PROCFL_WAIT_FOR_CHILDS

cl_ooman::BANK_API_OOMAN_GET_METHOD

Get the method, whichimplements the interfacemethodAT_END_OF_PROCESS ofthe interface if_proc .

<Prozeßklasse>:AT_END_OF_PROCESS:

BANK_DB_PROCFL_SET_STATUS


BANK_OBJ_PROCFL_GET_FATHER

[No father process available ]

[else]

status =closed /aborted

BANK_OBJ_PROCFL_START_SUCC

DB::COMMIT_WORK


Job control : Customizing

Definition of a flow pattern from Report processes in IMG Create and maintenance of variants per Report of the pattern Dynamic Variables (e.g. Date values ) can be defined as selection

variables (in the table TVARV) Stipulation of predecessor- successor–relationships If necessary definition of own reports for inclusion in Report-Flow

pattern Stipulation of the behaviour during restarting


Job control : Start and control of a flow

Start through BAPI JobControlFS.createNetwork bzw. JobControlFS.startProcess Report RBANK_JC_START

Transaction BANK_JC_START Planning as background job possible

Monitoring and control of the procedural flow through report RBANK_JC_OVERVIEW (Transaction: BANK_JC_OVRVW)

Network plan diagram Restarting Stopping a flow („Emergency stop “)


Parallel processing : Concept

Framework for Applications , which are to be parallelised

Classification of the quantity of objects to be processed in to work packages

Processing of the work packages in to parallel background jobs

Generation and Synchronisation of the background jobs through Financial- Services-Job control


Parallel processing : Performance features

Number of parallel jobs + Distribution to Server(groups) adjustable in the customizing

Standard-Package templates enable reusability of once prepared object classification by many randomly paralleised mass processings

Definition of the packages as Interval or as discrete Object list

BTE and BAPI for definition of packages through customer 0BANK010 ParallelProcessingFS. PackAdmCreate

Start of the parallel processing through BAPI is supported

Transactions for automatic generation of Callback-FuB and for checking the interface consistency

BANK_PP_GEN und BANK_PP_CHECK


Parallel processing : Data model I

- Mass run-ID- Application type- current StartNo- current realStartNo.

Mass run

- Mass run-ID- Simulation code- No. current step- No. last step- No. of repetitions- No. maximal step- Status

Start Data1 1..2

- work reserve-ID- number of objects in status <xx>

work reserve

1..*

1

{abstract}

Application

Account closing(example)

- PVerw-ID- Kz. Standard- Def.Type of packs

Pack administrator

Job control

+ at_start+ at_end_of_process

- Job-ID- Jobstatus

{abstract}Process

General process

generated

1..*

1

parallelsub process

1

1..*

- PVerw-ID- PackNo.- Interval limits

Pack template

1 0..*

- work reserve-ID- PackNo.- Code.: requested- number of open Obj.- number of returned Obj.

Work package header

1..*1

- PVerw-ID- PacktNr- Object-ID

Pack template-objects

- work reserve-ID- PacktNo- Object-ID- Status

work package

1

1..*

1 0..*

0..1 0..*

10..*


Parallel processing : Data model II

Assigned DB-Tables Mass course (run)

BKKMPARUNHD, BKKMPARUN, BANK_PP_STEPDATA

Package administrator BANK_PACKMAN_HD

Package template (pattern) BANK_PACKMAN

Package template objects BANK_PACKMAN_OBJ

Work stock(reserve) BANK_WORKL_HD, BANK_WORKL_CNT

Work reserve header BANK_WORKL_PACK

Work package (objects) BANK_WORKL_OBJ


Parallel processing : Flow I

:Mass run-API

BANK_MAP_PP_START :Mass run-

OBJ

BANK_OBJ_PP_PREPARE_START

:work reserve-API

BANK_API_WORKL_CONSTRUCTOR

:PVerw-API

BANK_API_PACKMAN_CONSTRUCTOR

[if I_PACKMANID_EXT initial]

BANK_OBJ_PP_EXECUTE_STEP

Call in UPDATE TASK, ifI_X_TRIGGER... = 'X'

BANK_OBJ_PP_START_CONTAINER

In process flow within aupdate(entry) work process(WP-Types 2 and 6)BANK_OBJ_PP_START_CONTAINER is called with theaddition STARTING NEWTASK

:Application

Method 0100: Start of a mass run

Method 0205: generate package templates

Method 0120: Set Parameter

Method 0160:Checks prior to start of a mass run


Parallel processing : Flow II

Mass run ::BANK_OBJ_START_CONTAINER

Application ::Method 0160

Mass run::BANK_OBJ_PP_JOBMass run::BANK_OBJ_PP_JOB Mass run::BANK_OBJ_PP_JOB

Mass run ::BANK_OBJ_PP_DET_STAT_AT_EOSTEP

Mass run::BANK_OBJ_PP_CONT_WITH_STEP

[Status Start next step , repeat current step]

Application ::Method 0300

[Status closed, aborted, in asynchronous processing]

After Synchronisation withthe help of Object statuscounter (TableBANK_WORKL_CNT) andthe job status a mass runstatus is determined.

If the percentage of theobjects in the status 'reset'exceeds the threshold valuestored in the Customizing(TBCAMPPAPPL) , thecurrent step is repeated.

If the maximal step numberis not yet reached and therun status not 'aborted' or'in asynchronousprocessing', the next step isstarted.

Mass run ::BANK_OBJ_PP_PREPARE_START


Parallel Processing: Flow III

:Mass run-OBJ :work reserve

BANK_OBJ_PP_JOB

BANK_API_WORKL_GET_PACKAGE

:Application

Method 1000: initialise package

Method 1100: select data

BANK_API_WORKL_CHANGE_STATUS

Method 1260: check data

Method 1270

BANK_API_WORKL_CHANGE_STATUSMethod 1270

Method 1300: process data

BANK_API_WORKL_CHANGE_STATUS

:PVerw-API

BANK_API_PACKMAN_GET_LIMITS

BANK_API_WORKL_GET_OBJECTS

BANK_API_PACKMAN_GET_OBJECTS

BANK_DB_WORKL_GET_PACKOBJ

Method 1200: select data for list

First processing of packagewithout preselection

Follow up processing of the package orfirst processing with preselection

[1.Pack processing]

[Follow up processing]

Method 1400: Start of processing in a parallel Job

BANK_OBJ_PP_PACKAGE

Method 1410: End of the processing in a parallel Job

* [ till no unprocessed packageis available ]

End of the packageprocessing loop


Parallel processing : Connecting an application

Maintenance of the System-Customizing (Transaction BANK_CUS_PP)

Definition of a type of application Implementation and assignment of the Callback-Functions

Calling the functional module BANK_MAP_PP_START or BANK_MAP_PP_RESTART (during restart )


Other Literature

Design-Document on Job control in the InfoCenter Title „Job control “ \\pwdf0626\infocenter\protected\2001\07\27\c42375be8f46ba44a509b675a7bdb

151.doc

Developer docu for parallel processing in the InfoCenter Titel „Parallelisation 2.1“ \\pwdf0626\infocenter\protected\2001\09\06\2694861e939edf4b8c12f8fa44c1ba

a7.doc

H. Knallinger, D. E. Knuth, J. v. Neumann(†), A. Turing(†) (Eds.): „The Art of Parallel Processing“, 347th. Ed., Berlin, Heidelberg, New York, 1999, ISBN: 0-123-4567-8

Documents

Job Control and and Parallel Processing-En