COAA Major Projects Benchmarking Associates Training · 2013-10-14 · Procurement Plan for Major...

COAA Major Projects Benchmarking Associates Training

University of CalgarySchulich School of Engineering

Dr. Elias IkpeJatinder Kumar

Agenda• COAA Benchmarking History and Staff

• COAA Performance Assessment System (PAS)

• Code of Conduct and Confidentiality Policy

• Assessing Project Performance

• Industry Partners’ Participation in Benchmarking

• Industrial Projects Data and Metrics

• Pipeline Projects Data and Metrics

• Well Site Projects and Well Pad Projects Data & Metrics

What Is Project Benchmarking And Why Should I Care?

“Benchmarking is the process of gathering information aboutother companies in your industry to compare your performanceagainst and to use to set goals.” (F. John Reh)

COAA Benchmarking History

Phase I (2006 – 2009)Task 1: Metric and report definition.

Task 2: Program data collection instrument and report.

Task 3: Conduct onsite training and commence data collection.

Task 4: Collect and validate data, perform analysis, develop an Alberta data set for benchmarking, and prepare project and company aggregate reports.

Task 5: Prepare Alberta Report documenting performance and quantifying the impacts of local factors on productivity and performance metrics.

Phase II (2010 – 2014)Task 1: Development of new metrics (Pipelines, Well Sites, SAGD).

Task 2: System development (Performance Assessment System)

Task 3: Alberta report #2.

COAA Benchmarking Committee

CII / U. of Calgary Benchmarking & Metrics Staff Questionnaire development & management

BA training

Database management, administration, & security

Implementation assistance

Analysis & reporting

Coordination

Staff and Functions

Dr. Elias IkpeAccount Manager (University of Calgary)

(587) 228-0311eoikpe@ucalgary.ca

Jatinder KumarGraduate Student (University of Calgary)

(403) 462-4433jkumar@ucalgary.ca

Dr. Stephen MulvaAssociate Director (CII Benchmarking & Metrics)

(512) 232-3013smulva@cii.utexas.edu

Getting Help…

COAA Benchmarking Performance Assessment System (PAS)

COAA Performance Assessment System

• COAA PAL Website: https://coaa.construction-institute.org/NextGen/

Useful Information

• For security and performance reasons, a logged in user that has not been active for a period of 1 hour is timed out.

• Data entered into a browser page is not recorded until the “Save” button is clicked.

Session Timeout

• Internet Explorer 5.0 or later• An Internet connection, 56K or faster• Java and JavaScript enabled• Cookies on• Adobe Acrobat Reader (PDF file)• Flash MX Player 10.2

Browser and Connection

Requirements

• Company data are considered confidential.• Data can be used to support CII/COAA benchmarking, research, and related academic activities only if the confidentiality of companies submitting the data is protected.• Access to data is limited to CII and U. of Calgary staff and authorized researchers only.• All persons with access to COAA data must sign confidentiality agreements and abide by COAA confidentiality policies.• When data are provided in support of research activities, all confidential identifiers will be removed and only essential subsets of data will be provided.• All data published and/or presented must reflect the aggregate of at least 10 projects from 3 separate companies.• Reports and data files containing only individual project or company data are considered confidential will not be published or provided to researchers.• In cases where a disproportionate amount of data are provided by a single company, CII/COAA will suppress publication of results until the data set is sufficiently large to mitigate confidentiality or bias concerns.

COAA Benchmarking Code of Conduct and Confidentiality Policy

Note: each quartile contains ¼ of the Number of Projects

• Cost• Schedule• Safety• Changes• Rework

Current Metrics

75th Percentile

50th Percentile

25th Percentile

Mean

Median

Category(n = 20 )

Displayof

MetricRespondent:

Cost Category :Project Type :

Location:Industry Group:Project Nature:

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

Number of projects

BetterBetter

• EngineeringProductivity

• ConstructionProductivity

Assessing Project Performance

Budget Factor - Contractors

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Cost Growth - Owners

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

Median (-3%)

“Best in Class” (-19%)

Median (0.98)

“Best in Class” (0.83)

OWNER CONTRACTOR

COST GROWTH BUDGET FACTOR

Explaining Statistics – Cost Example

Explaining Statistics

Mean = arithmetic average

Median = mid-point of allprojects (is equivalent to the Second Quartile Q2)

Inter Quartile Range (IQR)refers to the range betweenthe First Quartile (Q1) andthe Third Quartile (Q3)

Outlier Projects are notIncluded in statistics

What Do Different Quartiles Mean?

Good

Poor4Q

3Q2Q

1Q

11% Improvement (4th to 3rd Quartile)

• 26% Improvement (4th to 1st Quartile)

Key Report

Benchmarking Information Output

Benchmarking Key Report

Data Miner

Benchmarking Information Output

Data Miner Features(Metrics – Parameters – Filters – Variables)

CompanyBenchmarking

Associate

Project Manager

Project Manager

AccountManager

(CII)

AssociateDirector (CII)

SystemsAnalyst (CII)

ResearchEngineer (CII)

Industry Partners’ Participation in Benchmarking

CompanyBenchmarking

ManagerCOAA

AccountManager(UofC)

Major Responsibilities

Project Manager

Benchmarking Associate

Benchmarking Manager

Board of Advisors BOA

BMMAN

BMASSOC

BMPM BMPM

BMASSOC

BMPM

• Company level benchmarking people hierarchy

Role of Benchmarking Manager• Benchmarking Manager is the key person of

an organization to manage their benchmarking efforts with COAA/CII

• Set up the benchmarking goal• Set up the benchmarking hierarchy for internal

benchmarking purpose (optional)• Setup the project profiles (optional)• Coordinate the people participating in

benchmarking• Manage the projects/database

Role of Benchmarking Associate• Company’s trained knowledge resource for

COAA/CII Benchmarking

• Facilitates selection of project representatives

• Expedites completion of questionnaire

• Verifies completeness and accuracy of data

• Assists company with the use of COAA/CII Benchmarking products

Role of BenchmarkingProject Manager

• Works with project teams

• Collects and reviews data for selected projects

• Submits projects to a Benchmarking Associate

COAA Benchmarking ProcessProject

ManagersBenchmarking

AssociateCompany

LeadershipCOAA/CII/UofC Staff, Account Mgr/Committee

Benchmarking Manager

Commit to Benchmarking  

Coordination & Attend COAA/CII Training

Make Decision / Commitment to Benchmark as 

Basis for Improvement

Commit to Project  Benchmarking & Improvement

Develop/Improve Metrics, Processes, Policies & 

Procedures

Develop Data Collection & Reporting Tools

Perform Validation Checks, Run Analysis & Generate 

Reports

Provide Input to Research & Implementation

Conduct Training & Feedback Sessions

Validate Questionnaires

ReleaseQuestionnaire to 

COAACII

Initiate Questionnaire During Project Execution

Complete & Submit Questionnaire at Project Close‐out

Review and Act on Interim COAA/CII On‐line Recommendations

Managing the benchmarking efforts with COAA/CII 

Select Projects for Analysis & Preload/Initiate Project in CII Database

Perform Self‐Analysis & Develop Improvement Plan

COAA Benchmarking Framework

Level 1 Level 2 Level 3

Upstream(Oil Exploration/

Production)

Oil Sands SAGD

Cogeneration

Central Plant Processing Facilities

Pad and Gathering

Oil Sands Mining/ Extraction

Oil Sands Mining

Central Plant Processing Facilities

DownstreamOil Sands Upgrading

Naptha Hydrotreater Unit

Hydrogen Plant

Oil Refining Utilities and Offsite

Natural Gas Natural Gas Processing

Pipelines

Well Sites / Well Pads

• Cost

• Schedule

• Safety

• Change

• Rework

Procure Forms

5

5

Mobilize

10

3

Layout Exc 1

15

4

Forms Conc 1

30

3

Set BasePlate1

35

1

Layout Exc 2

20

4

Forms Conc 2

40

3

Set Base Plate 2

50

1

Layout Exc 3

25

4Forms Conc 3

45

3

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

Procure Forms

5

5

Procure Forms

5

5

Mobilize

10

3

Mobilize

10

3

Layout Exc 1

15

4

Layout Exc 1

15

4

Forms Conc 1

30

3

Forms Conc 1

30

3

Set BasePlate1

35

1

Set BasePlate1

35

1

Layout Exc 2

20

4

Layout Exc 2

20

4

Forms Conc 2

40

3

Forms Conc 2

40

3

Set Base Plate 2

50

1

Set Base Plate 2

50

1

Layout Exc 3

25

4

Layout Exc 3

25

4Forms Conc 3

45

3Forms Conc 3

45

3

Set Base Plate 3

55

1

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

• EngineeringProductivity

(Design Hours/IFC Quantities)

• ConstructionProductivity

(Work-Hours/Installed Quantities)

Industrial Projects Data and Metrics

COAA Consensus Benchmarking Definitions

Aligning Phase Definitions

COAA Phase Definitions

Detailed Engineering

Single Project

Concept Adopted &

Formal Project Team

Established

Project Authorization

Contract Award to

Engineering Firm

Release of all Drawings & Specs

Development of Procurement Plan for Major

Equipment

All Major Equipment Delivered to Site

Mechanical Completion

Mechanical Completion

Custody Transfer

Beginning of Continuous Substantial

Construction Activity

Procurement

Construction

Start-up

Front End Planning

Benchmarking Phase Definitions, Activities & Cost Elements

Project Phase Start/Stop Typical Activities & Products

Typical Cost Elements

Front End PlanningTypical Participants:• Owner Personnel• Planning Consultants• Constructability

Consultant• Alliance / Partner

Start: Single Project Concept Adopted, Formal Project Team Established

Stop: Project Authorization

• Options Analysis• Life-cycle Cost Analysis• Project Execution Plan• Appropriation Submittal

Pkg• P&IDs and Site Layout• Project Scoping• Procurement Plan• Arch. Rendering

• Owner Planning Team Personnel Expenses

• Consultant Fees & Expenses • Environmental Permitting

Costs• Project Manager / Construction

Manager Fees• Licensor Costs

Detail EngineeringTypical Participants:• Owner Personnel• Design Contractor• Constructability Expert• Alliance / Partner

Start: Contract Award to Engineering Firm

Stop: Release of All Approved Drawings & Specs for Construction (or last package for fast-track)

• Drawing & Spec Prep• Bill of Material Prep• Procurement Status• Sequence of Operations• Technical Review• Definitive Cost Estimate

• Owner Project Management Personnel

• Designer Fees• Project Manager / Construction

Manager Fees

Benchmarking Phase Definitions, Activities & Cost Elements

Project Phase Start/Stop Typical Activities & Products

Typical Cost Elements

ProcurementTypical Participants:• Owner Personnel• Design Contractor• Alliance / Partner

Start: Development ofProcurement Planfor MajorEquipment

Stop: All MajorEquipmentDelivered to site

• Supplier Qualification• Supplier Inquiries• Bid Analysis• Purchasing• Engineered Equipment• Transportation• Supplier QA/QC

• Owner PM Personnel• Project/Const. Manager Fees• Procurement & Expediting

Personnel • Engineered Equipment• Transportation• Shop QA/QC

ConstructionTypical Participants:• Owner Personnel• Design Contractor

(Inspection)• Construction Contractor

& Subcontractors

Start: Beginning of Continuous Substantial Construction Activity

Stop: Mechanical Completion

• Set Up Trailers• Site Preparation• Procurement of Bulks• Issue Subcontracts• Construction Plan for

Methods/Sequencing• Build Facility & Install

Engineered Equipment• Complete Punchlist• Demobilize Construction

Equipment

• Owner PM Personnel• Project/Const. Manager Fees• Building Permits• Inspection QA/QC• Construction Labour,

Equipment & Supplies• Bulk Materials• Construction Equipment• Contractor Management

Personnel• Warranties

Project Phase Start/Stop Typical Activities & Products

Typical Cost Elements

Start-up/CommissioningTypical Participants:• Owner Personnel• Design Contractor• Construction Contractor• Training Consultant• Equipment Suppliers

Start: MechanicalCompletion

Stop: Custody Transferto User/Operator

(steady stateoperation)

• Testing Systems• Training Operators• Documenting Results• Introduce Feedstocks

& Obtain First Product• Hand-off to User/

Operator• Operating System• Functional Facility• Warranty Work

• Owner PM Personnel• PM/Const. Manager Fees• Consultant Fees & Expenses• Operator Training Expenses• Wasted Feedstocks• Supplier Fees

Benchmarking Phase Definitions, Activities & Cost Elements

Industrial Projects Data and Metrics

COAA Questionnaire

Benchmarking Cost

Questionnaire – Reporting Cost Data

Procurement Phase CostsCost of Major Equipment including process and mechanical

Cost Definitions

Direct & Indirect Construction Cost

•See next slides for the tables

•The page numbers refer to the questionnaire printout

Direct & Indirect LabourTable(Construction Productivity Table)

Cost Performance Metrics– Project Level

Project Cost Growth Total Project Cost – Budget at Authorization*Budget at Authorization*

Delta Cost Growth | Project Cost Growth |

Project Budget Factor Total Project CostBudget at Authorization* + Approved Changes

Delta Budget Factor | 1- Budget Factor |

* Owner Definition; for Contractor Budget at Contract Award

Phase Cost Factor (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up

Phase CostTotal Project Cost

Phase Cost Growth (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up

Phase Cost – Phase Budget at AuthorizationPhase Budget at Authorization

Cost Performance Metrics– Phase Level

Benchmarking Schedule

Questionnaire –Reporting Schedule Data

Project Schedule Growth

Total Project Duration* – Predicted Duration at AuthorizationPredicted Duration at Authorization

Delta Schedule Growth | Project Schedule Growth |

Project Schedule Factor Total Project Duration*

Predicted Duration at Authorization + Approved Changes

Delta Schedule Factor | 1- Schedule Factor |

*Owner Definition Beginning of detailed engineering through start-up*Contractor Definition Duration for scope of work

Procure Forms

5

5

Mobilize

10

3

Layout Exc 1

15

4

Forms Conc 1

30

3

Set BasePlate1

35

1

Layout Exc 2

20

4

Forms Conc 2

40

3

Set Base Plate 2

50

1

Layout Exc 3

25

4Forms Conc 3

45

3

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

Procure Forms

5

5

Procure Forms

5

5

Mobilize

10

3

Mobilize

10

3

Layout Exc 1

15

4

Layout Exc 1

15

4

Forms Conc 1

30

3

Forms Conc 1

30

3

Set BasePlate1

35

1

Set BasePlate1

35

1

Layout Exc 2

20

4

Layout Exc 2

20

4

Forms Conc 2

40

3

Forms Conc 2

40

3

Set Base Plate 2

50

1

Set Base Plate 2

50

1

Layout Exc 3

25

4

Layout Exc 3

25

4Forms Conc 3

45

3

Forms Conc 3

45

3

Set Base Plate 3

55

1

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

Schedule PerformanceMetrics – Project Level

Phase Schedule Factor (Owner only)- Front End Planning- Detailed Engineering- Procurement- Construction- Start-up

Phase DurationOverall Project Duration*

Phase Schedule Growth - Front End Planning- Detailed Engineering- Procurement- Construction- Start-up

Phase Duration – Predicted Phase Duration at AuthorizationPredicted Phase Duration at Authorization

Procure Forms

5

5

Mobilize

10

3

Layout Exc1

15

4

Forms Conc1

30

3

Set BasePlate1

35

1

Layout Exc2

20

4

Forms Conc2

40

3

Set Base Plate 2

50

1

Layout Exc3

25

4Forms Conc3

45

3

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

Procure Forms

5

5

Procure Forms

5

5

Mobilize

10

3

Mobilize

10

3

Layout Exc1

15

4

Layout Exc1

15

4

Forms Conc1

30

3

Forms Conc1

30

3

Set BasePlate1

35

1

Set BasePlate1

35

1

Layout Exc2

20

4

Layout Exc2

20

4

Forms Conc2

40

3

Forms Conc2

40

3

Set Base Plate 2

50

1

Set Base Plate 2

50

1

Layout Exc3

25

4

Layout Exc3

25

4Forms Conc3

45

3

Forms Conc3

45

3

Set Base Plate 3

55

1

Set Base Plate 3

55

1

SS0

SS2

FS2

FS2

SS2

FS2

* Includes Front End Planning Front End Planning through Start-up

Schedule PerformanceMetrics – Phase Level

Benchmarking Safety

Questionnaire –Reporting Safety Data

Safety Metrics

Benchmarking Changes

Questionnaire – ReportingCost and Schedule Changes

Change Metrics

Benchmarking Rework

Questionnaire – ReportingCost and Schedule Rework

Rework Metrics

Benchmarking Project Impacts and Outcomes

Questionnaire –Reporting Impacts / Outcomes

Benchmarking Best Practices

COAA/CII Best Practices

1.Front End Planning2.Project Risk Assessment3.Team Building4.Alignment during Front End Planning5.Design for Maintainability6.Constructability7.Materials Management8.Change Management9.Zero Accident Techniques10.Quality Management11.Automation/Integration Tech12.Planning for Startup13.Prefabrication/ Preassembly/ Modularization14.Workface Planning

Example – Reporting Best Practice Use Information

The COAA Productivity Benchmarking Program provides:

• A statistically credible resource for member companies to support productivity improvement through external benchmarking.

• Industry research to increase the understanding of engineering and construction productivity drivers and constraints.

COAA Productivity Objectives

Benchmarking Engineering Productivity

01020304050607080

Proj A Proj B Proj C Proj D

WH/Dwg

Company Average = 58

Traditional Engineering Productivity Measurement

Piping Engineering Productivity

00.10.20.30.40.50.60.70.80.9

1

Proj A Proj B Proj C Proj D

WH/LF

Company Average = .74

Piping Engineering Productivity

COAA Engineering Productivity Measurement

• Quantity Based Metrics• Focus on Significant Activities• Appropriate Level of Detail• Only Direct Work-Hours• Common Definitions for Direct/Indirect Accounts• Actual Productivity

Raw Productivity =Actual Direct Engineering Wk-hrs

IFC Quantity

COAA Engineering Productivity Metrics

Direct

Indirect

• Document Control• Reproduction Graphics• Project Management• Project Controls

• (cost/schedule/estimating)• Project Engineer• Secretary/clerk• Procurement

• (supply management)• Construction Support

• (test package support, commissioning, etc.)

• Quality Assurance• Accounting• Legal

• Discipline Engineer• Designer• Technician

COAA Engineering

Hours

By Discipline– Concrete– Structural Steel– Equipment– Piping– Electrical– Instrumentation

COAA Engineering Productivity

COAA Engineering (Concrete) Productivity Metrics

Questionnaire – Reporting Engineering Productivity

Piping Example

Benchmarking Construction Productivity

• Focus on Significant Activities• Appropriate Level of Detail• Only Direct Work-Hours• Common Definitions for Direct/Indirect Accounts• Actual Productivity

Raw Productivity =Actual Direct Construction Wk-hrs

Quantity Installed

Construction Productivity Metrics

Consensus Work HourDefinitions

(See Next Slide)

• Direct Craft Labor• Foreman • General Foreman• Load and Haul• Oilers • Operating Engineer• Safety Meetings• Scaffolding• Truck Drivers Direct

Direct

Indirect

Direct & Indirect LabourTable

By Craft– Concrete– Structural Steel– Equipment– Piping– Electrical– Instrumentation– Insulation– Scaffolding– Module Installation

COAA Construction Productivity

Level I Level II Level III Level IV

Project Total Concrete

Total SlabsOn-GradeElevated Slabs/On DeckArea Paving

Total Foundations

< 5 CM5~20 CM21~50 CM>50 CM

Concrete Structures NA

Construction Productivity Levels(Concrete)

Questionnaire – Reporting Construction Productivity

Piping Example

PipingCalculation of Weighted-Average Diameter The following equation is provided for purposes of calculating the weighted-average diameter of piping for each of the material classifications.

weighted-average diameter

where:x = is the length of pipe (in metres)d = is the diameter of the pipe (in mm)i = is the summation index for pipe diameter for a given materialn = is the summation limit which is equal to the number ofdifferent pipe diameters for a given material type.

Productivity Metrics Calculations

Pipeline ProjectsData and Metrics

COAA Benchmarking Framework

Level 1 Level 2 Level 3

Upstream(Oil Exploration/

Production)

Oil Sands SAGD

Cogeneration

Central Plant Processing Facilities

Pad and Gathering

Oil Sands Mining/ Extraction

Oil Sands Mining

Central Plant Processing Facilities

DownstreamOil Sands Upgrading

Naptha Hydrotreater Unit

Hydrogen Plant

Oil Refining Utilities and Offsite

Natural Gas Natural Gas Processing

Pipelines

Well Sites / Well Pads

Questionnaire – Reporting Pipeline Data

Pipeline Metrics

Project Cost per Spread Total Project Cost*Number of Spreads*

Pipe Cost Factor Pipe Cost*Total Installed Cost (TIC)*

... additional metrics … additional calculations

Current Pipeline Research efforts

Revise existing metrics and develop new metrics

for Pipeline Projects

* Owner Definition; for Contractor Budget at Contract Award

Well Site Projects andWell Pad ProjectsData and Metrics

COAA Benchmarking Framework

Level 1 Level 2 Level 3

Upstream(Oil Exploration/

Production)

Oil Sands SAGD

Cogeneration

Central Plant Processing Facilities

Pad and Gathering

Oil Sands Mining/ Extraction

Oil Sands Mining

Central Plant Processing Facilities

DownstreamOil Sands Upgrading

Naptha Hydrotreater Unit

Hydrogen Plant

Oil Refining Utilities and Offsite

Natural Gas Natural Gas Processing

Pipelines

Well Sites / Well Pads

Questionnaire – Reporting Well Site Data

Well Site Metrics

Cost versus Flow Rate Total Installed Cost (TIC)*Design Flow Rate*

Schedule versus Flow Rate

Total Project Duration*Design Flow Rate*

… additional metrics … additional calculations

Current Well Site Research efforts

Revise existing metrics and develop new metricsfor Well Site Projects and

Well Pad Projects

* Owner Definition; for Contractor Budget at Contract Award

Current Database Research Efforts• Review addition of Facility Capacity data• Project data templates (for Project Teams)• Develop new metrics for Industrial Projects,

especially SAGD Projects• Develop new Reports• Develop new Data Mining capabilities• Review Indirect Costs• Review Commissioning/Start Up Costs• Review Modularization• Review Quality of data input and output• … to be continuedNote: we will continue to work with our Industry Partners in these Research Areas

Questions

Comments

Discussion

Dr. Elias IkpeAccount Manager (University of Calgary)

(587) 228-0311eoikpe@ucalgary.ca

Jatinder KumarGraduate Student (University of Calgary)

(403) 462-4433jkumar@ucalgary.ca

Dr. Stephen MulvaAssociate Director (CII Benchmarking & Metrics)

(512) 232-3013smulva@cii.utexas.edu

Thank youContact us