Office of the County Auditor - Broward County, FloridaComputronix (CX) for PLCP at a cost of approximately $1.6 million. CX developed the system using their Public One-Stop Service

PostImplementation Review of the Building Code Automation System at the Permitting, Licensing, and Consumer Protection Division

June 3, 2009 Report No. 09-11

Office of the County Auditor Evan A. Lukic, CPA

County Auditor

TABLE OF CONTENTS

Topic Page

EXECUTIVE SUMMARY................................................................ 1

BACKGROUND ............................................................................ 2

SCOPE ........................................................................................ 2

OBJECTIVES AND PROCEDURES.................................................. 2

FINDINGS AND RECOMMENDATIONS .......................................... 3

SYSTEM DESIGN AND IMPLEMENTATION FINDINGS................. 4

SYSTEM OPERATIONS AND MAINTENANCE FINDINGS ...........10

Permitting, Licensing & Consumer Protection

EXECUTIVE SUMMARY

This report presents the results of our post-implementation review of the Building Code Automation System used by the Permitting, Licensing, and Consumer Protection Division (PLCP).

The Building Code Automation system was custom designed and developed by Computronix (CX) for PLCP at a cost of approximately $1.6 million. CX developed the system using their Public One-Stop Service System (POSSE) product. POSSE provides a flexible platform for the custom development of specific functionality based on design requirements established by the user.

Findings resulting from our review are principally categorized within the following two areas:

• System design and implementation

• Operations and maintenance of the system

The system design and implementation issues result from deficiencies in the system design. The CX system design was approved by staff before commencement of system implementation. The delivered system functions as designed. However, PLCP staff subsequently determined the design specifications omitted desirable processing functionality and management reports.

Concerns related to the operations and maintenance of the system include:

• Control weaknesses - Access and programmed code change controls are not adequate to protect system and data integrity.

• Excessive use of system utilities - Supplemental internal controls are necessary to compensate for processing difficulties encountered by PLCP staff using the system. PLCP staff frequently finds it necessary to interrupt system routines in the course of routine permit processing.

• Vendor support - From a perspective of continued operations, both the POSSE application and database software components are behind current vendor releases.

As a result of the above, the PLCP Building Code Automation System operates with a lesser degree of effectiveness and efficiency than originally anticipated. Furthermore, the current use of the system indicates a lack of programmed controls and flexibility that reduces PLCP’s ability to rely on the system and fulfill transaction processing needs.

Office of the County Auditor

1


BACKGROUND

In 2000, Broward County licensed the “Public One-Stop Service System” (POSSE1) software from CX and embarked on the first of several projects to automate permitting and licensing activities. Under the direction of the County’s Chief Information Officer (CIO) and agency management, the County’s POSSE systems were designed and developed in part by CX, staff from Enterprise Technology Services (ETS), and agency staff. In concept, the POSSE software was customized to provide business solutions for each individual user agency based upon each agency’s specifications.

In the course of automating permitting and licensing activities, the County contracted with CX for the replacement of the existing Building Code Automation System2. The replacement system design specified an “integrated, Internet-enabled, licensing, damage assessment, permitting, inspection, plan review, building code enforcement, voice response, and land management system with the capacity to accommodate electronic plan review permitting.” The Building Code Automation System went ‘live’ in February 2005, at a cost of approximately $1.6 million. The Director of Permitting, Licensing, and Consumer Protection issued final acceptance of the system at the end of August 2006.

SCOPE

In consideration of the $1.6 million fee paid to Computronix for system development and the complexity of administering the Building Code regulations, the scope of our review focused on the Building Code Automation system implemented for use by the Permitting, Licensing and Consumer Protection Division3 (PLCP) of the Growth Management Department.

OBJECTIVES AND PROCEDURES

Our objectives were to evaluate:

• System development in accordance with design specifications;

1 Item #19 approved 12/12/2000, RLI031799-RB. Initial Software License Agreement between County and Computronix. Article 1, Definitions and Identifications, sec 1.9: “Licensed Software means POSSE, a "Public One-Stop Service System"... POSSE is a modular work management system that combines configurable workflow, desktop mapping, electronic documents, Internet access, and remote computing technologies into a single, integrated multi-media database.”

2 Item #52E approved 8/19/2003, RLI 121701-RB. 3 The current PLCP Division was formerly named the Building Code Services Division.


2


• Compliance with contract terms;

• Effectiveness and efficiency of system functionality;

• Automated and manual controls over permit processing; and

• Adequacy of programmed controls over fee calculations and collections.

In order to accomplish these objectives, we performed the following:

• Reviewed contract documents, including the statements of work, requirements analyses, deliverables and payment schedules and final acceptance criteria, to obtain an understanding of the contract requirements for the system delivered to PLCP.

• Observed system processing and programmed workflows to substantiate the delivery of contracted application software and the current use of the system.

• Compared stated requirements with delivered system functionality.

• Reviewed system testing and acceptance procedures.

• Reviewed project records to recalculate and verify fees charged and collected for a sample of building permits.

• Reviewed system reports of access permissions, corrections to permits, fee adjustments, and use of system utilities.

• Reviewed the process for maintaining the system, including access privileges and history of change control activity.

FINDINGS AND RECOMMENDATIONS

Our findings are principally categorized in the following two areas:

• System design and implementation –pages 4 to 9

• System operations and maintenance – pages 10 to 14

The system design and implementation findings present concerns with the attainment of a satisfactory fully functional system. The system operations and maintenance findings identify opportunities to improve management controls and increase the effective and efficient use of the system.


3


SYSTEM DESIGN AND IMPLEMENTATION FINDINGS

Findings in this section result from our evaluation of system functionality and processing against requirements set forth in the approved CX Design Document, Systems and Services Agreement and related statements of work, and records of project events.

Finding 1. The Approved System Design Omitted Certain Operational Features.

In January 2003, CX completed an analysis of business operations at the Building Code Services Division, the Fire Marshal, and the Zoning Code Services Division. The resulting deliverable was a Design Document for a Data Management and Workflow System using POSSE.

In August 2003, the County engaged CX to develop a replacement Building Code Automation System, based on the CX Design Document. Final acceptance of the system occurred on August 31, 2006.

Other than as noted in finding number 4. (page 7), CX delivered the system as specified in the system design document. The CX system design was approved before CX was authorized to proceed with the implementation of the system. However, during implementation and in subsequent use of the POSSE product, PLCP staff found that the system design lacked desirable operational features. As a result, PLCP staff compiled a listing of 182 features and reports that would enhance operational effectiveness and efficiency of the system (Details at Exhibits 1 and 2 at end of report). For example:

• Display a warning flag when contractors or property owners have unresolved violations, to prevent issuance of follow-on permits.

• Email plan reviewer comments to contractor and inspection tickets to inspectors.

• Provide a checklist of items for each inspection type to be printed on the inspection ticket.

• Report on the number of inspections that are scheduled in a selected month by trade and jurisdiction broken down by weekly total and monthly grand total.

PLCP management requested a cost quote from CX to incorporate the additional functionality into the system. CX provided minimum/maximum estimates for the individual tasks with a total cost estimate ranging between $440,000 and $740,000. PLCP management decided against further investment in CX’ system development services.


4


Recommendations

We recommend the Board of County Commissioners direct the County Administrator to take the following actions within 90 days of adoption:

1) Determine why the design process failed to identify and provide for operational features and reports, which were noted during development and subsequent use of the custom software solution.

2) Revisit the PLCP listing(s) of features and reports not addressed in the POSSE system and determine whether the listed items remain valid. Prioritize the features and reports based on operational needs on a cost versus benefit basis.

3) Develop an action plan to address the deficiencies in POSSE features and reports, based upon priority and availability of resources.

Finding 2. The System, as Delivered, Does Not Accommodate Typical Changes to Permits during Routine PLCP Transaction Processing.

PLCP staff makes manual adjustments to permits due to changes encountered during permit processing. The system was apparently designed without the functionality to efficiently and effectively process changes to permits. As a result, when changes to permits occur, PLCP staff use system utilities4 which circumvent the programmed transactional workflow. Staff commonly invokes two utilities: one utility to manually adjust fees and a second utility to “RESCUE” a permit.

The manual fee adjustment feature allows PLCP staff to input a line item adjustment and increase or decrease a previously established permit fee. A review of manual fee adjustments activity reports during the last nine months of fiscal 2008 revealed 57 PLCP staff created 2,576 manual fee adjustments.

The “RESCUE” feature enables PLCP staff to reverse and restart or change a step in the permit process. A review of “RESCUE” activity reports from July to November 2008

In computers, a utility is a small program that provides an addition to the capabilities provided by the inherent system. A utility is a special and nonessential part of the system. The print "utility" that comes with the operating system is an example. It's not absolutely required to run programs and, if it didn't come with the operating system, you could perhaps add it.


5

4


revealed 41 PLCP staff used the “RESCUE” utility 3,565 times in the four month period, or an average of 40 times/work day.

System utilities, when subject to appropriate controls, are acceptable for limited use when it is necessary to manually manipulate the system for valid exceptions. However, the frequency of manual adjustments and RESCUE use, combined with the number of users indicates a system lacking adequate functionality (See finding number 8, page 12 for discussion on related control issues). The system should accommodate the majority of daily transactions with minimal manual intervention or use of system utilities. The level of manual intervention required to process daily transitions indicates a significant impairment of system effectiveness and efficiency.

We found the following as apparent contributing factors causing manual adjustments and “RESCUE” transactions:

• The system lacks sufficient functionality to automatically calculate and assess recurring fees based on permit criteria;

• PLCP staff may lack an adequate understanding of how to use available system functionality;

• The system design was inadequate to support the many facets of processing permits.

Recommendations


4) Develop a comprehensive plan to address system inefficiencies which require excessive manual intervention by PLCP staff.

Finding 3. The System Lacks the Functionality to Match PLCP Labor Costs with Permit Revenues.

The system was designed without the ability to match permit revenues with operational costs. While unnecessary for permit transactional processing, the matching function facilitates periodic reviews of the permit fee structure. PLCP management should operate PLCP using permit fees which adequately fund processing costs for each category of permit. Insufficient permit fees result in the County subsidizing contractor costs while excessive fees add an unnecessary financial burden to construction projects. System functionality designed to report fees received and cost incurred for each permit would greatly facilitate fee analysis by PLCP management.


6


The nature of the permitting process precludes the use of manual methods to efficiently match revenues and costs. A permit application generates fee revenues based upon several variables, such as the size, duration, and complexity of the construction project. In response to a permit application, PLCP staff performs various tasks, such as reviewing plans, inspecting construction sites, and related administrative functions. The actual PLCP staff assignments and amount of processing time required for a permit will depend upon the attributes of the construction project. As a result, a permit may result in numerous PLCP staff with a wide range of hourly rates working on the permit. In addition, the duration of a permit from application to project completion may range from a few months to several years. Staff may therefore work on permits for which the fees were received in a prior fiscal year. This further complicates the matching of permit fees with processing costs.

The complication of multiple PLCP staff hours and rates combined with various aspects of processing permits appropriately necessitates developing system functionality to efficiently match the revenues and costs.

Recommendations


5) Develop a PCLP strategy for tracking permit revenues and associated costs. The strategy should address:

• Capturing costs by permit of the incurred processing costs.

• Reporting on the actual costs of permit services in relation to revenues collected.

• Analyzing of revenues and costs by permit and permit types.

6) Present a plan for implementing the approved strategy, based upon priorities and available resources.

Finding 4. PLCP Management Accepted the System without Requiring Delivery of a Specified Feature. As a result, the System Lacks the Functionality to Automatically Update Parcel, Zoning, and Jurisdiction Information Used in the Permitting Process.

The permit process requires current folio data. As such, the System Scope of Work included a requirement for CX to develop an ongoing interface with the County Records system for a daily update of folio data. At the time of final acceptance, CX completed an initial download of folio data, but the recurring interface was not developed. PLCP


7


management made a decision to accept the system without the specified automated property folio update feature.

According to project documentation, PLCP and the County’s Enterprise Technology Services Division (ETS) management decided that ETS staff would develop the interface with the County Records database. Upon delivery and testing of the ETS developed interface, PLCP staff discovered a major problem. The interface updated property reference data in all permits, both closed and active. When parcel jurisdictions changed, the change triggered revisions to previously established permit fees and related historical transaction data. PLCP management decided to discontinue using the interface. We found no evidence of further efforts to rectify the interface deficiencies.

As a result, the PLCP system database contains property data as of 2004, when the system was installed. Without the automated interface, PLCP staff must manually obtain, verify, and enter the property folio information for each permit. Because of a manual rather than an automated process, each permit application review requires additional PLCP staff time.

We found no record of negotiations for accepting the system without the property folio update feature. PLCP and ETS staff made the decision to delete the property update feature without following the County’s procedures for amending the terms of an agreement. Compliance with the amendment process facilitates obtaining an equitable adjustment of price and/or other terms resulting from changes in the scope of services.

Recommendations

We recommend the Board of County Commissioners direct the County Administrator take the following actions within 90 days of adoption:

7) Provide remedial instruction to Contract Administrators on the County policies and procedures for amending contractual scope(s) of services;

8) Establish a requirement for a Contract Administrator’s statement in subsequent requests to the Board for final payment that certifies the County received all goods and services as specified in the agreement.

9) Determine the feasibility of automating a data interface for folio data updates. If feasible, develop a strategy for implementing; if not, design a formal manual process for verification of folio data at time of permit application and again at final inspection, to enable adjustment of permit fees if necessary.


8


Finding 5. PLCP Staff Manually Calculate Zoning Code Services Permit Fees for Entry into the System.

Zoning Code Services became a part of the PLCP division in April 2005, after the design and implementation of the system. Accordingly, the system lacks the functionality required for automated calculation of zoning and landscape permit fees. As a result, PLCP staff manually calculates the fees for each zoning and landscape permit by completing a paper worksheet. A PLCP zoning specialist then enters the fees into the system using the fee adjustment function.

The lack of system functionality creates concerns regarding the consistency, accuracy, and reliability of the processed transactions. In our review we found inconsistencies in the process of assessing zoning code permit fees. Upon attempting to reconcile system fees with paper worksheets, we found fees entered into the system without worksheets on file. We also found completed worksheets on file without corresponding fee assessments entered into the system. In addition, the process lacks a report for verification of zoning/landscaping fee transaction. A transactional report could help identify incomplete and inconsistent fee assessments.

Recommendations

In order to facilitate the consistent billing of zoning/landscaping fees, we recommend the Board of County Commissioners direct the County Administrator to take the following actions within 90 days:

10) Develop a process to help ensure PLCP staff consistently document the calculation and assessment of zoning code fees;

11) Develop and review a transactional report to verify the consistency and accuracy of the processed zoning and landscaping fees;

12) Evaluate the cost effectiveness of automating the fee calculation and assessment process for zoning and landscaping permits.


9


SYSTEM OPERATIONS AND MAINTENANCE FINDINGS

Findings in this section result from our evaluation of general information technology (IT) controls to obtain reasonable assurance that the purchased system operates as intended and that data is reliable. IT control objectives relate to the confidentiality, integrity, and availability of data and the overall management of the IT function as a subset of an internal control framework.

Finding 6. PLCP Management Controls over Manual Fee Adjustments of Permits and Licenses Require Improvements.

In our review of controls over manual adjustments to system data, we found the following:

• The system accepts manual fee adjustments on permits and licenses by PLCP staff without entry of PLCP supervisory approval of the adjustment, an omitted preventive control feature.

• Reports of adjustment activity for routine PLCP supervisory review were not delivered with the system to identify unusual or irregular transactions, an omitted detection control feature.

• The system accepts fee adjustments without data input validation controls.

In the absence of appropriate system controls, management needs to actively review transactions for inappropriate or incorrect fee adjustments. Without the mitigating control of management review, the integrity of permit fees charged and collected becomes vulnerable to compromise.

Recommendations

In order to reduce the risk of incorrect or inappropriate manual permit fee adjustments, we recommend the Board of County Commissioners direct the County Administrator to take the following actions within 90 days of adoption:

13) Define and document the acceptable parameters for manual fee adjustments.

14) Develop a recurring report identifying adjustment transactions processed through the system.

15) Require supervisory review of adjustment activity, to facilitate detection and investigation of any unusual or irregular items.


10


Finding 7. PLCP Management Operates the System with Inadequate Access Controls.

General Information Technology controls, particularly controls over user access to systems and functions, are fundamental to protecting data integrity. Appropriate access controls help ensure that users are restricted to perform only their assigned tasks. Restricted access also creates accountability for actions performed on the system. In the absence of adequate system access controls, transactional data becomes vulnerable to manipulation, which may impair data integrity.

In our review of system access controls, we found the following:

• PLCP staff use system access privileges which exceed those required to perform their assigned duties.

• System access privileges lack appropriate segregation of duties between PLCP staff and PLCP supervisors, such as access to create, adjust, and remove permit fees.

• System access privileges remain active for terminated PLCP staff.

• System administrator access privileges allow alteration of permit records without appropriate supervisory oversight.

• Temporary staff use generic system access which precludes identification of transactional history by specific individuals.

• PLCP staff system access includes unrestricted use of system utilities which allow alteration of permit transactional processing

• The system was implemented without basic password parameters, such as minimum length, use of symbols, and expiration at predefined intervals.

Recommendations

In order to improve access controls, we recommend the Board of County Commissioners direct the County Administrator to take the following actions with 90 days of adoption:

16) Review and update user access to eliminate incompatible functions.

17) Develop written guidelines for granting temporary additional system access required for special staff assignments.

18) Maintain evidence of supervisory authorization when granting or changing staff system access.

19) Periodically validate system access for compliance with established guidelines.


11


20) Discontinue the practice of using generic user access to the system. Instead, create individual accounts for all users to maintain accountability for system transactions.

21) Investigate solutions to implement improved system password security parameters.

Finding 8. PLCP Staff Process Changes to Permits with Inadequate Controls and PLCP Management Review.

PLCP staff use system utilities to process changes to permits. The system utilities provide considerable individual latitude when processing changes. The utilities allow PLCP staff to enter amounts, codes, and change justifications without data entry controls. Data entry controls validate data for reasonableness and consistency thereby helping to prevent input errors. Further compounding the issue, the system was not delivered with transactional audit reports which could facilitate the detection of processing errors and irregularities. As result, the ability of PLCP management to review changes processed by PLCP staff is impaired.

PLCP staff commonly invokes the “RESCUE” system utility, which essentially re-sets the status of a permit. For example, PLCP staff may manually change a permit status from “BCS Plan Review” to “Ready to Issue”. There are no standard justifications required or prior PLCP management approval required when changing permit status.

System utilities, when used with appropriate management controls, provide an acceptable mechanism to process valid transactional exceptions. However, PLCP staff use the utilities in routine processing without adequate access controls, input standards, and management reviews.

Recommendations

In order to improve controls over permit operations and data integrity, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions within 90 days of adoption:

22) Establish written guidelines for:

• Assigning access to system utilities;

• Acceptable use of system utilities, including standard reason codes; and

• Change thresholds requiring prior PLCP management approval.

23) Develop and deliver PLCP staff training on acceptable use of utilities, including provisions for periodic ‘refresher’ sessions.


12


24) Establish a PLCP management process to monitor the use of sensitive system utilities, including fee adjustments and transaction rescues. The process should provide for:

• Development of system reports which list the adjustments and changes processed by PLCP staff and

• PLCP supervisory review of adjustment activity to identify and investigate any unusual or irregular items.

Finding 9. The Programming Change Controls Are Inadequate to Prevent Unintentional or Malicious Changes to the System Configuration.

PLCP staff prepares system configuration changes, which are promoted to production by ETS without an independent review of the actual programming script or change. The promoted item may lack compliance with County programming and configuration standards. Further the promoted change may introduce erroneous code, faulty logic, or malicious functionality undetected during testing by PLCP staff. In the absence of an independent technical review, the PLCP program changes could compromise the permit processing integrity of the system.

ETS and PLCP staff currently use the following program change management process for the system:

1. The PLCP developer/analyst manages the design, creation, and testing/acceptance of any changes to the system.

2. Upon user testing and acceptance of specified changes, the PLCP developer/analyst provides the changes to ETS staff for implementation.

3. The ETS change control committee coordinates and implements, without independent verification, the changes provided by the PLCP analyst.

Recommendations

We recommend that the Board of County Commissioners direct the County Administrator to take the following actions within 90 days of adoption:

25) Require an independent review of proposed system changes for the inclusion of inappropriate or unauthorized programming code changes prior to implementation.


13


Finding 10. The Building Code Automation Application and Database Software are Behind Current Vendor Versions.

We found that the PLCP system is operating behind the current vendor versions for both the application software and the database. The County developed the system application using POSSE application software and an Oracle database. CX, the vendor for POSSE, currently recommends POSSE version 6.1. The division operates several releases behind with POSSE version 5.4.7.5. A similar situation exists for the Oracle database software. The current Oracle database is version 10G while the division uses version 9i.

A delay in installing new versions places the County’s investment in the system at risk. Software vendors routinely terminate support and maintenance for older versions of their products. Without on-going support and maintenance, the County may incur system malfunctions without contractual recourse for timely resolution by the vendor. CX has advised, without releasing a firm date, support for POSSE version 5.4.7.5 will terminate. Support for the Oracle database version 9i expired in July 2008.

Recommendations

In order to ensure technology resources are available to meet business needs, we recommend the Board of County Commissioners direct the County Administrator to take the following actions within 90 days:

26) Report on the estimated level of County staff effort and costs required to update to the current releases of the system application and database software.

27) Develop a strategy to address updating the system and database to current releases.


14

Permitting, Licensing, & Consumer Protection Division

Exhibit 1: Additional System Features Identified by PLCP Staff During Implementation and Use

Features/Functionality by POSSE Module Count Minimum Quote Maximum Quote

Certificate of Use Configuration Change 1 $ 6,450 $ 12,339

Code Enforcement Board Hearing 1 10,950 17,627 Violations 1 6,150

WorkFlow Change 2 3,750 5,288 Contractor Case

Various 1

Elevator Configuration Change 6 8,250 13,220 Conversion Script 1

Electronic Documents 1

Email 1

Expired Permit / License 1

Interface 1

Web 1 1,050 1,763 WorkFlow Change 1 1,500 1,763

Enforcement Case Various 2 2,250 2,644 WorkFlow Change 1 1,500 1,763

Exam Board BCS Record Exam Results 1 1,500 1,763 Configuration Change 2 6,750 7,051 Contractor Case 2 3,750 8,814 Email 1 14,400 17,627 Interface 2

New Job 1

WorkFlow Change 1

Fees Configuration Change 5 14,700 19,390

Inspections Permit 2 3,750 8,814

Invoice

A cost quote was not provided for this item.


15


Exhibit 1: Additional System Features Identified by PLCP Staff During Implementation and Use

Features/Functionality by POSSE Module Count Minimum Quote Maximum Quote

Configuration Change 1 7,350 7051 Interface 2

Licensed / Unlicensed Configuration Change 4 13,950 24,678 Contractor Case 2

Electronic Documents 1

Unassigned Issues Various 17 37,575 48,034

Parcel Configuration Change 1 4,800

Permitting Configuration Change 7 18,150 27,322 Contractor Case 1 3,600 5,288 Email 2

Fees 4 7,650 19,390 Inspections 7 23,550 28,203 Issuing CO & CC 3 7,350 10,576 Master Permit 2 4,800 7,051 Missing Designation 1 750 881 Permit 1 1,500 1,763 Plan Review 6 15,150 23,797 Related Parties 1 1,500 1,763 Search Window 1

Violations 1 1,500 3,525 Web 1 6,300 8,814 WorkFlow Change 1

Plan Review Fees 1

WorkFlow Change 1 3,750 17,627 POSSE

Email 1

Interface 1

Records New Job 1 2,700 3,525

Total Cost Quote for Features and Functionality 111 $ 248,625 $ 359,153


16


Exhibit 2: Additional System Reports Identified by PLCP Staff During Implementation and Use

Reports by POSSE Module Count Minimum Quote Maximum Quote

Administration 1 $ 3,300 $ 8,814 Certificate of Use 5 20,850 29,966 Code Enforcement 17 26,550 49,356 Elevator 12 34,200 89,898 Exam Board 9 20,700 29,966 Fees 2 2,700 5,288 Licensed / Unlicensed 16 56,700 121,627 Unassigned Reports 2 4,500 8,814 Permitting 7 21,750 37,017

Total Cost Quote for Reports 71 $ 191,250 $ 380,746


17