Coordination and management of a large multicenter screening trial: The prostate, lung, colorectal and ovarian (PLCO) cancer screening trial

ELSEVIER

Coordination and Management of a Large Multicenter Screening Trial: The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial

Barbara O'Brien, MPH, Leah Nichaman, Joyce E.H. Browne, David L. Levin, MD, MS, Philip C. Prorok, PhD, and John K. Gohagan, PhD, FACE for the PLCO Project Team Westat, Inc., Rockville, Maryland (B.O., L.N.); and Early Detection Research Group (J.E.H.B., J.K.G.) and Biometry Research Group (D.L.L., P.C.P.), Division of Cancer Prevention, National Cancer Institute, Bethesda, Maryland

ABSTRACT: The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial is a large and complex multi-institutional, multifaceted organization with a 20-year horizon. The implementation of the trial began with the creation of an organizational structure that supports strong leadership, cooperation, and effective communication among the trial collaborators including an operational framework for the development, review, and pretest of instruments, data collection, and management procedures; the setting of high-quality standards for training of trial staff; and the development of a comprehensive assessment plan for evaluation of all trial activities. This paper describes the process and methods used in the coordination and management of the PLCO trial. These include the role of the steering committee and its subcommittees and working groups, the establishment of regular and ad hoc communications among collaborators, the training of screening center coordinators and examiners, the PLCO manual of operations and procedures, and the development and implementation of a comprehensive quality assurance plan. Control Clin Trials 2000;21:310S-328S © Elsevier Science Inc. 2000

KEY WORDS: Multicenter clinical trials, data collection, screening, cancer, coordination, operations, management

I N T R O D U C T I O N

In 1992, the Nat iona l Cancer Inst i tute (NCI) u n d e r t o o k a t w o - a r m e d trial wi th 148,000 m e n a nd w o m e n , ages 55-74 years at entry , to be enrol led at ten

Address reprint requests to: Dorothy Sullivan, Early Detection Research Group, Division of Cancer Prevention, National Cancer Institute, EPN 330, 6130 Executive Blvd., Bethesda, MD 20892-7346 (E-maih [email protected]).

Received March 27, 2000; accepted May 31, 2000.

Controlled Clinical Trials 21:310S-328S (2000) © Elsevier Science Inc. 2000 0197-2456/O0/$-see front matter 655 Avenue of the Americas, New York, NY 10010 PII S0197-2456(00)00099-4

Coordinating a Multicenter Screening Trial 311S

~ k _ ~ o n i t o r i n g and Advisory ~'~ NCI OfficersC°ntracting NCl(ocPProjeCtand DCEG)Officers ! ~ Panel (MAP) •

Coordinating Center (CC) Westat Committee

I Biore[ 'sit°ry I

Laboratory

Biological Sample Storage

q Analytic I Laboratory

Screening Centers (SCs) University of Colorado Health Sciences Center, Denver, CO Georgetown University Medical Center,

Lombardi Cancer Research Center, Washington, DC Pacific Health Reseamh Institute, Honolulu, HI Henry Ford Health System, Detroit, MI University of Minnesota, Minneapolis, MN Washington University School of Medicine, St. Louis, MO Cancer Institute of Brooklyn at Maimonides, Brooklyn, NY

(discontinued 1997) University of Pittsburgh Cancer Institute, Pittsburgh, PA

Satellites: Latrobe Area Hospital, Latrobe, PA; Jameson Health System, New Castle, PA; Trinity Health System, Steubenville, OH

University of Utah School of Medicine, Salt Lake City, UT Satellite: St. Lukes Meridian Medical Center, Boise, ID

Marshfield Medical Research and Education Foundation, Marshfield, WI University of Alabama at Birmingham, Birmingham, AL (added 1997)

___L_ ~omputin 8 Support Provider

IMS

Figure I The PLCO organizational structure.

screening centers throughout the United States. Half of the participants in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial are randomized to receive their usual medical care, while the other half are randomized to screening. In the screened arm, men and women receive chest X-ray and flexible sigmoidoscopy, men receive digital rectal examination (DRE) and prostate-specific antigen test (PSA), and women receive a CA125 test and transvaginal ultrasound exam (TVU). All screening exams with the exception of the flexible sigmoidoscopy are administered at intake and annually for up to 5 years. The flexible sigmoidoscopy exam is administered at baseline and again during the final screen. Details of the protocol are given elsewhere in this supplement [1]. Because the primary goal of the trial is to determine whether screening can reduce mortality, follow-up of all participants for cancer incidence and disease-specific mortality is scheduled to continue for at least 13 years from randomization. For all participants diagnosed with prostate, lung, colorectal, or ovarian cancer, the medical records are reviewed and ab- stracted for diagnostic and treatment information. Pathology slides are obtained for confirmation of the diagnosis of cancer through a central review process. Vital status is determined at yearly intervals after enrollment for all participants, and death certificates are obtained for all participants reported as deceased, both as confirmation of death and to establish cause of death. A central committee reviews all participant deaths for verification of cause of death.

In this paper we describe the implementation, coordination, and management of this large and complex trial.

O R G A N I Z A T I O N A L S T R U C T U R E

The PLCO trial (Figure 1) is directed and funded by the NCI via institution- specific contracts and is managed centrally by the project officers and contract

312S B. O'Brien et al.

officers in the NCI's Division of Cancer Prevention (DCP) and Division of Cancer Epidemiology and Genetics (DCEG). The institutions participating in the trial under contract to the NCI include the coordinating center (CC), which performs data management and overall trial coordination, ten screening centers (SCs), the central laboratory (LAB) at UCLA for blood testing, the biorepository (composed of an archival facility where samples are stored, a processing laboratory that receives samples from the SCs, and an analytic laboratory for molecular research), and a computing support provider. Key members of these entities collectively form the steering committee, chaired by the senior project officer, which provides overall scientific direction for the trial. The NCI also appointed a data safety and monitoring board--the Monitoring and Advisory Panel (MAP)--comprised of independent experts in screening, clinical trials, medical ethics, biostatistics, and the four cancers to provide external monitoring of trial progress.

National Cancer Institute, Division of Cancer Prevention and Division of Cancer Epidemiology and Genetics

Scientific leadership for the trial is provided ultimately by the NCI project officer and associate project officers. They direct all trial activities and monitor and evaluate the performance of each participating organization. They also direct statistical data analysis and reporting and develop statements of work. Approval by the project officer and the publication subcommittee (of the steering committee) is required for all releases of information or data. All contractual issues are handled by the contracting officer with scientific input from the project officer.

Screening Centers

The SCs are responsible for recruitment, screening, follow-up, and collection and shipment of blood samples to the NCI's biorepository in Frederick, Mary- land and to the LAB in Los Angeles, California. They are also responsible for all data collection, local data management, and data-processing activities as required by the trial protocol. The organizational structure of each SC (Figure 2) varies from site to site but includes, at a minimum, the principal investigator (PI), a study coordinator, recruitment staff, certified medical personnel for conducting screening examinations, medical record abstractors, data management staff, and other clerical staff. Several SCs also retain physician consultants in gynecology, urology, gastroenterology, and radiology. Some SCs have satellite centers to provide services related to recruitment and screening. These satellites may have their own coordinators, but are under subcontract to the parent SC with limited functions, and all communications with the satellite are accomplished via the parent SC.

Coordinating Center

The CC (Westat, Inc.) works closely with NCI project officers and trial investigators to develop, implement, monitor, and refine the trial protocol. The CC produces and distributes standardized trial forms and materials, including


Principal ] Co-principal Investigator Investigator

/ [ Screening / Physician ~ Center

Consultants II I Coordinator

I I I I Me!ical / / Coordinator Record Abstractors

Recruitment Support |

Staff Personnel I

Figure 2 Typical screening center (SC) organization.

the manual of operations and procedures (MOOP). It is responsible for data management, quality assurance monitoring, and central coordination. The CC trains the SC coordinators, monitors work completed at the SCs through the receipt of reports and data, and maintains regular telephone contact with the coordinators at each SC. The CC also disseminates quarterly reports to the NCI on the status of recruitment, randomization, screening, and follow-up and conducts site visits and audits as necessary. The CC designed and manages the implementation of computer systems to support recruitment, randomization, receipt control, trial management, data entry, and data editing at the SCs. The SCs scan standardized data forms, and both the SCs and the LAB send computerized data to the CC at predetermined dates. The CC performs quality assurance checks on all data received and reports results to the NCI, the individual SCs, and the LAB.

The CC is organized into functional groups representing project development, administration, coordination, data preparation, systems development, and quality assurance and analysis (Figure 3).

Prior to October 1997, NOVA Research Company in Bethesda, Maryland was responsible for monitoring and coordinating protocol development for biospecimen and etiologic data collection. However, because of the integrated nature of the screening activities and data collection related to etiologic studies, the NCI decided to merge these functions into the role of the CC.

Central Laboratory The Tissue Typing Laboratory of the University of California at Los Angeles

has responsibility for performing all PSA and CA125 assays for this trial. The


Corporate H Project I Officer Director

I Deputy I Project Director

Study Managers Research Assistants

Central Study Coordination Development

Training Quality Assurance

Monitoring

I D= Prepa Supel

Computer I Systems Director

1 [ Computer / I Systems |

ta I ..... Systems visor I Analysts ation Administratorl I

I I o oom0r I Figure 3 Coordinating center (CC) organization.

SCs send samples to the LAB on a weekly schedule, and the LAB makes results available for downloading to the individual SCs within a week of receipt. The LAB works with the CC and the SCs to assure careful and timely shipping and handling of all samples and associated data. The LAB is also responsible for monitoring and maintaining quality assurance of the assays. The LAB reports results of quality assurance testing to the NCI on a quarterly basis. The LAB also conducts special projects regarding assay performance as directed by the project officer.

Biorepository The biorepository is a collaborative endeavor of DCP and DCEG [2]. It

consists of an archival repository where samples are stored and two labora- tories: one for processing samples as they arrive from the SCs and one for analysis of samples. SCs ship aliquots of serum, plasma, and buffy coat ac- cording to the protocol for storage at -70°C. The biorepository maintains a computerized inventory of all samples and tracks all movements of the samples within or outside the facility.

Computing and Analysis Support Information Management Systems (IMS), located in Rockville, Maryland,

provides support for statistical analysis for the PLCO. IMS staff work under the direction of the PLCO senior statistician.

Steering Committee The steering committee is chaired by the senior NCI project officer. NCI,

SC, CC, and LAB investigators comprise the steering committee (Figure 4). All


Screening Center Principal Investigators and Study Coordinators

Laboratory Principal Investigator

Steering Committee

Coordinating Center Principal Investigator

I Working Groups for Operations:

• Brochure • Results Reporting • Operations Evaluation • Periodic Survey of Health • Medical Record Abstracting

Subcommittees

• Protocol • Prostate • Lung • Colorectal • Ovarian • Ancillary Studies • Publications • Internal/External Communications

(discontinued 1998) • Modeling and Simulation (new 1999) • Cost Effectiveness and Quality of Life

(new 1999)

Figure 4 The PLCO steering committee organization.

major protocol initiatives or revisions are reviewed and discussed by the steering commit tee and its relevant subcommittees a n d / o r working groups. Consul- tants are engaged by the NCI for specialized expertise related to issues under consideration. The steering committee meets once a year for 2 days in conjunction with a meet ing of the PLCO coordinators to ensure that not only scientific but operational issues are fully discussed and reviewed. Quarter ly PI and coordinator conference calls provide additional opportuni t ies to discuss issues, problems, and new developments related to trial design, operations, data analysis, and publications. Additionally, project officers and PIs annual ly meet sepa- rately to develop and moni tor progress of data analysis and manuscr ipt preparation.

Subcommittees

Subcommittees are comprised of a combinat ion of NCI-appointed and volunteer representatives f rom collaborating institutions and consultant advisors, as necessary, and each is chaired by an SC PI. NCI project officers are ex-officio members of all subcommittees. The steering committee initially established five subcommittees to address specific aspects of the trial, including the protocol subcommit tee to address trial design issues and issues related to enrol lment of participants, pa thology review, and death review; the ancillary studies subcommittee to review and approve proposals for ancillary studies; and the data


analysis and publications subcommittee to approve proposals for data analysis and materials for publication. For the first 5 years of the trial, two organ site subcommittees (prostate/colorectal and ovarian/lung) addressed protocol development issues and monitored operations related to the screening examinations. In late 1997, their responsibilities were expanded to include data monitoring and analysis, and they were restructured into four organ-specific subcommittees. As data analysis operations were transferred to the four organ-site subcommittees, the data analysis and publications subcommittee was reorga- nized as the publications subcommittee to establish publication guidelines and review for publication manuscripts, abstracts, and other information released from the PLCO.

The NCI also established two subcommittees, the internal and external communications subcommittees, to provide ongoing support and coordination for recruitment effort at the ten SCs, as well as to coordinate the dissemination of public health information from the PLCO trial. These subcommittees were comprised of SC coordinators, representatives from the NCI's DCP and Office of Cancer Communications, and outside marketing experts. In 1998, these two subcommittees were disbanded, although interaction with the NCI Office of Cancer Communications continues.

The modeling and simulation subcommittee, formed in 1999, serves to coordinate mathematical and statistical modeling projects involving PLCO data. The subcommittee serves as a resource for the PLCO investigators interested in utilizing modeling techniques to address questions of scientific or public health interest and acts to encourage such utilization where appropriate. The subcommittee also helps to disseminate the results of modeling efforts among the investigators.

Working Groups

As the need arises, working groups are formed to address specific protocol or other operational issues. These groups include representatives from SCs, the CC, the NCI, and the LAB. The flexible working group format facilitates the evaluation of proposed protocol modifications and expedites the decision- making process. It reduces the number of individuals involved in deliberations while maintaining adequate representation from each collaborating institution. The CC provides primary support to these groups by scheduling meetings and conference calls, developing and distributing materials for discussion, and communicating the outcomes of discussions. To date, the working groups established have addressed the following topics: brochure design, results reporting, operations evaluation, design of the periodic survey of health, and medical record abstracting.

IMPLEMENTATION AND OPERATION OF THE PLCO SCREENING TRIAL

The implementation of the PLCO trial commenced with establishment of a multiyear schedule and lines of communication, development of the detailed protocols for data collection and reporting, development of trial documents (such as the MOOP and data collection forms), development of data management systems, and training of SC staff. As the trial has progressed, the CC has


implemented additional procedures for issue resolution and protocol modification.

Trial Schedule

Following the awarding of contracts to the SCs and the CC, the 24-month pilot phase was programmed for (1) developing and testing trial protocols, procedures, and forms, and (2) obtaining clearance from the federal Office of Management and Budget (OMB) and approval from institutional review boards. During the final 11 months of the pilot phase, beginning November 16, 1993, recruitment of "vanguard" participants was initiated at all centers for the testing, evaluation, and modification of trial forms and procedures. Initia- tion of recruitment for the main phase was begun on September 30, 1994 with enrollment scheduled to conclude on September 29, 1997. Follow-up was planned through September 29, 2007.

Early impacts on the trial schedule included delays attributed to obtaining OMB clearance, difficulties in gaining consensus on complex design issues, and a decision to switch to optical mark-sense forms for data collection. These delayed the initiation of the pilot-phase recruitment from September 29, 1993 to November 16, 1993 and reduced the size of the vanguard group from a planned 15,000 participants to an actual 12,000 participants.

Following the pilot phase, the NCI revised the trial schedule several times. Between 1994 and 1996, the NCI modified the eligibility criteria to exclude from participation individuals who had received at least one flexible sigmoidoscopy, colonoscopy, or barium enema examination or more than one PSA blood test in the 3 years prior to eligibility determination. This decreased the level of contamination in enrolled participants but decreased enrollment rates at most SCs, requiring reductions in enrollment goals at some SCs. The recruitment period was extended 2 additional years to compensate. In 1997, an SC was added to improve recruitment of minority participants and was given 2 and a half years to reach recruitment goals, effectively extending the recruitment period to September 2000. Attempts to add other minority-focused SCs were not successful, and several SCs were given larger enrollment targets and extra time to recruit to assure reaching the PLCO enrollment target. A current time line for the PLCO appears elsewhere in this supplement [1].

Communications

In such a large multicenter trial as the PLCO trial, effective communications are the key to engendering a collaborative spirit and binding the disparate and distant organizational entities into a team with uniform goals. The CC is responsible for establishment and maintenance of these lines of communication, which include meetings of key personnel, training sessions both at the CC and at the individual SCs, conference calls, electronic mail, phone calls, written communications including updates to the MOOP, decision logs, and periodic reports to the SCs and to the NCI. In the PLCO, one or two main contacts in each organizational entity function as the conduit for providing information to other members of that organization. For subcommittees and working groups, the chair performs this function. At the NCI, the major contacts are the project


officers, and at the SCs they include the PIs, coordinators, and data managers. At SCs with satellite centers, all communications to and from the satellites flow through the parent SC. The CC has a coordinator who is expert in all aspects of the protocol and communicates regularly with the SC coordinators and data managers. This CC coordinator routinely conveys changes in policy and procedure, discusses SC progress, investigates protocol violations, obtains information regarding specific procedural issues at the request of the NCI or trial collaborators, and answers SC questions regarding protocols and operational procedures. In addition, the CC maintains a systems user support helpline from 1:00-6:00 p.m. EST, Monday through Friday. A PLCO computer systems expert is available to answer questions regarding data entry, scanning of data collection forms, interpretation of reports or edits, and diagnosis of computer problems. Summary status sheets for each SC are prepared from these communications and used as a basis for weekly reports at CC staff meetings and at biweekly NCI meetings.

To enhance communication between the SCs and their participant popula- tion, in 1998 the NCI established a semiannual PLCO newsletter prepared under the direction of the PLCO project officer and information officer, which is distributed to all participants by the individual SCs. In addition, five of the SCs--Alabama, Detroit, Honolulu, Marshfield (Wisconsin), and Georgetown (Washington, DC)--publish and insert in the NCI Newsletter their own local PLCO newsletter. The PLCO also maintains a web page (http://dcp.nci.nih. gov/plco) for disseminating information.

Protocol Development

The essential structure of the PLCO screening protocol was contained in the PLCO concept approved by the Board of Scientific Counselors (BSC) of the NCI Division of Cancer Prevention and Control in January 1991, and the concept of the biorepository protocol was approved by the BSC in October 1992. Follow- ing contract awards, detailed protocols for recruitment, randomization, screening, contamination assessment, follow-up of positive screens, cancer and vital status ascertainment, central pathology review, and central death data review were developed. Over the next year, prior to the beginning of pilot-phase enrollment, the protocol subcommittee addressed such major issues as methods of randomization and appropriate eligibility and exclusion criteria. The organ- specific subcommittees further refined the screening examination protocols, the format for reporting results of screening exams to participants and physicians, and the protocol for diagnostic follow-up of positive screens.

Once the main phase was underway, several major modifications and addi- tions to the protocol were implemented. Given the size of the operations at the SCs, each modification required significant effort by the CC, the NCI, and the SCs to develop new procedures and systems, to conduct training, to document the new protocol and systems, and to monitor its implementation. The major protocol modifications that have been implemented or are currently in development are listed below in chronological order.

• Redesign most data collection forms in an optical-mark format (June 1993).


Revise the Periodic Survey of Health to eliminate questions about routine screening exams (July 1994). Change the TVU screening examination protocol to redefine a positive and negative examination (August 1994). Change the chest X-ray screening examination protocol to redefine a positive and negative examination (September 1994). Change the eligibility criteria to exclude individuals who report at least one colonoscopy, sigmoidoscopy, or barium enema in the previous 3 years and men who report more than one PSA test in the previous 3 years (April 1995). Change the eligibility criteria to expand the eligible age range from 60-74 years to 55-74 years (January 1996). Change the eligibility criteria to include women whose ovaries have been removed (October 1996). Change the flexible sigmoidoscopy screening protocol to discontinue screening in the fourth project year if the participant was diagnosed with an adenoma and is being monitored with periodic colonoscopy (Decem- ber 1996). Change the ovarian cancer screening protocol to discontinue screening women with TVU, ovarian palpation, and CA125 if their ovaries have been removed (December 1996). Add a contamination assessment component to assess contamination in a randomly selected sample of approximately 100 control participants in each SC (October 1997). Add a supplemental consent process to obtain participant consent for collection of additional blood and buccal cells and to use all biologic samples for studies involving genetic research (May 1998). Collect additional blood from intervention participants for etiologic studies at screening year 3 (May 1998). Add the death review protocol (July 1998). Add a protocol to confirm all cancers reported in control and intervention participants for the duration of the trial (July 1998). Add a dietary questionnaire administered to all control participants at enrollment and to intervention participants during their fourth project year (December 1998). Extend follow-up by 3 years so all participants will be followed at least 13 years from randomization (December 1998). Change from a 3- to 5-year interval for the flexible sigmoidoscopy examination (December 1998). Discontinue the third annual re-examination chest X-ray for "never" smok- ers (December 1998). Discontinue the ovarian palpation examination (December 1998). Add screening years 4 and 5 for PSA and CA125 blood tests (December 1998). Eliminate criteria excluding women who have taken Tamoxifen or Evista/ Raloxifene in the past 6 months from participation (April 1999). Add a buccal cell collection from control participants (April 1999).


SC Tasks

Repeat cycle until consensus is reached

L I Review/comment on draft

forms and procedures

I Conduct pretest J

I Estimate form use ]

CC Tasks I Establish timeline I

Draft forms and procedures ]

Revise forms and procedures l

Revise forms and procedures I

Plan pretest I

Evaluate pretest I

Finalize forms and procedures I

C NCI approval

Develop systems specifications I

Program and test study I management system I (receipt, editing, reports)

Program and test data I entry and editing system I

Order and distribute printed forms I Distribute final procedures memo I

to SCs, NCI, collaborators I

Upgrade SC computer systems I

Figure 5 Protocol development methodology.

Collaborator Tasks

Review/comment on draft forms and procedures

• Collect additional blood from intervention participants at screening years 4 and 5 (July 1999).

• Add collection of pathology materials (pilot in development).

The requirements for protocol development and major modifications are driven by the NCI project officers who develop the statements of work and direct implementation. The CC has developed standard procedures to control the process for development and implementation of protocol modifications and issue resolution and to facilitate communication between the CC, the SCs, the NCI, subcommittees, and working groups (Figure 5).

The CC project director designates one study manager as the development task leader to coordinate all activities related to development of a specific action. Typically, the CC task leader initiates the development process by establishing a time line that includes all activities, e.g., initial design, internal


and external review, pretesting, development of systems support, and implementation. Consultants are notified in advance of when they will be expected to review a document and how much time will be allotted to the review period.

The task leader prepares a first draft of procedural documents, data collection forms, and a description of controversial issues for internal review. The internal review is completed by staff representing the different functional groups within the CC to assure conformation to protocol requirements, feasibility of the plan, and compatibility with the data management system. The draft documents are then submitted to the consultants for review and discussion during planning meetings or conference calls. The task leader refines documents based on consensus decisions of the consultants and submits the draft documents to the SC PIs and coordinators for review. After comments are received from the SCs, the documents are further refined, and the task leader plans a pretest. The pretest may be conducted at the CC or, more often, several volunteer SCs will conduct it. The CC processes the data from the pretest, and IMS analyzes the data and reports results to the NCI project officers.

Following the NCI's evaluation of pretest results, the CC development task leader revises the procedures and forms as necessary and obtains final NCI approval for implementation. The task leader then develops systems requirements for receipt of the data into the SC study management system (SMS) and scanning and editing of the form in the data entry and editing system (DEES). These systems, installed on a local area network at each SC, provide for distributed data entry and editing, SC management of trial activities, transfer of data from the SCs to a central trial database and the various other trial sites, and production of data files for analysis. The design and evolution of the data management system is described elsewhere in this supplement [3].

Protocol Implementation

A fully developed protocol is implemented at the SCs with training, functional and technical documentation, supply of data collection forms and other materials, and, finally, the installation of the computer systems to support the protocol.

Training The development and conduct of intensive training for clinical and LAB

personnel, to ensure that the protocol is clearly understood and that standard procedures are followed across all field sites, is an ongoing effort. The training sessions for SC staff provide the opportunity for hands-on practice with reference materials, data collection instruments, blood-processing protocols and equipment, and computer hardware and software.

Intense comprehensive training of clinical staff began in September 1993 prior to initiation of data collection for the pilot phase. The main objective of the initial weeklong training session was to give the SC coordinators a thorough understanding of the trial objectives, protocol, and schedule. Emphasis was placed on the implementation of the PLCO protocol for recruitment, eligibility determination, randomization, scheduling of screening visits, performance of screening examinations, reporting of results to participants and physicians,


and cancer and vital status ascertainment. Written training manuals provided to SC staff at all training sessions served as their reference document during and after training. The SC coordinators, thus trained by the CC, have ongoing responsibilities for training all their staff on protocols, forms, and administrative procedures. In addition, SC coordinators must arrange for training of nonphysi- cian medical personnel on the performance of PLCO screening examinations. Training requirements for examiners are described in the protocols for the screening examinations.

To ensure that PLCO computer systems would be used properly and that the data contained in the system would correctly represent trial activities, staff at all SCs received three major training sessions on the systems and the interpretation and use of edits and other reports. A training manual including exercises, a user's guide for the SMS, a user's guide for the DEES, and a network (PLCONet) user's guide was provided to all trainees.

A program of periodic retraining of key personnel ensures that new and replacement staff at SCs are brought quickly to competence and that changes of systems and trial procedures are instituted accurately. SC coordinators are trained on new and revised protocols via teleconference or in conjunction with annual steering committee meetings. In November 1996, as medical outcomes began to accumulate in substantial numbers, a 2-day central training focused on the completion of medical record abstract forms. In 1995, 1996, and 1997, data managers were retrained on critical components of the SMS and the DEES during the annual 2-day system users group meetings. In November 1997, the CC conducted a weeklong training session for new SC coordinators at existing SCs as well as the coordinator and data manager from a new SC.

To supplement in-person training, the CC coordinator, who routinely communicates with the SC coordinators, conducts phone calls to review all new and modified protocols and to ensure complete understanding of the new requirements.

Documentation

All protocols; operational, technical, and management procedures; data collection forms; and specifications for completion of data collection forms are contained in the PLCO MOOP, which is the source document and training guide for standard implementation of all procedures. The CC is responsible for its design, production, distribution, and maintenance. Each page of the MOOP is identified with a version date. The manual is structured so that sections dealing with procedures for a particular task can be readily identified and can be provided by the SC coordinator to the individual involved in that particular task without providing the entire manual. For new data collection activities or protocol modifications, the CC task leader prepares and distributes to the SC PIs and coordinators a detailed memorandum with final data collection procedures, forms, and form completion specifications. These memoranda serve as the written protocol until they are incorporated into the annual update of the MOOP. With the distribution of each update, MOOP holders are requested to replace the appropriate pages in their MOOP.

While the CC is responsible for the design and development of procedures and materials, the SCs develop some informational documents provided to


participants. Typically, the CC provides sample versions of these materials (introductory letters to participants, recruitment materials, instructional materials describing preparation for screening examinations, requests to institutions for release of medical records, etc.), and the SCs modify them to reflect their specific institutional requirements and style. CC staff review these materials to verify conformity to the PLCO protocol, and the NCI approves them prior to use.

Materials Inventory and Distribution

In addition to the development and distribution of the PLCO MOOP, one of the CC's major responsibilities is the acquisition and distribution of trial forms and supplies. The process for acquiring and distributing optical mark- sense (op-scan) forms, brochures, blood specimen labels, and vacutainer tubes begins with annual SC inventories of materials on hand and estimated use for the next contract year. This process is initiated approximately 5 months prior to the proposed supply date, which is usually the end of the contract year (September 29). The inventories are submitted to CC staff who compare SC- estimated expected use with expected use based strictly on trial recruitment and screening goals. If the estimates differ significantly, the CC will investigate and resolve the difference in consultation with the SC coordinator. Op-scan forms and brochures are drop shipped to the SCs from the printers. Blood sample identifier (ID) labels are shipped to the CC, where they are tested for accurate scanning and then shipped to the SCs. Samples from each lot of trace metal-free vacutainers are shipped to the Centers for Disease Control and Prevention in Atlanta, Georgia, where they are tested for trace metals. After successful testing, the lots are sent to the SCs directly from the distributor.

To track shipment and receipt of materials, the CC distributes to each SC a listing of the exact number of forms/brochures/ labels/ tubes to be received at each SC. SCs are expected to inventory their materials upon receipt and to notify the CC of any discrepancies in the order. If an SC anticipates exhausting its supply of forms, brochures, or blood labels prior to September 29, the SC coordinator must request additional materials at least 8 weeks in advance of need. A reorder form was developed to facilitate this process. To date, over 60 data collection and administrative forms are inventoried by the SCs and tracked and supplied by the CC.

System Installation and Maintenance

Each new protocol component is considered "implemented" when the computer systems to support it are installed. System upgrades to support protocol modifications are performed approximately three times a year. The system upgrade containing support for a new or modified protocol component is installed 2-3 weeks after the distribution of the procedure memorandum. Instal- lation is performed online by the network administrator. Computer systems at the LAB and biorepository may also be upgraded to support modifications to the protocol. Once the systems are installed, data are transferred on a routine basis from the SCs to a central trial database, between the SCs and the LAB,


from the SCs to the biorepository, and from the biorepository and the LAB to the central trial database.

Issue Resolution and Protocol Modification

The NCI and the CC identify issues through questions received from the SCs, data reviews, site visits, steering committee meetings, MAP meetings, or during conference calls. The CC documents all issues and proposed resolutions for discussion with NCI staff at a biweekly meeting. If requested by the NCI, the CC presents an issue to the appropriate subcommittee or working group for resolution. In some instances the issue may be presented to the entire steering committee for resolution. Upon final resolution, the CC documents the issue in a numbered, dated entry to the PLCO decision log, which is distributed to all trial collaborators by mail. The annual revision of the MOOP includes any new entries into the PLCO decision log since the previous revision.

If an issue necessitates protocol modification, development and implementation proceed as outlined above, with changes to data collection forms, documentation, and systems, as needed. Some protocol modifications are simple changes to manual procedures and can be implemented via a documentation update and a training telephone call. For example, when the definition of a positive screen for the flexible sigmoidoscopy examination was expanded to include visual or palpable evidence of a mucosal abnormality, the change was documented in the decision log, and the specifications for completion of the examination form were modified and sent to all SC coordinators. The CC coordinator followed this modification with telephone calls to all coordinators to review the change.

Some protocol modifications require extensive revisions to procedures and systems, in-person training, and ongoing monitoring. For example, when the blood collection protocol was expanded to include additional samples for etiologic studies, the op-scan data collection form was modified and reprinted, and the specifications were revised. The chapter of the MOOP devoted to blood collection and processing was also revised. The computer systems, both the SMS and the DEES, were modified, and the system changes were included in the upgrade documentation sent to the SCs. The SC coordinators participated in a training session at Westat on the new blood protocol, and the CC coordinator and user support staff continue to provide support to the SCs for this change. CC staff revised the existing monitoring reports, and new reports were developed to monitor this effort.

Some issues necessitate system enhancements to simplify SC management tasks but do not involve protocol changes. For example, several screens in the SMS were improved to reduce the number of keystrokes required to enter the data. For changes to the computer systems, both modifications and enhancements, the CC has implemented a procedure for the SC and CC staff to use in requesting changes to the SMS and the DEES. The requestor submits a software change request form, and CC systems staff log the request. Each quarter, as part of the planning process for the systems upgrade, the CC distributes the log of requests to the SC data managers for prioritization of requests. With this method, all SC data managers have the opportunity to review a request to determine if it will help or hinder their operations and to prioritize it for a


specific upgrade. The CC collects the SC priorities, then works with the NCI to finalize the requests that will be implemented in each system upgrade.

Routine review of SC data by CC data preparation and project staff may identify anomalies that can be the result of a data collection error or a systems error. These problems are routinely documented, then investigated by CC testing staff. In addition, if the CC or SC staff identify a needed correction to SC data, the SC staff are given specific instructions to make the corrections, or the CC staff log on to the SC's local area network to make the changes to the SC database online.

QUALITY ASSURANCE

The main objectives of the PLCO quality assurance program are the collection of consistently high-quality data across SCs and over time. Quality assurance practices have been implemented at all levels, trialwide. Quality assurance as it pertains to governance, communications, standardization and monitoring protocol adherence and procedures, training and certification of examiners, data editing and analysis, and reporting to the NCI were described above. Computer systems security [3] and quality assurance for screening examinations [4] appear elsewhere in this supplement. Quality assurance as it relates to data is discussed below.

Data Editing Data editing is conducted in several phases. The SCs are expected to manu-

ally review and edit 100% of data collection forms against the form completion specifications to ensure that all items are completed properly and that there are no stray marks or improperly marked bubbles on the op-scan data collection forms. With the initiation of each new or revised data collection form, the CC also manually reviews a sample of SC-completed forms without participant IDs and provides feedback to the SC regarding recording errors. SCs scan manually edited forms into the DEES, and the scanning program identifies errors in key field such as participant ID, project year, visit number, or sample ID. The SC must correct these errors before additional scanning can be performed. The DEES provides multiple reports to monitor scanning progress including listings of forms scanned in a batch, forms requiring verbatim keyed entry, forms with duplicates in the SC system, and summaries of forms scanned by form type. The DEES also provides an edit report that lists data errors and inconsistencies identified from approximately 15,000 edits developed for the 29 optical-mark data collection forms. The SC must attempt to resolve all errors before a form can be finalized. After resolving all DEES-identified data errors or inconsistencies, the SC transfers a subset of data from the DEES to the SMS. During this process, a number of notification reports may be generated. Since DEES data overwrite SMS data, these reports notify the user when key data items (such as examination results) are overwritten. These reports also identify records that are rejected during the DEES to SMS update process, e.g., records with invalid IDs or dates. Finally, the SCs run the SMS intraform edits (errors within one form) and interform edits (errors between forms) at least once per week to identify data inconsistencies and errors in the SMS data. At least


semiannually, without warning, the CC requests copies of DEES and SMS edit reports from each SC to verify that critical editing is completed.

Site Visits and Record Audits

As part of the ongoing quality assurance program, NCI staff and consultants conduct site visits to the CC, the LAB, and the SCs on an annual basis. These annual site visits are required under federal contracting regulations and have been a useful and efficient way to monitor (and adjust when necessary) the progress of the trial. The CC and the NCI carry out the regular, on-site monitoring of the SCs and the LAB. This monitoring includes auditing primary data records and observing trial practices and procedures. The standardized record audit includes a review of the records related to all aspects of the SC's responsibility including recruitment, screening, and follow-up. Auditors review documents related to all major trial endpoints, including eligibility screeners, baseline questionnaires, consent forms, examination forms, medical record abstracts, and biomaterials inventories. All SCs underwent on-site monitoring and random chart audits within 18 months after entry of the first participant and approximately every 12 months thereafter, but may be randomly selected for an audit at any time. The CC randomly selects cases for audit with an emphasis placed on accruals since the previous audit. Complete participant files are audited for eligibility, data entry, missing data, date consistencies, proper refer- ral, and follow-up to ensure that performance standards are being met.

The CC developed a series of standard checklists for record audits and CC observations of trial procedures and processes to ensure that all elements are reviewed. Auditors document the accuracy, completeness, consistency, and overall quality of record keeping and note any problems or discrepancies found such as protocol violations, record errors, or missing information on the audit checklist. Auditors sign the checklist and include it as an attachment to the site visit report. Auditors verify adherence to established trial procedures through observation of recruitment, randomization, screening, forms processing, results reporting, obtaining and abstracting medical records, and obtaining death certificates. CC auditors also observe SC practice of procedures for maintaining data security in the areas of personnel control, physical object control, disaster preparedness, control of data access, and telecommunications and hardware.

The CC auditors conduct an exit interview with the responsible investigator and staff to provide immediate feedback and submit a written report to the NCI project officer. This report is attached to the NCI project officer's report. The NCI contract officer summarizes the findings from the site visit in writing to the appropriate contact at the SC. The NCI follows up all protocol violations, record errors, and instances of missing data or forms, and the monitored institution provides thorough and complete explanations and evidence of corrections of both the cited problems and any SC practices that may have contributed to the problems.

Laboratory Quality Control

The central LAB maintains strict quality control procedures for PSA and CA125 assays. Licensed technologists perform assays following the manufactur- er's specifications. Each test run includes both a reagent kit control and a Bio- Rad tumor marker bilevel control, which are commercially prepared specifically


for use in tumor marker testing. Samples are run in duplicate. Assays are repeated for any pair of results with out-of-range values or with more than a 15% difference between duplicates. The average of the second pair of results are calculated and returned to the SC. Quality control data for assay runs are stored, and control statistics are reported to the NCI each quarter.

The LAB staff report problems with receipt and labeling of serum sample vials to the SC coordinator and the CC for same-day resolution. Once each quarter, the CC produces a reconciliation report comparing PSA and CA125 results files from the LAB with the corresponding SC files to make sure that no results have been overwritten or deleted. The CC works with SC and LAB staff to resolve discrepancies.

Data Monitoring The CC generates monthly monitoring reports of overall trial and individual

SC progress in recruitment, enrollment, screening, and abstraction of medical records for information on the diagnostic evaluation process, cancer diagnosis, and cancer treatment. The CC also generates routine quality assurance reports to measure adherence to protocol requirements (e.g., completion of screening examinations within the required time frame or distribution of a written report of results to the participant and his or her physician within 3 weeks of the performance of the screening examination) and to assess examiner performance (e.g., ovary visualization and palpation rates or percent of screening examinations coded as inadequate). These monitoring and quality assurance reports are distributed quarterly to CC, NCI, and SC staff for review.

The SCs generate numerous production and progress reports through the SMS to monitor and evaluate their individual progress for all trial activities. For example, several recruitment and enrollment reports allow SCs to monitor their progress in enrolling participants based on gender, race, and age group and to track reasons given for nonparticipation. Several reports provide information on work completed (e.g., summaries of screening examinations performed or questionnaires completed) and pending (e.g., lists of outstanding screening examinations, annual questionnaires, or medical record abstracts).

The CC monitors identification and resolution of specific protocol violations and data inconsistencies. Protocol violations include errors such as randomizing an individual twice, randomizing an individual with incorrect information, screening a control participant, or reporting incorrect screening examination results to participants and physicians. SCs are required to report all protocol violations to the CC and the project officer. The standard protocol violation report describes the error, the circumstances, the method of discovery, recom- mended corrections or actions, and steps that will be taken to prevent the error in the future. Upon receipt of the report, the CC enters the information into a database and tracks investigation and resolution of any data errors that may or may not require CC-initiated changes to the SC database.

The CC also documents specific data inconsistencies identified from routine quality assurance reports on data investigation forms (DIFs). DIFs are sent semiannually to SC coordinators for resolution. SC staff are required to investigate the problem, clean the data as necessary, document the process on the


DIF, and return the completed form to the CC. The CC maintains a database to track the DIF process.

S U M M A R Y

The PLCO is an unusually large and complex trial. This places a heavy burden on the organizational structure and communications network to support strong leadership and promote active collaboration across organizational units. Quality assurance activities must be rigorously conducted to assure the validity of centrally archived data. A standardized computer system and trialwide training are essential. In the PLCO, one of the most significant challenges is adapting the protocol to the unique environments of the ten SCs. The NCI and the CC must allow each SC enough room to be successful in its own environment while maintaining high-quality and standardized data collection. High volume activities (i.e., the successful recruitment of 148,000 participants and the screening of 74,000 participants at six intervals) and long-term follow-up for complete cancer and vital status ascertainment over a period of up to 22 years are major challenges of the trial.

REFERENCES

1. Prorok PC, Andriole GL, Bresalier RS, et al. Design of the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Control Clin Trials 2000;21:273S--309S.

2. Hayes RB, Reding D, Kopp W, et al. Etiologic and early marker studies in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Control Clin Trials 2000;21:349S-355S.

3. Hasson MA, Fagerstrom RM, Kahane DC, et al. Design and evolution of the data management systems in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Control Clin Trials 2000;21:329S-348S.

4. Weissfeld JL, Fagerstrom RM, O'Brien B. Quality control of cancer screening examination procedures in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screen- ing Trial. Control Clin Trials 2000;21:390S-399S.

Documents

Coordination and management of a large multicenter screening trial: The prostate, lung, colorectal and ovarian (PLCO) cancer screening trial