Project Closeout Report for the Muon g-2 Project · 2018. 7. 12. · This is the Project Closeout Report for the Muon g-2 Project, a DOE project, which will enable the Muon g-2 Experiment

1

Project Closeout Report

for the

Muon g-2 Project

Project # SC-30YC

at

Fermi National Accelerator Laboratory

Office of High Energy Physics

Office of Science

U.S. Department of Energy

Date Approved:

April 2017

Month/Year

3

Project Closeout Report for the Muon g-2 Project

at the Fermi National Accelerator Laboratory

1. EXECUTIVE SUMMARY ........................................................................................................................ 4 2. INTRODUCTION ...................................................................................................................................... 5 3. ACQUISITION APPROACH ................................................................................................................... 5 4. PROJECT ORGANIZATION .................................................................................................................. 6 5. PROJECT BASELINE .............................................................................................................................. 7 6. CLOSEOUT STATUS ............................................................................................................................. 16 7. LESSONS LEARNED ............................................................................................................................. 16 8. PHOTOS ................................................................................................................................................... 18 9. PROJECT DOCUMENT ARCHIVES AND LOCATIONS ................................................................ 28 10. APPENDICES .......................................................................................................................................... 29

APPENDICES

A. Detailed Technical Performance Achieved B. Major External Reviews C. Project Risk Registry D. Detailed Technical Lessons Learned from CD-0 through CD-3 E. Detailed Technical Lessons Learned from CD-3 through CD-4 F. Transition to Operations

4

1. EXECUTIVE SUMMARY

The Muon g-2 Project was a DOE project to fabricate an experiment that seeks to improve the

measurement of the muon anomalous magnet moment, which is sensitive to new physical

interactions through interactions with virtual particles emerging from vacuum interactions. In

particular, Muon g-2 provides unique sensitivity to beyond Standard Model theories such as

supersymmetry and dark matter by measuring indirectly their impact on the muon spin

precession in a magnetic field. The project repurposed a storage ring from the E821 experiment

at Brookhaven National Laboratory (BNL) with the transfer of the BNL storage ring

accomplished in fiscal year 2013. Upgrades to the ring injection subsystems, detectors,

electronics, data acquisition, and field monitoring equipment were needed to utilize the high

intensity proton beam available at Fermilab, and control systematics at a level commensurate

with the 21-fold increase in statistics compared to prior measurements. A number of

modifications of the former Fermilab anti-proton source were required to deliver a high purity,

intense beam of muons to the storage ring. Some of the critical detector systems were provided,

in part, by in-kind contributions from non-DOE sources including National Science Foundation

(NSF) and international contributions, particularly from the UK and Italy. The Muon g-2

experiment offers a strategic opportunity to search for new physics that may be inaccessible to

the Large Hadron Collider (LHC). The project was successfully completed ahead of schedule

and under budget.

The Muon g-2 Project was completed within its baselines. The project has fully met its

Objective KPPs 18 months ahead of schedule and with $195 thousand of contingency remaining.

Construction of the Muon g-2 detector components and subsystems was completed in November

2017.

In addition, the Project, using contingency, was able to fund development and a portion of the

fabrication of a new inflector to potentially increase muon injection efficiency and provide for a

spare to this critical device. The new inflector was the highest priority on the optional scope list.

As contingency funds became available for optional scope (based on the retirement of project

risks), the new inflector was funded incrementally. The superconducting shielding material used

for fabricating the existing inflector in Japan, approximately 25 years ago, was so specialized

that the project could not locate anyone who knew how to re-make it. The project was able to

obtain the last remnants of the superconducting shielding material and superconductor cable of

the type used on the existing inflector. Since all of the requirements for CD-4 had been achieved

by November 2017, DOE-OHEP wanted the work on the new inflector to be transitioned to

operations at that time for greater operational efficiency. All but a relatively small amount of the

work needed for the new inflector was able to be accomplished on the project. Since the end of

the equipment fabrication project, Fermilab has resume the final winding of the coils and plans to

complete the assembly of the cold mass.

5

2. INTRODUCTION

This is the Project Closeout Report for the Muon g-2 Project, a DOE project, which will enable

the Muon g-2 Experiment to take place at Fermilab.

Muon g-2 will use Fermilab's powerful accelerators to explore the interactions of short-lived

particles known as muons with a strong magnetic field in "empty" space. Scientists know that

even in a vacuum, space is never empty. Instead, it is filled with an invisible sea of virtual

particles that, in accordance with the laws of quantum physics, pop in and out of existence for

incredibly short moments of time. Scientists can test the presence and nature of these virtual

particles by observing the rate at which the magnetic moment of a fundamental particle rotates

around the magnetic field, a phenomenon known as spin precession. This spin precession has

been measured to even higher precision with electrons, but the heavier mass of the muon makes

it uniquely sensitive to the presence of higher mass particle that appear in quantum field loops

around the muon.

The Muon g-2 experimenters will examine the precession of muons that are subjected to a

magnetic field. The main goal is to test the Standard Model's predictions of this value by

measuring the precession rate experimentally to a precision of 0.14 parts per million, which

requires 2x1011 muons to be stored and observed in the experiment. If there is an inconsistency,

it could indicate the Standard Model is incomplete and in need of revision.

3. ACQUISITION APPROACH

DOE acquired the Muon g-2 Project through the operating contractor of Fermilab, Fermi

Research Alliance (FRA). Construction of the conventional facility to house the Muon g-2

detector was carried out as an independent General Plant Project (GPP) as a multi-use facility.

The building is called the MC-1 Building.

The technical design and specifications for Muon g-2 Project were developed by the Muon g-2

collaboration, under the project management at Fermilab. Procurements followed the Fermilab

rules for contract awards and competitive bids.

In addition, the detector systems include in-kind contributions of scope from NSF funded

university groups, valued nominally at $3.2 million, for the construction of 24 PbF2

calorimeters, along with their associated electronics, and the data acquisition for the experiment.

This contribution from NSF is not part of the DOE scope or Total Project Cost (TPC).

Some components were delivered as in-kind contributions from international collaborating

institutions—notably, a consortium of UK universities primarily funded by the STFC provided

$2.25 million in funding for the construction of the straw tracker system, and a consortium of

Italian groups funded by INFN provided $0.5 million in materials and the effort needed to

construct a sophisticated laser monitoring system for the calorimeters. In the original Project

Execution Plan, these contributions were estimated as a $1 million contribution.

6

DOE DES Project Director

Fermi Site Office ort

Center


Office of Science (Acquisition Executive)


BTeV Program Manager

Muon g-2

Federal Project Director

Fermi Site Office Integrated Support

Center


Office of Science

(Acquisition Executive)


Muon g-2 Program Manager

Muon g-2 Project Manager

Project Management

Group

Muon g-2

Technical Board

Muon g-2

Collaboration

Finally, a DOE Early Career award for Brendan Casey in the amount of $2.5M over 5 years was

also used primarily to advance the straw tracker hardware and analysis and is not considered part

of the TPC associated with the Muon g-2 Project. In the original Project Execution Plan, it was

estimated that the Early Career Award would provide a $900k savings to the project construction

costs.

Commissioning with beam was not part of the Muon g-2 Project.

4. PROJECT ORGANIZATION

Figure 4.1 shows the management chain for the Muon g-2 Project, from the DOE Office of

Science to the Project Manager. The Project Management Group, which met monthly, was a

mechanism used by Fermilab to monitor the progress and address the needs of the project in a

timely fashion. Figure 4.2 shows the organization chart for the project itself. The project was

organized into five work breakdown structure (WBS) areas, as shown.

Figure 4.1: Project Management Chain

7

5. PROJECT BASELINE

Scope Baseline

The scope of the Muon g-2 Project was to deliver the existing superconducting storage ring

magnet from BNL to Fermilab, and to provide the accelerator modifications, injection system

upgrades, improved field monitoring, and detector systems not covered by in-kind contributions.

A set of General Plant Projects (GPPs) and Accelerator Improvement Projects (AIPs), known as

the Muon Campus GPPs and AIPs, were set up to provide some of the multiuse infrastructure

needed for the Muon g-2 and Mu2e Experiments. Each of these projects had a defined scope

and was managed under formal change control. Interface milestones were used to coordinate

key events between the Muon Campus GPPs and AIPs and the Muon g-2 Project and Mu2e

Project. Final ring magnet shimming after the vacuum chambers were installed and beam

commissioning was not included in the scope of the project as these are areas that will continue

to improve throughout the lifetime of the experiment as it operates over the next few years.

Details of the technical performance of these project elements are given in Appendix A and the

documents referenced within.

The Project met the scope baseline when the apparatus was demonstrated to be functioning by

achieving the Key Performance Parameters (KPP) listed below in Table 5.1. The KPPs

demonstrate performance capability of the system. The Threshold KPPs were the minimum

technical requirements for successful project completion. Objective KPPs were to be

accomplished by the project if sufficient project funds was available and if external factors

beyond the control of the project did not prevent it.

WBS 1.0

Muon g-2 Project

WBS 1.1

Project Management

WBS 1.3

Ring

WBS 1.2

Accelerator

WBS 1.4

Detectors

WBS 1.5

BNL Equipment

Transfer

Figure 4.2 Project Work Breakdown Structure

8

Table 5.1 Key Performance Parameter

Threshold Performance Objective Performance Results

Accelerator All accelerator components of Beamlines M2, M3, M4

and M5 are ready for

installation, dependent on

external factors. (Installation

of Beamline M2 or M3

components requires

accelerator shutdown for

personnel access, impacting

NOvA operations.

Installation of Beamline M4

or M5 components requires

the Beamline Enclosure

GPP.)

All other accelerator

components are installed and

ready for commissioning

with beam at nominal

voltages and currents,

represented by:

• Target Station

Momentum Selection

Magnet (PMAG) Pulsed

Power Supply achieving

15.3 kA peak current;

• Target Station Lithium

Lens Pulsed Power

Supply achieving 19 kA

peak current;

• Delivery Ring Extraction

Lambertson Magnet and

Power Supply achieving

1.13 Tesla-m integrated

field strength; and

• Delivery Ring Extraction

C-Magnet and Power

Supply achieving 1.68

Tesla-m integrated field

strength.

All accelerator components

are installed and ready for

commissioning with beam at

nominal voltages and

currents.

Objective KPP

achieved. (See

Appendix A for

technical details.)

Storage Ring Storage ring yoke pieces, pole pieces, and superconducting

coils have been cooled and

powered to full 1.45T field.

Objective KPP

achieved. (See

Appendix A for

technical details.)

9

Storage Ring

Subsystems

Storage ring subsystems,

including the electrostatic

quadrupoles, pulsed

electromagnetic kickers, and

inflector, are ready to install.



quadrupoles, pulsed


inflector, are installed and

ready for commissioning

with beam at nominal

voltages and currents.

Objective KPP

achieved. (See

Appendix A for

technical details.)

NMR

Systems

Nuclear magnetic resonance

(NMR) systems for

monitoring magnetic field,

including fixed probes,

plunging probes, and NMR

trolley, are ready to install.


(NMR) systems for

monitoring magnetic field,

including fixed probes,

plunging probes, and NMR

trolley, are installed and ready

for commissioning with beam

at nominal values.

Objective KPP

achieved. (See

Appendix A for

technical details)

Detector All calorimeters and trackers that have been received by

the time that the accelerator

system is ready to be

commissioned with beam

have been installed.

All calorimeters and trackers


Objective KPP

achieved. (See

Appendix A for

technical details)

In addition, the Project was able to use contingency funds for the development and partial

fabrication of a new inflector that could potentially increase the muon injection efficiency and

serve as a spare to this critical device.

Cost Baseline

DOE funding for the Muon g-2 Project came through Fermilab. The sum of the Fermilab

funding ($46.4 million) constitutes the project’s TPC. The Muon g-2 Project was finished with

an estimated final cost of $46.20 million and with $195 thousand remaining in available

contingency.

Table 5.2 shows the original CD-2 baseline. The major area with large contingency usage was

for beamline magnets and specialized power supplies. The contingency was used primarily for

additional engineering and designer resources needed to complete the project. Of the total of

$7.1 million of contingency used, almost all was on labor at Fermilab. As discussed in the Scope

Baseline section above, the Project was able to save enough contingency to fund the

development of a new inflector and a portion of the fabrication. The project contingency usage

is shown in Table 5.3, Table 5.4 and Figure 5.1.

10

Table 5.2 DOE CD-2 Cost Estimate by WBS Element ($ in Millions)

WBS

Element Item

DOE

OPC

DOE

TEC

DOE

TPC

1.1 Project Management 2.6 1.5 4.1

1.2 Accelerator 4.9 12.9 17.8

1.3 Ring 4.4 7.8 12.3

1.4 Detectors 0.1 0.4 0.5

1.5 BNL Equipment Transfer 4.2 0.0 4.2

Subtotal of above 16.2 22.6 38.9

DOE Contingency† 7.5

DOE Total 46.40

Table 5.3 Project Contingency Usage

Date or End

of Fiscal

Year

% Project

Complete TPC ($M)

Actual

Costs to

Date ($M)

Contingency

Remaining

($M)

Estimate to

Complete

($M)

FY2015 57% 46.4 23.36 6.95 11.33

FY2016 87% 46.4 37.65 3.08 5.67

Sept 2017 99.6 46.4 46.12 0.13 0.15

Dec 2017 100% 46.4 46.20 0.19 0

*$0.19 million remained in contingency at the end of the project.

Figure 5.1 Project Contingency History

11

Table 5.4 Contingency Use in Detail

WBS

Element Item

CD-2 Base

Cost

Estimated

Final

Cost*

Cont.

used*

1.1 Project Management 4.1 3.67 -0.43

1.2 Accelerator 17.8 21.76 3.96

1.3 Ring 12.3 15.95 3.65

1.4 Detectors 0.5 0.64 0.14

1.5 BNL Equipment Transfer 4.2 4.18 -0.02

DOE Total at Completion 38.9 46.21 7.31

Unused Contingency 7.5 0.19

DOE TPC 46.40 46.40

*Numbers based on end of December 2017 EVMS data

Schedule Baseline

The Muon g-2 Project was baselined with a CD-4 milestone date of June 2019. All the project

deliverables were provided by the end of November 2017. The CD-4 Review was conducted on

November 14, 2017.

The project was baselined with twenty-four months of schedule contingency. CD-4 was

approved on January 16, 2018, leaving 17.5 months of schedule float remaining. The major

elements of the project which took longer than originally scheduled included the assembly and

powering of the magnet prior to CD-2, the removal of the AP0 beam absorber, the design and

upgrades of existing pulsed power supplies in the target station, the construction of conventional

magnets for the beamline, the assembly and alignment of trolley rails and quadrupole plates

within the storage ring vacuum, and the in-kind production of the straw trackers.

In general, the completion of the Ring and Detector L2 branches stretched by approximately six

months from the original plan. Within the Accelerator L2, the extraction and replacement of the

AP0 beam absorber also took six months longer than planned. In the end, a number of the

subsystems all came together just in time for the fiscal year 2017 engineering run that started

early in June 2017. Production of the beamlines, which was the critical path and thought to be a

major schedule risk at CD-3, did not take longer than scheduled. Table 5.5, Table 5.6 and Table

5.7 summarize the major milestones and dates.

12

Table 5.5 Critical Decisions Milestones

Milestone Description Preliminary

Baseline Date

0.0 CD-0: Approve Mission Need Sep. 18, 2012 (A)

0.1 CD-1: Approve Alternative Selection and Cost Range Dec. 19, 2013 (A)

0.2 CD-2: Approve Performance Baseline Aug. 20, 2015 (A)

0.3 CD-3: Approve Start of Construction Aug. 20, 2015 (A)

0.4 CD-4: Approve Project Completion Jan. 16, 2018 (A)

Table 5.6 Level 1 Milestones

Milestone Description Baseline Date

1.1 KPP—Storage Ring Threshold KPP Achieved Aug 30, 2016

1.2 KPP—Storage Ring Subsystems Threshold KPP Achieved Aug 22, 2017

1.3 KPP—NMR Systems Threshold KPP Achieved Oct 4, 2017

1.4 KPP—Accelerator Threshold KPP Achieved Feb 27, 2018

1.5 KPP—Detector Threshold KPP Achieved Mar 27, 2018

13

TABLE 5.7 LEVEL 1 AND 2 MILESTONES

L1 Milestones:

L2 Milestones:

14

Work Breakdown Structure

The WBS for the Muon g-2 Project to level 2 is defined in Table 5.8 below.

Table 5.8 Muon g-2 Work Breakdown Structure Dictionary

WBS Title Description

1.0 Muon g-2 Project

1.1 Project Management Project Management support

1.2 Accelerator Accelerator modifications

1.3 Ring Storage ring magnet and associated systems

1.4 Detectors Calorimeters, trackers, auxiliary detectors, electronics, DAQ

1.5 BNL Equipment Transfer Transport of storage ring magnet and other reusable equipment

Funding Profile

The funding tables below present the funding profile planned at CD-2 and the actual funding

profile.

Table 5.9 CD-2 Planned Funding

FY 2012

FY 2013

FY 2014

FY 2015

FY 2016

FY 2017 Total

OPC-Other 0.601 2.742 3.2 6.543

OPC-Design 3.108 5.2 4 12.308

TEC-MIE 2.0 9 10.2 6.349 27.549

Total 0.601 5.850 10.4 13 10.2 6.349 46.400

Table 5.10 Actual Funding

FY 2012

FY 2013

FY 2014

FY 2015

FY 2016

FY 2017 Total

OPC-Other 0.601 2.742 3.2 6.543

OPC-Design 3.108 5.2 4 12.308

TEC-MIE 2.0 9 10.2 6.349 27.549

Total 0.601 5.850 10.4 13 10.2 6.349 46.400

15

Staffing Profile

The figure below shows the actual staffing profile for the Muon g-2 Project in hours per fiscal

year.

Environmental Requirements/Permits

The NEPA Categorical Exclusion (10CFR1021, Subpart B, Appendix B1.30, B1.31, and B3.10)

for the Muon g-2 Project was approved on December 20, 2012. No environmental issues were

encountered during this project.

Safety Record

Standard Fermilab safety procedures and policies were followed throughout the project,

including the use of Integrated Safety Management principles, clear Job Hazard Analysis

standards, and standing safety committees which conducted Operational Readiness Reviews of

custom equipment and frequent safety walk-throughs/inspections of the work areas.

Throughout the course of the Muon g-2 Project construction, there were a total of three notable

safety incidents. The first occurred when one of 24 pieces of steel (26T each) that form the

storage ring return yoke was dropped due to a failure in rigging equipment. The root cause was

traced to a new commercial softener that was used to protect the straps and failed to protect the

16

sling from being cut by the sharp 90 degree edge of the steel. The softener was brought to the

lift by a technician working for the project and had not been reviewed by either the rigging

supervisor or project engineering. The sling failed after one end of the steel had lifted several

inches from the deck of the flatbed truck. The load fell back onto the truck deck without any

damage to property or injuries to personnel. A stop work was issued; a root cause analysis was

conducted; procedures were reviewed; and the project took the opportunity to provide additional

training to the entire project team to emphasize safety ahead of schedule.

The second notable accident occurred as a technician was taking down a temporary barrier that

was used to discourage access to the storage ring during the time it was stored outside while

work on the building finished. The worker stepped in a hole and twisted an ankle resulting in

some time lost from work.

The final notable safety incident occurred near the end of the project as engineers and

technicians were testing the pulsed Li lens power supply used to focus the primary proton beam

onto the target. During the testing of one capacitor bank in the power supply, a second capacitor

bank was inadvertently charged due to a wire that had been previously connected and then

overlooked. When the tests concluded, the technician noticed a wire on the second capacitor

bank still needed to be reconnected and touched the wire to the charged bank anode. This

resulted in an accidental discharge of the stored energy, which vaporized the wire. The

technician suffered a small laceration on his finger due to quickly retracting his hand from the

cabinet. Proper PPE likely saved more serious injury as a portion of the vaporized wire

splattered onto the technician’s safety glasses. A stop work was issued; a root cause analysis

was completed that found some violations of best practice procedures; the procedures were

reviewed; and special training was provided for the department responsible for this type of work

at the laboratory.

6. CLOSEOUT STATUS

The financial closeout of the project will be completed following the CD-4 ESSAB.

Operations and Commissioning: Planning and Readiness

An accelerator readiness review was held on March 14, 2017, and beamline commissioning

began in April 2017. An engineering run served as an integration test of all the subsystems from

June 5, 2017 through July 7, 2017. Commissioning was resumed in November 2017, following

the completion of the summer accelerator shutdown. Further details are contained in Appendix

F: Transition to Operations.

7. LESSONS LEARNED

Some lessons learned are briefly presented below. Details for each of these lessons learned are

presented in Appendix E: Detailed Technical Lessons Learned from CD-3 through CD-4.

Management

17

Custom project management tools made work more efficient and effective.

Management team composition led to coherent, effective team.

Transition to operations of the Cryo Plant AIP should have been better planned.

Coordinated management of Muon Campus and Muon g-2 aided in accomplishing projects efficiently and effectively.

Funding

Project and scientific collaboration worked together effectively to achieve project goals.

Reuse of the anti-proton complex and other Tevatron equipment saved time and money.

Integrated approach to coordinating effort with detector deliverables funded through an NSF MRI were successful.

Cost

Cost overrun on conventional magnet construction were due to inefficiencies at lab.

A number of factors lead to cost overrun on Li lens and PMAG power supplies.

Estimation and planning of alignment and metrology effort lead to more efficient implementation.

Schedule

Custom project management tools made work more efficient and effective.

Tracking uncosted scientific effort ensured accurate progress measurement.

Technical

Costs overran on target station beam absorber.

Safety and Quality

Need to pay close attention to prevent overworking personnel when schedules become tight.

18

8. PHOTOS

Photo 2 Removal of Superconducting Coils from Building at BNL. Red Steel is Support

Structure for the Coils, 6/22/2013

Photo 1 Muon g-2 Experiment at BNL, 12/22/2005

19

Photo 3 Rings Traveling Across the BNL Site, 6/22/2013

Photo 5 Barge Traveling on the Mississippi River

by the Gateway Arch in St. Louis, 7/19/2013

Photo 4 GPS Tracking of Muon g-2 Rings on

Barge, 6/22-7/26/2013

20

Photo 6 Rings Going Under an Overpass in Illinois, 7/24/2013

Photo 7 Rings Traveling on I-88 in Illinois, 7/25/2013

21

Photo 8 Rings Traveling across the Fermilab Site, 7/26/2013

Photo 9 Rings on Display at a Public Event at Fermilab, 7/26/2013

22

Photo 10 Rings Moving into Experiment Hall,

7/30/2014 Photo 11 Rings Moving into Experiment Hall, 7/30/2014

Photo 12 Rings Lowered onto Yoke Steel, 8/2014 Photo 13 Storage Ring Assembled for Initial

Powering , 6/2015

Photo 14 Computer Display Showing that Storage

Ring Achieved Operating Temperature , 6/2015

Photo 15 Display Showing that Storage Ring

Achieved Full 1.45 T Field , 10/2015

23

Photo 16 Beamline Instrumentation Testing,

2/13/2017

Photo 17 Removal of Old Beam

Absorber and Module, 11/16/2016

Photo 18 Delivery Ring D-30 Straight Section, 3/1/2017

24

Photo 20 Kicker Plates Installed in Vacuum

Chamber, 3/2017

Photo 22 Trolley Drive Installation 4/2017 Photo 23 Fiber Harp (Beam Profile Monitor),

5/2017

Photo 21 Calorimeter Installation, 5/2017

Photo 19 Kicker Installation, 4/2017

25

Photo 24 Fiber harp monitor (7 horizontal fiber

optic cables seen in the center of the

photograph) installed in the Muon g-2 storage

ring. The monitor is in the inserted

measurement position. 2/2017

Photo 25 Tracker modules installed in the Muon g-2 storage

ring. 2/2017

26

Photo 31 Last Vacuum Chamber Installed, 1/2017 Photo 30 Remote Operations in ROC-West Tested,

10/2016

Photo 27 Quad Plates in Vacuum Chamber, 8/2016

Photo 26 High Voltage Feedthrough, 4/2017

Photo 29 Tracker Installation 5/2017 Photo 28 Calorimeter sled

2/2017

27

Photo 32 Muon g-2 Experiment in MC-1 Building at Fermilab 8/28/2017

28

9. PROJECT DOCUMENT ARCHIVES AND LOCATIONS

Project documents are archived by the Muon g-2 Project in a Fermilab-maintained document

database: http://gm2-docdb.fnal.gov; review access is available upon request.

http://gm2-docdb.fnal.gov/

29

10. APPENDICES

Appendix A: Detailed Technical Performance Achieved

In this section the elements required to demonstrate the completion of the KPP for the Muon g-2

Project are described in detail.

Threshold Performance Objective Performance

Accelerator All accelerator components of Beamlines M2,

M3, M4 and M5 are ready for installation,

dependent on external factors. (Installation of

Beamline M2 or M3 components requires

accelerator shutdown for personnel access,

impacting NOvA operations. Installation of

Beamline M4 or M5 components requires the

Beamline Enclosure

GPP.)

All other accelerator components are installed

and ready for commissioning with beam at

nominal voltages and currents, represented by:

• Target Station Momentum Selection Magnet

(PMAG) Pulsed Power Supply achieving

15.3 kA peak current;

• Target Station Lithium Lens Pulsed Power

Supply achieving 19 kA peak current;

• Delivery Ring Extraction Lambertson

Magnet and Power Supply achieving 1.13

Tesla-m

integrated field strength; and

• Delivery Ring Extraction C-Magnet and

Power Supply achieving 1.68 Tesla-m

integrated field strength.

All accelerator components are

installed and ready for

commissioning with beam at

nominal voltages and currents.

Storage Ring Storage ring yoke pieces, pole pieces, and

superconducting coils have been cooled and

powered to full 1.45T field.

Storage Ring

Subsystems

Storage ring subsystems, including the

electrostatic quadrupoles, pulsed

electromagnetic kickers, and inflector, are ready

to install.



quadrupoles, pulsed


inflector, are installed and

ready for commissioning with

30

beam at nominal voltages and

currents.

NMR

Systems

Nuclear magnetic resonance (NMR) systems for

monitoring magnetic field, including fixed

probes, plunging probes, and NMR trolley, are

ready to install.


(NMR) systems for monitoring

magnetic field, including fixed

probes, plunging probes, and

NMR trolley, are installed and

ready for commissioning with

beam at nominal values.

Detector All calorimeters and trackers that have been

received by the time that the accelerator system

is ready to be commissioned with beam have

been installed.

All calorimeters and trackers


Achievement of Accelerator Objective KPP:

All accelerator components are installed and ready for commissioning with beam at nominal

voltages and currents, thus achieving this Objective KPP.

The accelerator installation was completed in the spring of 2017. An Accelerator Readiness

Review was held on March 14 through 16, 2017, and permission to run beam was granted on

April 5, 2017. Although not part of the KPPs, commissioning-quality beam was established to

the g-2 muon storage ring on May 31, 2017. Some details of the technical accomplishments are

presented below.

The target station pulsed power supplies and beam absorber (a.k.a. beam dump) were ready

when needed for beam commissioning. The old beam absorber which had cooling water leaks

was removed. This required more effort than planned due to a broken bolt at the attachment of

the beam absorber to the shielding module, both of which were highly radioactive. A new

module and a new beam absorber were fabricated and installed.

All g-2 beamlines (M1, M2, M3, M4, and M5) were installed, including magnets, vacuum

systems, power supplies, controls, and instrumentation. In the M1 line, magnets used for the

final focus of the beam on the target were rearranged from their 120-GeV configuration to

provide a smaller spot size with 8-GeV beam. The M2 and M3 lines were created from the

former antiproton-source transport lines, with additional quadrupole magnets to better capture

muons from pion decay. The Delivery Ring was reconfigured for injection from the M3 line and

extraction to the M4 line. The new M4 and M5 beamlines were designed and installed to

transport muons from the Delivery Ring to the g-2 muon storage ring.

31

Figure A1 shows the current from the PMAG power supply during the 2017 commissioning

period. The supply was tested at currents above 15.3 kA, but only needs to run at 14.7 kA

Figure A1 also shows the current from the lithium lens power supply during the 2017

commissioning period. Currents above 19 kA have been achieved and provide better focusing of

particles from the target.

Figure A2 shows the Lambertson power supply operating at 1,400 amps which corresponds to an

integrated field of 1.16 T-m. Fermilab’s Technical Division tested the magnet as high as 1.32 T-

m, as shown in Fig. A3 [gm2-doc-8957].

Figure A2 shows the Extraction C-Magnet power supply operating at 1,040 amps which

corresponds to an integrated field of 1.69 T-m. Technical Division tested the magnet as high as

2.09 T-m, as shown in Fig. A4 [gm2-doc-8960].

The Muon g-2 Project re-used or re-purposed a significant amount of technical components and

enclosure including:

1km of tunnel complete with electrical infrastructure, cable trays, cooling water distribution system and safety interlocks;

former Pbar (antiproton) Target Station including target, lithium lens, pulsed momentum-selection magnet, collimation system, target vault, cooling systems, hot work cell, and

tunnel access points with overhead crane coverage -Service buildings with HVAC,

cooling water, controls communication infrastructure, extensive electrical infrastructure,

electronics racks, access roads and parking lots, etc

~250 Pbar magnets (>$20M) and ~30 BNL magnets plus 505m former Pbar Debuncher Ring used in-place as Delivery Ring;

power supplies repurposed where practical (new supplies are smaller and more energy efficient); and

Secondary Emission Monitors and other instrumentation from the Antiproton Source.

32

Figure A2: Power Supply Currents for the Lambertson (red)

and C-Magnet (green) in April and May 2017 when the

Delivery Ring was set up for 8 GeV Beam, which is when the

Highest Fields are Required.

Figure A1: Lithium Lens (green) and PMAG (red) Power

Supply Currents during the 2017 Commissioning Period.

33

Figure A4: Magnetic Field Strength as a Function of Current During C-

Magnet Testing.

Figure A3: Magnetic Field Strength as a Function of

Current during Lambertson Magnet Testing.

34

Achievement of Storage Ring Objective KPP:

The storage ring yoke pieces and pole pieces have been assembled, and superconducting coils

have been cooled and powered to full 1.45T field. This was accomplished in November 2015.

This satisfied the KPP. The magnet was used from November 2015 and for most of 2016 for

magnetic field shimming purposes. There was no separate Objective KPP since the magnet

being fully-functional was required midway through the project in order to complete the rough

shimming of the field before other equipment could be installed in the magnet gap.

The magnet was warmed up in the winter/spring of 2016-2017 to repair a helium leak and for the

removal of three loose charcoal panels in the lower inner cryostat. These flaws required the

magnet be powered down after 12 hours so that the charcoal could be regenerated. Figure A5

shows the removal of loose charcoal from the lower inner cryostat. The loose pieces have poor

thermal contact to 5K, and periodically release helium, which increases heat load on the magnet.

Six more charcoal panels remain in the upper and outer rings, but these are well secured to their

respective mandrels.

After the helium leak repair and charcoal removal, the magnet could be operated for weeks

without requiring powering down for maintenance. The magnet has been in regular operation

ever since. Improvements are continuing to be made, on the operations side to guard against

scenarios where quenches can be caused by power outages.

Figure A5: One of 3 charcoal panels removed from the lower inner

cryostat.

35

Achievement of Storage Ring Subsystems Objective KPP:

The inflector, quadrupoles, and kickers have all met their threshold and objective KPPs, as

described below.

In November 2016, the inflector was cooled to 5K and powered for eight hours at the design

current. The superconducting shield was also shown to be operational. This satisfies the

threshold KPP. Figure A6 shows the inflector ramped to full current.

The inflector was operated during June 2017 for the first engineering run. The new power

supply tripped several times in the initial part of the run. This is understood now to be very

infrequent 100 Amp current glitches from the power supply, which crosses the quench detection

threshold. This was mitigated by increasing the quench detection threshold from 25 mV to 120

mV, the same value used by BNL E821. This satisfies the objective KPP for the inflector.

The kicker system was installed in spring 2017. During the June 2017 engineering run, it could

only operate at about 70 percent of the design strength, and there was severe sparking, which

also affected the quadrupole operation. Since each spark deposits up to 125 joules into the

vacuum, and possibly various electrical insulators, it likely caused damage. The kicker HV

vacuum feedthrough was refabricated using a new design and was installed in September 2017.

Subsequently, it was commissioned to full power (55 KV) and at the required repetition rate, and

was operated with a negligible spark rate. Figure A7 shows the secondary transformer voltages

of the three kickers, reaching the designed kick. This satisfies the objective KPP for the kicker

system.

Figure A6: the inflector current being ramped to 2840 Amps.

36

The electrostatic quadrupole system was fully installed in spring 2017 (Figure A8). During the

June 2017 engineering run, it was operated in the range of 15-20kV, below the designed field

index. Above these voltages, the quads could not operate for long term without sparking. This is

now understood to be due to a combination of factors: the SRV vacuum value did not reach the

designed value and there was sparking in the kicker system, causing the quads to also spark.

Following the kicker HV feedthrough upgrade, and better vacuum, the quadrupoles were

demonstrated to achieve robust operation at 20.5 KV and at 25.4 KV. These are the voltages

expected to give optimal beam dynamics for g-2. The spark rate was sufficiently low, and

estimated to affect the data-collection efficiency at the few percent level. This satisfies the

objective KPP. The table below summarizes the quad spark statistics achieved in September-

October 2017.

Figure A7: Secondary transformer voltages for each of the 3 kickers, indicating operation

at the designed 55 KV.

37

Achievement of NMR Systems Objective KPP:

The fixed probes, plunging probe, and the NMR trolley comprise the systems relevant for the

NMR Systems KPP. All systems were installed and ready for commissioning beam at their

nominal values as of June 20, 2017. The Objective KPP was completed on March 29, 2017

when all systems were ready for installation in the ring. Full operational readiness clearance

approval for remote operation of these systems was subsequently granted after installation for:

the fixed probes on April 21, 2017; trolley on April 26, 2017; and the plunging probe on

September 5, 2017.

The fixed probe systems and the trolley were used throughout the June 2017 engineering run to

monitor and tune the magnetic field. While all systems were ready to install in late March 2017,

the plunging probe installation was deferred until a natural break point in the engineering run in

order to maximize the efficacy of the run and allow for independent testing in the storage ring.

All systems have operated well since meeting the KPP.

The fixed probes are embedded in the upper and lower surfaces of the vacuum chambers, and

their installation was required prior to the installation of the SRV chambers into the magnet gap.

Figure A9 shows a close-up of the probes installed in the vacuum chamber (left), and a view of a

vacuum chamber ready to install in the ring (right). The fixed probe system was fully

commissioned prior to the engineering run, and was used to monitor and stabilize the magnetic

field during that period. Figure A10 shows the Data Quality Monitor displaying information

Figure A8: Electrostatic Quadrupole Spark Rate measured during Sept-October 2017.

38

from a subset of the fixed probes during the run (left) as well as the stabilization of the magnetic

field based on these probes (right).

The trolley was installed in the ring in April 2017. The trolley can be seen through a storage ring

vacuum port during a transit around the ring in Figure A11 (left). Full maps of the magnetic

field were also produced based on this data, and were utilized to further shim the magnet during

the summer shutdown. Figure A12 demonstrates the azimuthally averaged magnet field, and the

average field vs. azimuth after the shutdown shimming period.

The plunging probe was installed during the June 2017 run. The successful measurement of the

field at nominal value in the ring demonstrated the objective KPP on June 20, 2017. Figure A11

(right) shows a view of the plunging probe from its vacuum port, while Figure A13 shows a

readout of the signal from this probe.

Figure A9: (Left) NMR Probes embedded in the surface of a vacuum chamber

and (Right) the vacuum chamber ready to install in the ring.

Figure A10: (Left) Data Quality Monitor shows the configuration of fixed probes on

Yoke C, as well as snapshots of signals from those probes. (Right) shows the ability

to stabilize the magnetic field by providing feedback to the main magnet power

supply.

39

Figure A11: (Left) Installed trolley visible through a vacuum

port during transit. (Right) The cylindrical plunging probe

with a copper holder is visible in the background. In the

foreground the 3D translation stage is visible through its port

on the storage ring vacuum, installed in the ring.

40

Achievement of Detector Objective KPP:

All calorimeters and trackers have been installed, thus achieving the Objective KPP for the

Detectors.

This KPP was structured such that the Project could still achieve the Threshold KPP even if

external partners were late in delivering the detectors. The KPP was structured this way because

the calorimeters were funded by a separate NSF MRI, while the trackers were funded through a

combination of Early Career grant and a large In-Kind Contribution from the STFC. In the end,

the partnerships proved reliable, and the equipment from separate funding source were delivered

in time to meet the project installation schedule.

The calorimeters were all delivered in time for the fiscal year 2017 engineering run. The Figure

A14 below shows the data acquired in each of the 24 calorimeters during the fiscal year 2017

engineering run that ended on July 7, 2017.

Figure A12: (Left) The magnetic field deviations from nominal, averaged over all ring

azimuth. The full range of +/-2 part-per-million (ppm) demonstrates excellent

uniformity, and the individual contributions to the moments in the table are all well

under 1 ppm. (Right) The average field at each point around the ring. This achieved

uniformity is 3-4x the uniformity achieved at BNL.

Figure A13: The free induction decay signal from the plunging probe, measuring

in the ring on June 20, 2017.

41

The first tracker station with 8 modules was also installed during the engineering run and Figure

A11 shows an example of how an event passing through the tracker modules appeared in the

FY17 engineering run.

The final 8 modules required for the second tracking station were delivered by the end of

October 2017 and installed just before the CD-4 review.

Figure A15: Straw hits as a particle passes through the 8 modules

of the 1st tracker during the FY17 engineering run.

Figure A14: Snapshots for each of the 24 calorimeters showing the energy spectrum of

decay electrons measured in each detector during the FY17 engineering run.

42

Appendix B: Major External Reviews

DOE CD-4 Review November 14, 2017

DOE Status Review April 6, 2016

DOE CD-2/3 Follow-up Review June 25 – 26, 2015

DOE CD-2/3 Review July 29 – 31, 2014

Director's CD-2/3 Review & IDR June 17 – 19, 2014

DOE CD-1 Review September 17 – 18, 2013

Director's CD-1 Review July 23 – 25, 2013

Independent Design Review June 5 – 7, 2013

Ring Transport Safety & Engineering Review May 21, 2013

http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=875http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=697http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=495http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=490http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=316http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=304http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=269http://gm2-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?conferenceid=266

43

Appendix C: Project Risk Registry

The table presented below contain select columns for High and Moderate risks from the Muon g-

2 Risk Registry. The complete risk registry and risk log are available upon request. Red in the

tables presented below is for threats that were realized or opportunities missed, and green is for

opportunities realized or threats avoided.

Prior to CD-1 Review

Muon g-2 Realized Opportunities & Threats

Risk

ID#Type

Potential Problem/Opportunity

(short description)Mitigation Action

Threat Feasible transport for superconducting coils or

catasthophic damage during transport

Significant R&D and test welds were performed before the original welds in

the interconnects were cut. No damage has been seen. R&D and test welding

will continue to perfect the procedures before the reconnection is made. If a

failure occurs, a detailed lessons learned analysis will be conducted before a

splice is attempted.

2204 Threat Inability to complete 12Hz Lens pulse testing or

conclude projections of lens operating at 12Hz with a

high degree of confidence could lead to complete

change of requirements, specifications and design of the

entire target station.

Redesign target station

1101 Threat NSF MRI funding falls through Keep lines of communications open with NSF, invite to reviews,

reinforcement from DOE that the project is going forward

2403 Threat AP30 Cryo Duct incompatibility Build new communication ducts between MI8 manholes and AP30 service

building. Requires going through MI8 line berm.

1102/

1103

Threat NSF MRI funding was reduce by $300k in award,

universities not fully able to make up for difference

Work with universities to understand what essential pieces need additional

funding

1107.1 Threat Project is delayed in FY14 due to reduced funding

profile (Recognized in FY13 reduced guidance delayin

Final Design by 3.5 mos)

Keep OHEP informed, monthly tracking of interface milestones to GPP and

AIPs, focus on keeping critical path activities on track

1107.2 Threat Project is delayed in FY14 due to reduced funding

profile (Recognized in FY13 reduced guidance delayin

Final Design by 3.5 mos)

Keep OHEP informed, monthly tracking of interface milestones to GPP and

AIPs, focus on keeping critical path activities on track

Prior to CD-1 Approval


Risk

ID#Type



1111 Opportunity Project is able to schedule a protion of the additional

$1.7M of final design work needed in FY14 to regain

technically-driven schedule

Keep OHEP/laboratory informed, work with universities on forward funding

strategies, and deliver preliminary design work at the base estimate to earn

contingency.

1112 Opportunity Project is able to schedule a portion of additional $6.0M

of Implementation work needed in FY15/16 to regain

technically-driven schedule

Keep OHEP/laboratory informed, work with universities on forward funding

strategies, and deliver preliminary design work at the base estimate to earn

contingency.

Prior to CD-2/3 Review


Risk

ID#Type



1114 Opportunity UK consortium joins with funding support from STFC

to support trackers, reduces project exposure to $250k

tracker risks in registry

Supporting letters from laboratory/OHEP to UK as needed

1105 Threat Early Career funding for trackers not sufficient Early Career funded trackers will be statused by project EVMS so that

problems can be recognized for early intervention

1115 Threat A new inflector is needed (added cost but major

technical opportunity)

Encourage UK collaborators to pick this up as a substantial technical and

intellectuatl contribution, carefully bench test old inflector

44

1116 Opportunity UK consortium joins and takes on new inflector as

contribution

Supporting letters from laboratory/OHEP to UK as needed. Continued

communication with UK collaborators to understand possibilities.

1118 Threat MC-1 Building does not complete its scope identified as

optional

Keep change request on the MC-1 GPP to a minimum since there is currently

$1M contingency remaining. Work with the Directorate to see if the GPP for

the MC-1 building can be authorized to spend at $9.5M instead of the current

$9.0M target.

1119 Threat Magnet test stand needed at BNL for quadrupole tests Fund R&D at BNL in FY14 to determine safety factor in absence of field.

Schedule full field tests as soon as storage ring is powered at FNAL.

1117 Threat Dedicated MRI magnet constructed at Fermilab (aded

cost but major technical opportunity)

Such a facility would be multi-purpose and is thus a good candidate for an

Early Career award. Looking into reusing LANL magnet used in the muonium

hyperfine experiments

2303 Opportunity Simplified "cross-over" scheme to the M3 line, located

further upstream.

Use the new scheme.

2304 Threat Quadrupole733 in M2 quadrupole may require new

type.

Find magnet or change design.

2310 Threat The new magnet designs, the Lambertson and C-magnet,

may prove to be more expensive to design and build

than anticipated.

Mature engineering design will show whether the original cost estimates were

valid or not.

2311 Threat Extraction devices, particularly the Lambertson and C-

Magnet, may require complicated radiation hardening

and shielding to withstand Mu2e operation.

Mu2e completes shielding plan so that the impact is known.

2413 Threat Not enough viable spare vacuum cans. Design and build new vacuum cans to house the SWICs.

2414 Threat Commissioning is not completed during FY'14 studies

period.

Implement device at a later time when funding and engineering support

become available.2202 Threat Default target does not provide required pion yield for

experiment after beam study period summer 2013 or

beginning of run 2016

If the default target in not capable of delivering the required pion production

the alternative design for the target will need to implemented

2207 Opportunity Once preliminary lens operational power supply is

complete, there may be no need to install 13.8Kv 480v

3phase transformer

don't install hardware

2206 Threat Final phase of 12 Hz lens testing shows significantly

more engineering will be needed to run at required rep

rate

Finish bench-testing ASAP in FY14 to mitigate risk so lens is not on critical

path

2308 Threat The design for the in tunnel shielding for Mu2e can have

great bearing on the plan for reconfiguring the 30

straight section. Demands for shielding can change

plans for reconfigurations.

Track and understand the Mu2e shielding design and plan accordingly for the

30 straight section reconfiguration

2408 Threat Fixed BNL SWICs with vacuum window breaks cannot

adequately measure beam profiles or create too many

losses due to scattering.

Build new FNAL SWICs in place of the planned BNL SWICs in the M5 line.

3807 Threat absolute calibration probe is damaged The probe is stored in a safe location and will be carried by hand from BLL to

Fermilab, A alternate, more modern absolute calibration scheme based on He

3 is being developed as a backup option should it be required.

3808 Threat plunging probe is damaged As we refurbish existing probes, we will make spares.

3202 Threat The pole pieces have developed surface imperfections

that take them out of the tight flatness tolerances

The pole pieces were disassembled very carefully and coated to prevent any

further oxidation during shipment and storage at Fermilab. The surfaces may

need to be repolished if they are out of spec. This could require extensive

setup and machining time

4202 Threat Final cost at time of order exceeds preliminary quotes Purchase fewer spares at slightly greater operational risk. Delay some fraction

of purchase. We will make this purchase as quickly as possible using off-

project funds and in-kind contributions.

4204 Threat transient signals are present that cause problems with the

precision magnetic field

These devices can be shielded from the main magnet. All low voltage and

signal lines will be run in parallel to avoid bare currents. However, the plan

must be tested. A field test stand will be set up to measure all DC and transient

fields produced by prototype modules and final boards. If necessary, the

mechanical enclosures might be revised.

4207 Threat Amplifier board performance problems We have made 7 prototypes. We are working with EE faculty now and our

own engineer on a next-generation design. We are following the literature

closely. We are prioritizing this task so revisions can be made rapidly and

prior to the milestone to commit to a final design.

4404 Threat 1 GSPS ADCs become necessary without a significant

reduction in component cost

AMC13 and host µTCA platform can already accommodate the increased data

rates. Timing circuitry local to WFD AMC

3506 Threat Tubes fail to trigger or do not hold off voltage Swap in spare

3507 Opportunity A supply of thyratrons is available from E821 that can

be reused if tests prove operational

45

1121 Opportunity Baseline plan assumed project would pay for vacuum

chamber mods in CD-2/3 Dir schedule, but Early Career

funds should be able to pay

Move funding type to Early Career after review is complete

2401 Threat Inability to complete cable pull Build additional communications ducts between the Central Utility Building

and CMH34

Through CD-23 Follow-up


Risk

ID#Type



3602 Threat Old pulser does not work Test on bench to determine if now pulser has to be manufactured.

3605 Threat Muon losses higher than spec. B-field shimming, active monitoring of radial magnetic field (Br)

1108 Threat Project is delayed by 3 months in FY15 due to:

- Inadequate funding profile

- Delay in delivery of cryogens from Cryo Plant AIP

- Delay in construction schedule of Beamline Enclosure

GPP

- Slip in critical path schedule

- FNAL divisions unable to provide resources

- An injury or ES&H incident results in delay

Keep OHEP/laboratory informed, monthly tracking of interface milestones to

GPP and AIPs, focus on keeping critical path activities on track

1124 Threat FY14 laboratory overheads increase (known at this time

that a $400k increase in FY14 is in the pipeline)

Not much that the project can do to mitigate this risk. Some discussion

ongoing about giving projects a fixed rate.

1104 Opportunity In-kind contribution from INFN for laser system comes

through

Supporting letters from laboratory/OHEP to INFN as needed

2313 Threat Tunnel GPP may not pay for tunnel cable trays. Will not be known until Tunnel GPP is nearing completion.

2401 Threat Inability to complete cable pull Build additional communications ducts between the Central Utility Building

and CMH34

2404 Threat Do not get approval for safety rated PLC for ODH. Obtain approval in as timely a fashion as possible.

2406 Threat Preamps are not sufficient to see the beam. Build new SWICs

2407 Threat SEM hardware not functional Build new SWICs

3201 Threat The yoke is in several pieces and it may not fit back

together on the first try.

The disassembly of the steel was done carefully and was well documented.

Fermilab staff was on site at BNL to understand the procedure. The steel will e

reassembled as soon as it can to ensure it does not become a schedule driver if

problems arise

4210 Threat Original design may not control the noise to the desired

level

Make prototype as early as possible. Work closely with Washington electronics

engineer group who are designing the summing and amplifier board to use

best practices jointly to reduce noise. If necessary, develop some level of

filtering to reduce noise on the low voltage lines.

4211 Threat Delay in selecting SiPMs impacts low-voltage design

timeline

Ideally we will design the low-voltage to be independent of final SiPM vendor

and compatible with any devices. This will introduce a two-stage approach

with a bias offset and a tunable final stage.

4201 Threat Vendor delay in fabrication and growth of crystals We have a local collaborating university in Shanghai, near the Vendor can

keep close track of the production and help to resolve any delays quickly.

4203 Threat instantaneous rate is too high and a gain adjustment

needs to be made during injection

We are performing in-lab flash tests that simulate the effect of high rate at

injection to determine if our boards can recover. We are continuing to

simulate the injection to gauge the expected rates. If necessary, our engineers

will have to introduce a switching circuit in the bias supply

4205 Opportunity SiPM performance improvement We will wait to the last moment to obtain the best products and we will be

testing SiPMs from various vendors to make compare technically to obtain the

best performance solution

4206 Opportunity SiPM cost reduction We are planning to make our readout boards modular and ready for a variety

of standard footprint SiPMs from different manufacturers so that we do not

lock in on only one company. This will allow for a competitive bid for

pricing.

4213 Threat Insufficient light to flash entire system A system involving multiple primary lasers, all synchronized and coordinated

to direct light flashes into a common mixer on the upstream side will have to

be employed.

4214 Threat System stability is insufficient or inconsistent among end

branches

As early as possible, we will built a prototype system with as many branches as

will be required for the full system. Additionally, we will carefully select

Monitor detectors and place them throughout the system.

46

4303 Threat instantaneous rate is too high and a gain adjustment

needs to be made during injection

We will continue to improve our full simulation of the entire accelerator

complex form the target to the ring. This will be verified with accelerator

studies that determine the beam composition. We will continue to simulate

and pin down the maximum acceptable rate in the straws. If necessary, the

circuit will be added.

4306 Threat get a batch of gas with impurities Purity specs will be well documented and communicated to the vendor before

each shipment. Purity will be tested on arrival and monitored in-situ. An

undetected problem could severely degrade the lifetime of the entire system.

4406 Threat required footprint for each channel does not allow 4

channels per board

A full layout will be completed before baselining. If the package does not fit,

more boards will be required.

CD-2/3 Approval to Project Compete


Risk

ID#Type



3607 Threat baseline design of Q1 outer plate technically unfeasible Perform R&D studies and testing of a prototype at early stages of the Project.

Consider DC HV option for Q1 outer.

1113 Threat After first cooldown in FY15, yoke has to be

disassembled to modify or repair cryostats resulting in

more ring PM due to schedule delay. Poles may need

another round of alignment.

Make sure initial alignment is meeting spec. Quality assurance/control during

assembly.

4307 Threat tracker production experiences delay The design for the modules has been simplified so that only one geometry is

being produced which will allow for faster production with only one module

type. Priority will be given two getting the first two trackers installed and it

would not compromise the physics results if the 3rd tracker was installed in the

first shut-down

3611 Opportunity reuse existing insulators Test the system for holding the HV

1120 Opportunity Opportunity to advance accelerator construction

schedule by 2 +/- 1 months

with an addition $2-4M in funds advanced from FY17 to

FY15/16. Current technically-driven schedule has to be

held out to meet funding profile.

Continue to accomplish work as close to baseline cost as possible in order to

move contingency into schedule work. Keep OHEP and lab informed of

difference between technically and financially-driven schedule

1131 Threat Although this is a cost threat, it is really driven by a

technical opportunity. The injection efficiency into the

storage ring could be doubled with a new inflector

Project and scientific collaboration will continue to look for means of alternate

funding, although there are no prospects at this time.

2416 Threat Bayonet and antivacuum can not able to hold required

vacuum levels.

Redesign the antivacuum box to provide a better vacuum seal.

2418 Threat Bayonet and antivacuum can not able to hold required

vacuum levels.

Redesign the antivacuum box to provide a better vacuum seal.

3606 Opportunity Reuse old insulators Test on bench to determine if quads hold voltage with old insulators, measure

leak current

4302 Threat system alignment out of spec Alignment will be done in a beam test and using lost muons. Individual straws

will be X-rayed to determine wire positions. A stand-alone alignment system

using for example hall probes may be needed.

4405 Threat Clock distribution via AMC13 does not meet

experimental requirements of stability, particularly for

clock frequency or phase drift over muon fill

1) Clock distribution performance will be characterized on test stand, currently

under assembly. 2) Option for separation of clock and control signals, with

dedicated clock input on new AMC13 mezzanine card; 3) Include front-panel

clock input on WFD AMC, bypassing AMC13 and µTCA distribution



- Slip in critical path schedule

- FNAL divisions unable provide resources


- The He leak in the magnet degrades such that

shimming is delayed


GPP and AIPs, focus on keeping critical path activities on track. The magnet

is being monitored for evidenc that the leak might be degrading, no evidence

so far. If the He leak did increase, then the pumping speed could be roughly

doubled in the area of the leak with relative ease.

4308 Threat Straws do not meet QC spec on diameter We are purchasing much more material than we need to allow for a low

throughput. Raw material will be converted into straws In 4 batches and QC

will be made after each batch.

1123 Threat Funding constraints push the start date for much of the

accelerator work into FY16. This leads to a significant

risk of schedule slip that is worth considering separately

from the schedule risks summarized in risks 1109/1110

Use any contingency that might be liberated in FY15 to start accelerator work

earlier

1132 Threat Some problem develop with the storage ring that

operational budgets are unable to pay.

Working with PPD to ensure operations budgets remain adequate. Minimizing

risk to storage ring damage with proper, regulary updated procedures and

operator training.

47

2402 Threat HRM Incompatibility Establish CAMAC links at MC-1

3811 Threat higher order multipoles exist that are out of dynamic

range of the shimming kit.

The field uniformity requirements are well specified and communicated. The

detector geometries are chosen to minimize higher multipole fields. The muon

beam is round to minimize the effects of higher order multipoles. In the event

that something does occur, either the source would need replacement or a

solution would need to be devised using a combination of the passive and

active shields.

4401 Threat Environmental noise, particularly noise correlated with

fill structure, introduces timing shift

1) Clock distribution performance will be characterized on test stand, currently

under assembly. Testing will include sensitivity to prototype kicker operation

to mimic a potential fill-correlated noise source. 2) Base design on integrated

components like TI LMK04010, rather than discrete components, to limit

sensitivity to noise; 3) Include clock monitoring within WFD

2302 Threat Desired spot size is not achieved or simulations

overstated improved pion yield from smaller spot size.

None

3806 Threat Current trolley frequency measurement (onboard

electronics) need to be replaced entirely.

Careful handling to prevent any damage. Careful evaluation of systematic

errors related to the frequency measurement in the test solenoid as soon as

possible to investigate this risk. Test solenoid is being established at Argonne

and past E821 experts are meeting there in June to test the full system.

3809 Threat Delayed CENPA electronics shop availability. Some projects in the shop's queue could be outsourced or additional part time

personnel could be hired.

2315 Threat Until design and constructions work is 100% complete

there is some risk that the kicker power supply might be

more expensive than anticipated

Complete design work at 100% level as quickly as possible, advance

contruction schedule if project funds allow



- Slip in critical path schedule due to large amount of

accelerator work scheduled in FY17

- FNAL divisions unable to provide resources



GPP and AIPs, focus on keeping critical path activities on track

2417 Threat Ion Chamber and Digitizer are not sensitive enough to

see 10^5 beam

Design a low intensity intensity measurement device based on scintillators.

3402 Threat Required pumping speed is much greater than the

pumping speed achieved at E821

We will collaborate closely with the quad and tracker groups to ensure any

changes in requirements are communicated as soon as possible. Secondary

containment vessels may be necessary to increase pumping speed locally. For

a drastic change, a new system may need to be designed and implemented.

3610 Threat ESQ system alignment out of spec Identify potential problems in advance (in R&D), upgrade problem

components, develop control and monitoring procedure

4403 Threat Transient currents that perturb precision magnetic field All DC and transient fields produced by µTCA crates, prototype modules and

final boards will be measured in the magnetic field test stand. Shielding

strategies will first be exploited if necessary, followed by locating the crates

farther from the field (longer signal cables) If these actions insufficient, a

large design iteration may be needed.

48

Appendix D: Detailed Technical Lessons Learned from CD-0 through CD-3

Three most significant “success” lessons for this project

Lessons Learned

Description, Impacts, and Solutions

Building a strong

and successful

project management

team

The project management team for the Muon g-2 project was assembled over a 2 year period. Key members of the team were actively recruited

for over a year. The team was augmented prior to CD1 by a deputy

project manager with significant project management experience.

The management team has become strong and successful and works together to lead the collaboration through the design and fabrication

process and through the associated project reviews. It has been

particularly important to have such a talented team to navigate reviews,

project execution, and the transition to operations.

Preparation of the project plan and all documentation for each DOE CD

review was preceded with a “dry run” managed by the Laboratory. This

enabled the project to benefit from feedback and make corrections in

the development of the project baseline prior to the formal DOE

review.

Early

implementation of

an Earned Value

Management

System

Starting the roll-out of the EVM system in the months prior to CD-1 allowed the project to assess very early where cost & schedule

problems were developing pre-baseline. It also allowed the project

team to benefit from a significant amount of time to learn how to use

EVM as it evolved from a basic analysis at CD-1 to the full EVM and

project management process deployed by the baseline review.

Project tasks were created with durations that were trackable on a monthly time frame so that progress could be monitored in conjunction

with the monthly budget reports.

For project management and engineering oversight, “level of effort” tasks are appropriate for managing the budget for these resources.

Project analysts needed to work closely with the technical managers to understand activity sequences before developing the schedule logic.

It is very important

to select high

quality vendors.

The major vendor contract for the g-2 project entailed the safe

transportation of the BNL storage ring from Long Island to Fermilab

Emmert International was selected even though they were the high bid due to the strong value added.

Safety was a key part of their company culture and they led meetings at BNL, FNAL and with the transportation agencies to ensure Emmert

was compliant with all safety protocols.

Emmert understood the delicate nature of the storage ring and worked with the project team to ensure damage to the ring would be avoided at

all steps of the transport.

Emmert also worked well with project and communications personnel to turn the transport of the ring into a very successful PR event for the

field.

49

Three areas of potential improvement and how they have or might have

impacted the project.

Lessons Learned


Estimating the cost of

refurbishment of old

equipment can benefit

from more

assessment in the

final design phase.

Estimating the cost of refurbishing equipment is difficult. The repeated

lesson seems to be that as soon as implementation work starts, new

information is discovered, and the plan for old equipment is likely to

change. Redefining the initial stages of that work to be final design and

completed ahead of baselining would be advantageous. Examples

include

Once the storage ring was moved into the building and could really be taken apart, it became clear that much more refurbishment of

valves and seals would be required, which also resulted in much

more leak checking.

Dismantling of the inflector lead box and valve can revealed that the corrosion and piping not meeting ASME standards could be repaired

more cost effectively than rebuilding a new system from scratch.

Extracting vacuum cages from the chambers and doing as-founds emphasized difficulties in meeting the alignment spec for quad plates

and trolley rails.

The original plan assume the electrostatic quadrupole pulsers would be reused with minor refurbishment. Initial analysis from engineers

determined that the cost would be prohibitive, the lack of

documentation would make it very difficult, and the end product

would not meet current standards.

Other areas where

costing lessons were

learned.

Drafting was an area where estimates were consistently low. For beamline installation it was found that the design needed to produce

a defensible beamline final design was about 1/3 of the final drafting

needed to be construction ready.

Alignment time, especially with a device like the g-2 storage ring where alignment errors directly generate systematics, was initially

underestimated by a factor of 2 to 3.

Beamline installation requires a degree of oversight from engineers and engineering physicists that was not in the original cost model.

50

Technical reviews Throughout the DOE CD review process the project had trouble communicating all of the technical details to the reviewers during the

short timescale associated with the review. The project learned of

the importance of contacting the DOE reviewers several weeks in

advance to start the dialogue as soon as possible. A much better

model is to conduct multiple, project-commissioned reviews with a

small panel of experts that are given 1-2 months to study the issues.

In the year between the first CD-2/3 review in summer 2014 and the

follow-up CD-2/3 review in 2015, the project conducted 3-4

technical reviews of particularly complicated subsystems. Having

these mini-review reports available to the DOE reviewers made their

job much easier and allowed the project to much more effectively

communicate the degree of construction readiness.

Other lessons learned.

Lessons Learned


L3 managers and

scientists with technical

expertise.

Accelerator L3 managers and installation coordinator each had more than 20 years of experience operating accelerators at

Fermilab.

Iteration on BoEs between L3 managers and engineers helped to ensure that all scope was accounted for.

Experience with beamlines being reconfigured had a positive

influence on new beamline design in avoiding known pitfalls and

developing creative solutions.

Strong engagement of the experimental research collaborators, supporting the project team, added much value to the quality of

project baseline development and execution.

Selecting an appropriate

number of CAMs The project entered the CD-2 review cycle with roughly one

CAM per L3 WBS element. A reexamination of the amount of

work in different WBS elements led to a redistribution of control

accounts with 7 CAMs, thus making the project more

manageable.

51

The risk-mitigating

strategy of delaying

construction approval

until the storage ring

could be tested had both

positive and negative

impacts on the project

Advantages:

Beneficial for minimizing project risk by requiring a full demonstration of the g-2 storage ring prior to full approval for

funding the implementation phase.

Helped to keep the ring work as a high priority at the laboratory.

Interim funding through special authorizations from the DOE Project Management Executive allowed the ring work to progress

($2.7M) and critical path accelerator implementation ($2.9M)

kept the schedule largely on track.

Disadvantages:

The next stage of funding being so closely tied to the ring demonstration created a very high pressure situation for the

management and engineers.

The finite ‘special authorization’ funding was set initially and was based on needing to use estimate uncertainty, but did not

cover all of the risk.

A couple of risks were realized during this time period including needing to repair a known cold leak from the BNL era and a

serious schedule delay that arose from discovering a poor indium

connection in the lead can.

The refurbishment of the ring cryo system and construction of new cryogenic lines was greatly underestimated by more than

50%.

In general, the combination of the finite funding running out (prior to CD-2/3) and the schedule delay conflicted with taking

the time to carefully implement quality control. As a specific

example, more time spent on the analysis of the resistance

measurements of the reconnected indium joint might have

revealed the need to repair this joint prior to the first cool-down.

52

Appendix E: Detailed Technical Lessons Learned from CD-3 through CD-4

WBS 1.0 Project Management

Lesson Learned


Success:

Custom project

management tools made

work more efficient and

effective.

The project office constructed a number of custom PM tools

that worked well with positive feedback from the managers,

many of which have had experience with tools used in other

projects. In particular, a standardized Basis of Estimate form

was created in Excel to assess costs and collect the activity-

level descriptions for constructing the P6 schedule. A second

extremely useful tool was a shared Google spreadsheet that was

used on a monthly basis to collect the status of all of the

activities for generating the monthly EVMS. Finally, a

spreadsheet based risk registry that performed Monte Carlo

calculations using Excel macros was used for tracking

remaining cost risk, but had the drawback of not calculating

remaining schedule risk. In general, Google shared documents

were used for a number of other areas including issue tracking

and compiling specifications. The simplicity of the tools

enabled broad buy-in and prompt response from managers,

including university-based collaborators.

Success:

Management team

composition led to coherent,

effective team.

The project team included a number of early career scientists

with technical skill and enthusiasm that made up for their initial

inexperience with the formalities of DOE project management.

The Project Manager, an early-career Wilson Fellow, had

extensive subject-matter expertise, having been a graduate

student on the previous Muon g-2 experiment at Brookhaven.

He and the other early-career scientists in key leadership

positions (Level-1 and Level-2 managers) inspired and

empowered the entire team. To compensate for their initial

dearth of management inexperience, a senior scientist and

skilled manager served as the Deputy PM prior to CD-2 and

helped guide and mentor the team during the project-

development phase, with additional help from the Fermilab

Office of Program and Project Support (OPPS) and

management training classes delivered by the Laboratory.

53

Success:

Project and scientific

collaboration worked

together effectively to

achieve project goals.

Throughout the course of the project, reviewers commented

about how effectively the project and the scientific

collaboration worked together. There were a number of factors

that contributed to this. One was having a project manager that

was also a subject matter expert, which made it easier to reach

consensus on priorities. It also helped that the spokespeople

and project leadership had weekly meetings to make sure that

everyone at the top of both organization charts were constantly

in agreement on the best course of action. The project actively

sought out ways to bring additional scientific effort from the

collaboration to assist with the project deliverables. This

included technical work in the construction and assembly,

simulations to inform design decisions, assignment of

management positions within the project, and R&D and testing,

and quality control. This had the two-fold benefit of reducing

the project cost compared to relying solely on laboratory

resources and helped to keep a large portion of the collaboration

positively engaged with the project.

Success:

Tracking uncosted scientific

effort ensured accurate

progress measurement.

The project relied on effort from the scientific collaborators,

much of which is not costed to the project, per DOE HEP

guidance. Understanding the effort needed, and measuring

progress on tasks performed using that effort, was a priority for

the project. The Project measured Earned Value for these tasks

by including all activities needed to complete project

deliverables, with estimates of how much uncosted labor was

Documents

Project Closeout Report for the Muon g-2 Project · 2018. 7. 12. · This is the Project Closeout Report for the Muon g-2 Project, a DOE project, which will enable the Muon g-2 Experiment