6/19/2014

PHENIX PHENIX WEEKLY WEEKLY

PLANNINGPLANNING

June 19, 2014June 19, 2014

Don LynchDon Lynch1

6/19/2014

This Week

• Run 14 Continues• Maintenance Access 6/16:

• VTX monitoring work• MPC-Ex board replacement

• VTX/FVTX Shutdown work review meeting (Monday)• Plan for 2014 Shutdown• Tech Support for Run 14 as required• Support for sPHENIX efforts as required

2

6/19/2014

Next Week

• Run 14 Continues• No scheduled access for remainder of Run 14?• VTX/FVTX Shutdown work review meeting part 2 (Monday)• Plan for 2014 Shutdown• Tech Support for Run 14 as required• Support for sPHENIX efforts as required

3

6/19/2014

2014 planned Technical Support & 2014 Shutdown

Support for run 14 2/3-6/30/2014Procure & Fabricate parts for MPC-Ex North and South 1/1/2014-6/30/2014Set up Physics lab for FVTX/VTX east 6/15/2014End of Run Party 6/27/2014MuID Efficiency Measurement (Itaru, requires cooling water & isobutane) 7/7-7/10/2014Start of Shutdown Tasks (purge flammable gas, disassemble and stow shield wall, remove collars, move EC to AH, Move MMS south, etc.) 7/14 – 7/25/2014Remove FVTX/VTX East to PHYSICS?, repair and reinstall 7/14 – 9/15/2014Remove MMS east vertical lampshade 7/28-7/30/2014Troubleshoot intermittent water leak in MMS 7/30- 8/8/2014Other Maint. In MMS TBDReinstall MMS east vertical lampshade TBDSummer Sunday prep AH, tours and restore AH 7/30-8/15/2014Install scaffolding in Sta 1 South 7/28/2014Remove MPC-Ex prototype 7/28-8/1/2014MuTr Sta 1 South troubleshooting and repairs 7/28-8/1/2014Maint. & Repairs for MPC South, BBC South, RPC1 South1 7/28-8/1/2014Assemble & test MPC-Ex North, ready for installation 8/1-9/5/2014Remove scaffolding from sta 1 south, Move CM South 8/4-8/5/2014Install scaffolding in Sta 1 North 8/6-8/8/2014MuTr Sta 1 & Sta. North troubleshooting and repairs 8/11-9/5/2014Prep MPC-Ex North installation area 8/11-9/5/2014Install new MPC-Ex North 9/8-9/26/2014Assemble & test MPC-Ex South, ready for installation 9/2-10/3/2014

4

6/19/2014

Remove Sta 1 N scaffolds, Move CM North, Install scaffolding in Sta 1 S 9/29-10/3/2014Install MPC-Ex South 10/6-10/24/2014Other detector support TBDInfrastructure Maintenance and Improvement TBDDecommissioning of obsolete PHENIX detector equipment TBDsPHENIX Support on-goingEnd of Shutdown Tasks (Move MS north, roll in EC , install collars,remove 10 ton cart, plates and manlifts, build shield wall, etc.) 10/27-11/26/2014Pink/White/Blue Sheets 12/1-12/19/2014End of Shutdown Party ????Start Flammable gas flow ????Close shield wall, install radiation interlocks and prepare for run 14 12/31/2014Start run 15 1/2/2015

2014 planned Technical Support & 2014 Shutdown (cont’d)

5

Muon Tracker Shutdown Work List – summer 2014

• testing as MPC-EX installed, particularly before closing Sta-1’s

• fix North Arcnet – N.2.7.1, North Sta-2 Oct-7 Chassis-1 (bad cable?)

• fix packets that were disabled for Run14

• 11035,36 – South Sta-1 Quad-4 Chassis-3

• 11267,68 – North Sta-2 Oct-7 Chassis-2

• replace boards for most frequent FEM problems from run

• 11195 - North Sta-1 Quad-3 Chassis-3?

• might have already done this; check history (changed RX 3/14/12)

• 11064 – South Sta-2 Oct-3 Chassis-3 - unreachable

• N341 HV trip problem?

• auto-reboots of ArcNet and iocondev’s for calibration?

• Access needed:

• South & North Sta-1

• Inside North Sta-2 on bottom

• Main Issue – Manpower

6/19/2014 6

6/19/2014

VTX/FVTX east repairs/upgrades required

West to remain installed ??

Need to get PHYSICS FVTX/VTX lab ready by ~ mid June

Meeting Monday 6/16 to discuss shutdown task details: follow up meeting next Monday 6/23 1:00 PM 1008

7

6/19/2014 8

Action Items:

1. FVTX group prioritize list of IR work based on work that needs to be done with detector “cold”

and what can be done when detector is “warm”.

2. M. Wysocki: Discuss needs for IR electronics testing with Miljko and report back to VTX group.

3. R. Pisani: Estimate cost and time to add a 4th chiller and split FVTX disks from VTX StriPixel

Ladders

4. E. Mannel: Estimate heat load on cold loop by FVTX disks and VTXS ladders

5. E. Mannel/R. Pisani/S. Boose: Investigate adding additional temperature/humidity sensors inside

the (F)VTX enclosure.

6. Mike L./ Rob P.: Look at ways of creating an “dry” zone around the (F)VTX enclosure using a

portable AC unit.

• End of run is scheduled for July 7, 2014. It is expected that the shield wall will remain up for at least a couple of days (until July 10). Once

the wall comes down, the IR will be subject to higher levels of humidity and larger temperature variations. Work can continue on the detector

after the wall comes down, but most likely will have to be at higher temperatures to avoid condensation issues.

• Current plan is to remove the East Detector only. De-cableing of the east half will begin sometime after July 14, but can be delayed if

needed for additional IR studies by the (F)VTX group. VTX StriPixels may request west half to be removed depending on understanding of

bias issue in the east half and IR studies immediately after the end of Run.

6/19/2014 9

• Post-Run Studies in the IR:

VTX Pixels: None

VTX StriPixels: Bias studies. Requires bias current measurements varying the N2 flow and warming the detector up. Warm up studies will be last after all “cold”

work by (F)VTX groups is complete in order to avoid thermal cycling of the detector. Expect the work to take or order 1 day (half day cold/half day warm). Unclear if

any IR work is requested by ORNL group for readout work. M. Wysocki will discuss it with ORNL group.

FVTX: FVTX group has a number of problems observed only in IR (see FVTX WIKI page). Group needs to prioritize the list based on what needs to be done “cold”

and what can be done once the detector is “warm” and estimate the amount of time for the work.

• Physics Lab Work:

•General: Mike L. has requested a couple of days once the detector is in the lab to look

•at the detector in order to see what can be done to improve the flow of N2 and better

•seal against condensation.

•VTX Pixel: Investigate one bad ladder on east detector (B1-L11). Will concentrate on

•one end first. Will require access to the SPIRO Board. This requires removal of FVTX 1⁄2 cage and Bigwheel, and a couple of Pixel Bigwheels. Depending on

what the problem is may require access to SPIRO board on opposite end.

•VTX StriPixels: Investigate one bad ladder on east detector (B3-L16). Requires access to LDTB. This requires removal of south FVTX 1⁄2 cage and south Pixel

Bigwheels. Depending on outcome of IR bias studies

•FVTX: Number of ROC issues. Can be done with 1⁄2 cages installed or removed • IR Infrastructure after East detector is removed

•Chillers: general repair and maintenance of the 3 chillers. Requires tech time and support for BNL HVAC group.

•Cooling lines: Replace Teflon tubing in the vicinity of the detector with stainless steel tubing. Look at any rerouting of stainless steel lines that were added during

the run.

•Interlocks: Review flow interlock hardware and update/modify as needed. Provide additional interlocks and monitoring for 4th chiller if added (see next item)o The

VTX StriPixel group is requesting that the FVTX and StriPixel ladders be put on separate chillers. This will require purchasing an additional chiller, running

additional cooling lines to the IR and installing additional manifolds at the detector. Need to estimate cost of materials and labor required to add 3rd chiller loop.

•N2: Look at ways of better sealing up the enclosure to reduce the leakage of N2 and migration of humidity into the enclosure. Concern that sealing the detector

too tight will impact the cooling of electronics. Right now there are significant gaps at the BigWheels in part due to the age of the foam seal. Need to also re-

evaluate the N2 distribution inside the detector.

•There was a suggestion to look at ways of creating a low humidity air volume around the enclosure. Idea would be to put some kind of “tent” around the (F)VTX

and pump dry air from an portable AC unit into the enclosed volume. Region would not have to be air tight, but provide a region of drier air around the enclosure.

Mike L and Rob P. will investigate options.

6/19/2014

MPC-Ex N & S Final Installations This summer

Working on North BP support design

Tungsten plates received, QC acceptable

Additional parts ordered from CS due end of June

Rescheduling meeting yesterday to update shutdown schedule

10

6/19/2014

(1) Stony Brook in charge of MPC-EX testing

(2) Tom Hemmick is leading a test beam effort at SLAC June 20-30 th: one MPC-EX hemisphere with 8 tungsten layers and 4X and 4Y carrier boards one micromodule on each layer, partial assembly to be returned to BNL by July 7.

(3) Current Parts Dispositions:

Sent to Stony Brook for tests:

8     (4 x and 4 y) carrier boards laminated to 'W' plates (new plates).3     loose carrier boards for testing (one missing a connector)50     brass spacer nuts4    SS    1/4 - 20 x 4" studs4    rapid prototyped spacers1    micromodule

Mike Lenz office:

1    Assembly fixture3    Delrin covers (in the shape of the 'W' plates)6    'W' plates (new plates)50     brass spacer nuts

At Central shops (due 6/27/14):

5 mounting tabs50 brass spacer nuts20 FEM mounting brackets20 FEM mount Isolation Boards

Jim LaBounty’s office:

Installation assembly parts (to be itemized)

(Additional parts in currently installed partial South prototype to be itemized)

MPC-Ex Project Summary

11

6/19/2014

Current Parts Dispositions (cont’d):

Sensors

Sensor testing at Yonsei is proceeding well, and we should have the remaining 200+ sensors from Yonsei by the end of June. We already have almost all the sensors we need for the south MPC-EX.

Micromodules:

450 ROC’s ordered:

100 received and used in building initial micromodules 100 in transit to BNL250 rejected by Sierra (bad solder mask) and being remanufactured expect shipment in ~ 1 week

The electronics for the FEM is designed and at Sierra awaiting a BNL PO for manufacture and assembly.  The FEMs have four readout inputs, so there will be eight FEMs per arm.  two FEMs per box, four locations on the magnet for each arm.

(4) Assembly Plan

•After receipt at BNL, ROC’s are inspected and sent to Quik Pak with SVX4 chips for wirebonding the chips to the ROC•ROCs returned to BNL and inspected•BNL wirebonds the sensors and assembles the micromodules•Micromodules sent to Stony Brook for testing•Final assembly into MPC-Ex N & S at Stony Brook, PHENIX tech to participate and assure QC•Return assemblies to PHENIX for installation in accordance with current shutdown schedule

MPC-Ex Schedule considerations, Cont’d

12

Getting Ready For DOE Science Review 7/1-7/2Outer HCAL

Cryostat

Inner HCALEMCAL

VTX

6/19/20146/12/2014

13

Basic sPHENIX model cutaway (updated)

6/19/2014

Design efforts over the next few months will include several areas of emphasis:

(a) incorporating a more detailed flux return (cap) design

6/12/201414

sPHENIX open, with racks6/19/2014

(b) A more detailed structural support design, with support structure envelope details to be included in detector envelope

6/12/201415

(c) Initial concept for external support, racks and upper bridge platform to be more detailed and conform to envelope restrictions

6/19/20146/12/2014

16

6/19/2014

(d) “Innie” / “Bothie” tile configuration to be updated

17

Magnet (BABAR) (Magnet Div)

• Acquire rights Done• Evaluate Transport Done• Design Transport fixtures

Done• Fabricate Transport

Fixtures 6/15/14• Evaluate BNL test facility requirements

8/1/14• Prepare test facility 9/1/14• Install Transport fixtures

9/1/14• Transport to BNL

9/15/14• Install in test facility

10/1/14• Design tests/test equipment 10/1/14• Fabricate/procure test equipment 1/1/15• Run tests

6/1/15• Design Magnet modifications 12/1/14• Fabricate magnet modifications 6/1/15• Install magnet modifications

8/1/15• Evaluate final installation

9/1/15• Design installation support equipment

12/31/2015• Fabricate/procure support equipment 7/1/2018• Final test before transport to 1008 5/1/2019• Transport to 1008 IR 6/10/2019• Install

8/30/2019• Test

10/25/2019• Commission

11/22/2019

6/19/20146/12/2014

18

6/19/2014

Babar Magnet Schedule: Mike Anarella

6/12/201419

6/19/2014

ePHENIX

2ft high x 1 ft wide clearance needed for e-ring components

20

6/19/2014 21

From Ray Karol:Lessons Learned: Door switches fail due to improper installation

Discussion: During a Search and Secure procedure for the BNL, NSLS-II storage ring, two adjacent door switches were found in the closed position when they should have been open. By the switches remaining in the closed position, the monitoring system "thinks" the door is closed when in fact, it could be open and access could be gained to the accelerator enclosure. Note that the locks and a second set of switches remained fully active during this period and the safety of staff was maintained at all times.

Investigation of the event revealed that the installation of the switches was inadequate resulting in improper strike angles and/or interferences, which ultimately caused damage to the switch bodies and the premature failure of the switches.To ensure uninterrupted protection, the door switches are installed in two independent logic chains and are also redundant within that chain (i.e., two switches per chain). Due to the specific design of the switches and the method of installation, both switches failed in one of the two chains. Protection of staff was maintained throughout the period through the second chain.Findings: -Inconsistent installation. -Varying strike angles. -Interference between the switch body and the face of the door.

6/19/2014 22

Analysis: A "5 Whys" analysis was performed to assess the cause of the failure:1) Two PPS Switches failed simultaneously by getting jammed in the closed (made) orientation (why?) 2) Either (a) the door struck the leading edge of the limit switch-body causing the internal components to be damaged or (b) the door struck the switch lever-arm while the arm was at a near-zero strike angle with respect to the door, which caused the internal components to be damaged (why?) 3) Prior to installation, a thorough review of the mechanical elements of the PPS Door/Gate Interlock System was not conducted by an experienced mechanical engineer (why?) 4) The PPS interlock system is mostly electrical in nature. The mechanical interaction between the door/gate, limit switches and "Mag Lock" appear to be straight forward but in actuality, there are many things to consider that would normally be reviewed by a mechanical engineer. The participation of a mechanical engineer was not considered in the scope of resources required to design and implement the PPS Interlock system (why?). 5) The PPS design and installation practices have not included a review by a cross-cutting team for this system. In summary, the mechanical assessment that took place on January 20, 2014, identified two probable causes for the switch failure: inadequate clearance and improper strike angle.

Recommended Actions: 1) Review the Design Review process to ensure all systems are reviewed by an appropriate team of engineers and/or technical staff. All installations should be reviewed by a mechanical engineer or other qualified person prior to placing them into operation to ensure installation meets manufacturer requirements. 2) Periodically inspect all switch installations to ensure proper clearance and strike angles are maintained.

6/19/2014

PHENIX Safety and Security

23

6/19/2014 24

6/19/2014

Where To Find PHENIX Engineering Info

http://www.phenix.bnl.gov/WWW/INTEGRATION/ME&Integration/DRL_SSint-page.htm

Run 14 Continues!Just over 2 weeks to go !

25

6/19/2014 26