High Voltage

PHONCS23

In the PHENIX Control Room, a DELL PC, phoncs23, is dedicated as the console of the HV control of PHENIX. It is located in the Southwest corner of the Control Room. During the Run, SA1 is responsible for the HV of all subsystems, and he/she should use phoncs23 to monitor and control the HV.

IOCs or IOCONDEVs

Phoncs23 is a console terminal for HV control, and the actual HV control is done by VME167 computers called Input Output Controllers, or IOCs for short. In the present configuraiton, 5 IOCs (iocondev3, iocondev4, iocondev5, iocondev6, and iocondev7) are used for LeCroy HV control. In addition, iocondev1 is used for TOF HV control( TOF-HV manual here ), which uses a CAEN HV system.

The present configuration of the LeCroy HV and HV IOCs are documented in HV_ARCNET.html by John Haggerty.

Desktops for HV control

Pasted at the bottom of the screen of phoncs23, is a map of the OpenDesktop menu. It shows which iocondevs are controlled from which desktop.

In this case, there are 6 iocondev desktops setup for the machine, named: "iocondev3","iocondev4","iocondev5","iocondev6","iocondev7", and "ioc consoles". You can click one of those boxes to select a desktop. In the figure, a desktop named "iocondev4" is selected.

The desktops "iocondevn" (n=3,4,5,6,7) are set up to control HV channels that are controlled by these iocondev HV controllers. The desktop "ioc consoles" is a screen to talk to IOCONDEVs directly. Note that there are 3 additional desktops in the menu dedicated to the alarm handler, the Pad chamber control, and the DC HV monitor GUI.

HV control GUI

If you select one of these 5 desktops, the desktop should have a main menu of the HV control GUI. It is like the figure below.

This is a main menu of the HV GUI for iocondev3. In this screen, you can trun ON/OFF LeCroy HV mainframe. In this figure, there are 5 LeCroy HV mainframes cotrolled by iocondev3 (shown at top). A pair of "ON","OFF" swithces and a colored box (yellow in the figure) corresponds to one LeCroy HV mainframe. The name of the mainframe and the subsystem served by it is indicated below the buttons. By clicking the "On" button by mouse, you can turn on the mainframe. The status of the mainframe is shown in the color of the box next to the button. It is yellow if the mainframe is OFF and green when ON.

These mainframe buttons only turn on/off the mainframe power. A HV channel is actually turned on when and only when the channel is also "enabled". Individual channels are controlled by next level of the menu screen. You can select a channel control screen from the menu. First, you move your mouse to one of the boxes in "Voltage/Current" column. For example, the mouse on the box in "RICH_W_S" row. Then, a cascade menu appears as shown below. The cascade menu indicates that there are 7 sub-menus for "RICH_W_S" (RICH detector West-South part). Move mouse on the top of one of those 7 lines, and click it. Then, the HV control page corresponds to the selected channel should appear.

In some cases, there is only one sub-menu associated with the selection box. In this case, the HV control page pop up without a cascade menu step.

The figure below shows one of the HV control page. This is a page for BBC HV control.

The page has Enable/ Disable swiches, measured voltage, Demanded voltage, Set voltage, and measured curent, and the status for each HV channel. If you push "Ena" button, the channel is enabled, and if you push "Dis" button, it is disabled. The colored box at the middle is yellow when it is OFF, and it becomes green when it is turned on. The status is also shown by number in the right most box.

Note that the response of the system is VERY SLOW. Depending on the activity, it can take ~20 seconds for a change to be instantiated.

Consoles of IOCs

In principle, you can control the HV by the GUI explained above. However, sometimes you may want to talk to the IOCs directly. For instance, the iocondev has crashed and you have to reboot it.

The desktop named as "ioc console" is set up to display serial consoles (green screen) to the HV IOCs. When you click "ioconsole" box in OPEN DESKTOP, you will see a screen link the figure below. Note that "ioc consoles" box in the OPEN DESKTOP menu is selected in the figure.

Each of those 4 green screens are "serial console" to each of 4 iocondevs. In the figure above, you see that iocondev3 is generating many Warning messages. This is not good. The message indicates that the iocondev3 failed to talk to LeCroy MainFrame #49 and #53, and this could be due to some hardware error. All other three iocondevs seems OK.


ALARM HANDLER

One of the task of a SA1 is to deal with HV alarms. Here is a link to A Guide to HV alarm handler. written by Stephen Pate.

starting up HV control GUI

Usually, the five iocondev desktops should have been set up and the HV control GUI should be running there. However, sometime you need to restart the HV control GUI. You can do this as follows:

  • Select the desktops. For example, iocondev3.
  • Open a terminal widow. This can be done by clicking the terminal icon in the OPEN DESKTOP menu.
  • Upon opening new terminal window, you should be at /home/phoncs23
  • ssh -l phoncs phoncs0 under the phoncs account.
  • phoncs0:~> source epics/scripts/setup_epics
  • phoncs0:~> iocondev3

  • In the last step, you should use command iocondevn (n=3,4,5,6,7) that is appropriate. It starts up the HV control GUI of the corresponding iocondev.

    Rebooting IOCs

    Sometimes, an IOC is crashed or hang, and you need to reboot it.

    The first thing you (shift crew) should do is to copy the last 5 - 10 lines from the terminal of that iocondev, save it into some file, and send it by E-mail to Ed Desmond (desmond2@bnl.gov) for debugging. Then, you can reboot the iocondev.

    The rebooting of an iocondev can be done by

  • cntr-X

  • in the console window of the IOC. Here cntr-X is a standard reboot key of VX-works.

    After the IOC rebooted, type

  • < load

  • from the console window if necessary. The reboot script may have done this last step. See the last line from the reboot process. The last line can be
    	    #< load
    	    
    and this means that "< load" command is commented out. In this case you need to type "< load" by yourself to complete the reboot process.

    Opening the "green" console for IOCs

    Usually, 5 green window consoles for the 5 IOCs should be in "IOC consoles" desktop, as described before. If you do not find them in the desktop, you need to re-open the console. For example, you do not find the console window for iocondev3, and you want to open it. This can be done as follows:
  • Open a terminal window. (Click "terminal" icon in the OPEN DESKTOP Menu). You can use existing terminal window if there is one.
  • ssh -l phoncs phoncs0 under the phoncs account
  • phoncs0:~> ts iocondev3

  • The last command is for iocondev3. If you want to open the console to iocondev5, type "ts iocondev5".

    Note that this "ts" command works on other computers in the control room. But there is only one console window allowed for one IOC. If you issue "ts" command to IOC which already has a console open somewhere, the existing console window is destroyed and then a new console open in your screen.

    Hardware RESET of IOC

    It is rare, but it is possible that you can not reboot an IOC by "cntl-X" from the console screen. In this case, you need to reset/reboot the IOC by hardware reset switch. Go to the Rack Room next to the Control Room, and find the IOCs you want, and push the reset switch. There are two switches "abort" and "reset" in front of the IOC card. Push both of them. The location of the IOCs is written in 1008 PRR East Row section of this manual.

    Note: It is possible that for some reason, the IOC does not come up after the first reboot attempt. To try a second reboot, it is necessary to wait for ~2 minutes. There is a 'lock out' time after a reboot during which the IOC will not respond to an additional reboot attempt.

    HV Manuals and Web Pages

    ALARM HANDER GUIDE (by Steve Pate)

    For Run2001, Steve Pate wrote a guide to the HV alarm handler.

    Documentation and Web for RUN2000

    For Run2000, a few documents were written for the HV controls and they were linked in Run-1 manual. Since they are written for Run 2000, some of the content can be outdated, but they are still very useful.

    Lars Ewell developed the GUI and EPICS control of PHENIX HV control system. He has written a:
    Users- ( Postscript ([an error occurred while processing this directive], [an error occurred while processing this directive]) or PDF (347K, Thursday, 24-May-2001 13:53:05 EDT version))
    and Operations- ( Postscript ([an error occurred while processing this directive], [an error occurred while processing this directive]) or PDF ( 89K, Thursday, 24-May-2001 13:53:05 EDT version))
    manual for the HV system. Those manuals are written for Run2000, and therefore some of the content is outdated, but they are still very useful.


    Johann Heuser has set up a Web Page entitled "PHENIX Drift Chamber High Voltage Control" but it contains more usefull information than the title implies ;-)
    TOF has a Web page and a PDF file (340K, Thursday, 24-May-2001 13:53:07 EDT) for their HV system.

    Original Run-00 version by A. Franz. Achim@BNL.GOV
    Run-2001 version edited by Y. Akiba akiba@BNL.GOV