DOMMB Testing Plan

Azriel Goldschmidt, LBNL
Oct 9 2003

In the coming weeks we will start the testing of the DOMMB V3 boards (16 boards).
This testing plan will evolve and get ironed out by excercising it with this batch of boards
(and possibly one or two more pre-productions runs to come). Please send comments
and suggestions to make the testing plan (and/or this document) more useful and correct.

Goals (a prioritized list)

1. Select well functioning DOMMB boards (including delay board, assumed throughout)
suitable for integration into DOMs to be deployed in IceCube/IceTop.
2. For boards which are not well functioning, provide some information to aid in
understanding/debugging/fixing.
3. Provide some DOMMB characterization.

Infrastructure
0. Space: The control room at the Bevatron (second lab from Gerry's door)
is available for use (thanks to Bob S and Kevin Lesko). We should start clearing
the room to begin using it by the third week of october. Do we want to fix the roof
and make the place less depressing? This will work as the main testing location.
Temperature cycles could also be made in that room.

If the control room is insufficient, we can use part of Gerry's lab space
for storage of DOMMB's, etc.

1. Computers:
 1. DOMHUB computer being sent by Darryn from Wisconsin: expected by Oct 10.
 2. Computer for DOM testing, with nice big monitor, should this be a Linux or Windows?
     It has to be able to run Jim Brown GUI STF stuff (find out), and whatever
    we use for database query: Need to purchase machine.
 3. Computer for software downloads and for other miscellaneous uses (second person in
     the testing room). would prefer TWO parallel ports to load both PLD and Software/firmware
     (or a PCI or ISA card with an additional parallel port).
     (do we envision loading PLDs at LBNL?): this computer could be found in someones
    junkyard? It should run windows (for altera and xilins downloads)

2. Cables:
 1. DOM to DOR (define length after the setup in the room is thought out.)
 2. Power Supply to DOR (ask Kalle)
 3. (terminal server cables, if not using DOR)
 4. (Cow to DOM cables, if not using DOR).
 5. a serial cable from loading PC to DOMMB serial input.

3. Software Loading cables: Ask Gerry about getting cables like the ones at the lab.

4. Special Key Connectors: Ask Gerry to get additional key made.

5. Connectors:

6. DOR cards:  1 + 1 (if we want to do heat cycles thru DOR cards, why not?)
Here, assuming 8 DOMMBs being tested "at a time" and that we can test using
two DOMMBs per pair: Is this reasonable?

7. Terminal Server: Hopefully just as Backup initially.

8. Power Supply: 1 for DOR-DOM for at least 8 DOMs (Kalle just bought it)
                             1 for loading software on DOMMBs, etc
                             (1 for long heat cycles? or if all tests thru HUB first power supply needs to be bigger).

9. Environment chamber: Shop around for 8-Board enviroment chamber. It would be nice
 if this is big enough to fit a fully assembled sphere (DOM) for specialized tests at cold temperatures
(a mini-wisc).

10. Delay Boards: Need 1 per DOM tested (and it will travel with It) Jerry-Wisconsin.
 (where do the DOMMB and DLB get married?)

11. "PMT" Terminators: Prepare ~10, 100-Ohm terminators. Possibly attach the terminators
    to the power cable to avoid loosing them or forgetting to plug them. Preferably the connector
   is of the type that the PMT connects thru.

12.Ethernet Hub.

13. PMT with base, for functionality test of base control. One of the two currently at Gerry's.

14. DVM

15. Digital Scope (?)

16. Basic Toolbox

17. Antistat Mats.

18. Rack/table for testing of boards

19. Rack for storing boards

20. Rack for holding boards in enviroment chamber [+65,-40] Celsius

21. Cow for power distribution for TS path/temperature cycles. Do we need a new one?

Setup

Preferably, only DORs should be used for testing/temperature-cycles because
it provides a full control of power/communications, however, it seems that we are not fully
ready to drop the TS pathway (besides readiness, the only argument I know of
is the problem of testing the DOM COM DAC -some ideas exist using an amplitude-modified
RapCal, if we do this is the "quiet" time within the wire pair ensured by the DOR firmware?).

o If we use DORs, can we have two DOMMBs per pair? are there issues with the fact that
if one needs to be powered off both DOMs get powered off? It seems that threre should not
be problems with running on 2 DOMs per wire-pair, as in the DAQ implementation.

1. Setup for testing with Only DOR Path (assuming 2 DOMs per wire pair):


o Added 100 Ohm terminator for PMT  "emulation"
o A single PMT-Base will be used to test the Base interface.
o A single flasher board  will be used for the flasher interface.

[Once the testing operation has ramped up, one can envision having a number, say 8,
PMT-Base and flasher boards to streamline the testing of these interfaces.]
 

2. Setup for loading PLD and Software/Firmware

Hardware: Loading PC + special load cables (with key), a power supply cable (50-80 V) always On
(connected to a Cow channel or DOR or separate PowerSupply). Serial Jumper on DOMMB.
Serial cable from DOMMB to PC.

Software: altera and xilinx loading programs. Hyperterminal to verify boot.

3. Setup for temperature cycles

Hardware: Environment chamber and OTHER STUFF: Depending on procedure (see below)

4. Setup for Cold testing

5. Setup for testing

Procedures + Flow

  1. All testing work to be done with antistat wrists for good grounding.  Sticky mat at the entrance to the room.
  2. Only qualified / tranied individuals shall perform these procedures. Bob Minor will be in charge of  safety including proper training requirements.
  3. If any soldering is needs it has to be done with soldering irons with well grounded tips.
  4. Unpack DOMMB -it should already be matched to a Delay Board-, if not, connect to a Delay Board. Add Delay Board ID to traveler and/or db.
  5. Paper record or traveler:  at this point it would be good to bring the database to include the info in the paper traveler. 
  6. ID tag (human readable): verify that it is on the board. Add to traveler and/or db. If it is not there or not readable: what do we do?
  7. Visual inspection: Follow the instruction (to be filled by gtp). If it does not pass the visual inspection, put in dedicated "failed visual inspection" pile and record in traveler and/or db.
  8. Move board to "Setup for PLD and Software Loading".
  9. Connect to power (special jig for safe powerup). Read power/current meter from power supply. Add info to traveler and/or db.  If current is below X1 or above Y1 put in dedicated "failed initial test of power" pile and record in traveler and/or db.
  10. Connect dedicated cables for PLD and Software load. (need clear labeling , if ambiguous connectors). Check or add jumpers needed for Load?
  11. Load PLD using iMpact (?) program. Need instructions to know where to find the correct default PLD design.
  12. Hit the hard reset reset button after load (label of button?). If PLD load fails put in dedicated "failed PLD Load" pile and record in traveler and/or db.
  13. Load config boot program (need instructions, but it can be done from dos prompt, I think).
  14. Load all the software into flash memory chips with the altera program  (from a DOS window) need detailed instructions and what file to download. If Software load fails put in dedicated "failed Software Load" pile and record in traveler and/or db.
  15. Connect serial cable from PC to DOMMB. Put jumper needed for serial comms. Reset  (hit the reset button) the board to reboot into iceboot.
  16. Using the Hyperterminal program verify that we get boot prompt. Need instructions (is it just hitting return?). If no boot prompt is obtained,
     put in "failed Initial Boot" pile and record in traveler and/or db.
  17. Get the PLD and software version off the iceboot (how?)
  18. Read power/current meter from power supply. Add info to traveler and/or db.  If current is below X2 or above Y2 put in dedicated "failed postboot test of power" pile and record in traveler and/or db.
  19. Next we want to perform (in the most automated possible way) any tests that cannot be perform in the DOR system [maybe we can add a DOM DAC test in the stratup scripts?]. If whatever tests made at this point fail put in dedicated "failed post loading tests" pile and record in traveler and/or db.
  20. Burn-in/test/powercycle cycles will be comprised of 7 (this number needs revision) identical sub-cycles, a day long each.


Cold testing/Heat cycles/Loading/Initial boot prompt/PowerConsumption/Full-testing thru DOR/

Testing thru TS?/HV-control/Flasher-control
Handling Instruction: grounding/antistat/mechanical stresses/power-on-off, etc.

Flow: Beginning 3/04 expect 55 boards a week. Expect 2 weeks board processing time.
Need capablity to test (burn in , etc) 70/week

Nominal burn-in cycle: 5 days long. 10 cycles from +65 to -40 C. These should include
power off cycles. What testing? How?

Here's the flow chart from Bob Minor which will be the starting point to define a test procedure:

Ed's Version of the Burn-in phase

Staffing

1. Techs:
2. Students:
3. Physicists:

Software
 GUI/scripting version of STF:
Jim Brown: GUI application to replace Arthur's web STF environment.
Some version of it running in Dave Hayes'. Simon and Jim sorting out
what packages it uses for standarization.

 DataBase capability:
to be added (easily) to work with Dave Glowacki's.

 DOMHub application: used to access the DOMs on DOR cards at testing.
Dave Hayes' relelease?

 STF tests:
Need to test the existing tests.
Need toexpand the list of descriptions and tests coded.
Need to match list with requirements DOC.