DART Project RS-485 Repeater Module --PRELIMINARY-- Wednesday, March 08, 1995 Revision 2.1 John T. Anderson Greg A. Deuerling Electronic Systems Engineering Department Computing Division Fermi National Accelerator Laboratory The RS-485 Repeater Module is designed to allow the extension of DART data cables past limits imposed by cable losses, number of loads and load distribution; as a side benefit it also simplifies termination of the DART data cables in complex systems. The Repeater takes RS-485 data in on its Upstream (input) side and regenerates the same data on the Downstream (output) side with a regenerated Data Strobe. In order to allow a wider variety of architectures the Repeater participates in Permit token chains such that the Permit chain may flow through the Repeater to multiple modules behind the Repeater. Two basic Repeater architectures exist. In the first, a cluster of Data Sources hides behind a Repeater, which is the sole driver of a piece of cable which terminates in a Data Destination. In this architecture, the Repeater need regenerate and re-drive the data and all control signals, but the Repeater need not concern itself with the Permit token which circulates between the Data Sources. In the second architecture, Data Sources exist on both sides of the Repeater such that the output side of the Repeater must share the data cable with Data Sources. In this second architecture, the Repeater must participate in the Permit token chain so that the Data Sources present on the cable between the Repeater and the Data Destination may take control of the cable in turn. Both of these architectures are shown in Figure 1 below: Figure 1 The Repeater receives data words at rates up to 10 Mhz (one word per 100 nsec). Data is internally latched and the Data Strobe signal is reshaped and retimed to clean up any skew or amplitude loss. In addition, trigger signals are received and redriven. Status indicators (LED’s) on the front of the Repeater give visual indication of data transfer. Logic state analyzer connections, also located at the front, allow diagnosis of data transfer errors without having to break the data cable and insert the test equipment. The Repeater is packaged in a 1U high, 19-inch wide rack-mounted box. Power is supplied through a standard 110 VAC line cord. Status indicators are on the front panel. DART cable connectors, fuse access, power cord and diagnostic connections are located at the rear. 1.1 System Introduction DART consists of a set of Data Source modules including the Fastbus Smart Crate Controller (FSCC), Crate Trigger Interface/Readout Controller (CTI/RC) and Damn Yankee Controller (DYC). These Data Sources drive differential RS-485 data over a cable to a Data Destination module, the Access Dynamics DC2 (and it’s daughter board, the DM-115). A series of Data Sources may be linked together on the same cable (a “stream’) to in turn drive the Data Destination, and this cable may connect to multiple Data Destinations, each one of which is resident in a unique VME chassis. Commonly, data flow is defined as progressing “downstream” (from data source towards data destination) or “upstream” (from destination back towards source). These terms will be found throughout this and other DART system documents. The data flow is carried on a parallel differential cable using RS-485 signal levels. The internal data path is carried on two parallel ribbon cables, one of 34 pins and a second of 50 pins. The organization of the cables is as follows: 1) Low 16 data bits plus control signals on the 50-pin connector; 2) High 16 data bits on the 34-pin connector. The control signals defined are: 1) DSTROBE (Data Strobe) is pulsed up and down once per data word sent from Data Source to Data Destination. 2) EOR (End of Record) is pulsed up and down once per block of data words sent from one or more Data Sources to the Data Destination, and is used to delineate ‘events’ or ‘blocks’ of data. 3) WAIT is asserted by the Data Destination to signal the Data Sources that transmission of data should be halted. The WAIT signal propagates backwards along the cable with respect to Data, DSTROBE and EOR. A DART system and it’s relationship to other systems within an experiment may be shown graphically as follows: Figure 2 In response to triggers generated from the Trigger System, the Data Sources take data from the front end Converters and pass it down a cable to the Data Destination. If required, Repeater modules are used to insure that the data arrives intact at the Data Destination. The Event Builder takes the data from the Data Destinations and uses it to build and store complete events. Multiple Data Sources are allowed on a single cable. Data Sources use a Permit token signal to pass control of the data cable amongst themselves. For this Permit scheme to function, one Data Source must be the ‘first’ and one other must be defined as the ‘last’, with all others defined as ‘middle’. The ‘first’ will be the first to send data, and the ‘last’ carries the responsibility of sending the End of Record signal to the Data Destination. ‘Middle’ and ‘last’ Data Sources wait to send data until the receipt of a Permit token. The Repeater must be able to respond to the Permit token in similar fashion to the Data Sources as the cable position of the ‘first’, ‘middle’ and ‘last’ Data Sources is not predetermined. The Repeater must be cognizant of the state of the Permit token on both its “downstream” and “upstream” sides to correctly control the tri-state enable of it’s output data drivers. To accomplish this, a Repeater contains Permit In and Permit Out connectors on both the data input and data output sides, allowing the device to monitor where in the system the token is at all times. 1.2 Description Of Component & How It Fits Into The System Repeaters are used to control signal reflections and restore lost signal amplitude, whether caused by total cable length, number of or distribution of loads on cables. Conceptually, a Repeater is a Data Destination (on the “upstream” side) connected by an internal cable to a Data Source (the “downstream” side). At present, only one form of Repeater exists, the Data Acquisition Hardware Group’s FOCEX system, which provides a fiber-optic based extension of up to 1 kilometer per repeater set. The FOCEX requires that all Data Sources be on the input side of the FOCEX system and that the FOCEX output directly drive the Data Destination as the only Data Source. In addition, the FOCEX is rather expensive and does not connect to or understand Permit logic. In order to accomodate those architectures in which the Data Sources are more spread out, and also those which do not need the full extension length of the FOCEX, the Repeater has the following basic characteristics: 1. Ability to extend the DART data cable; 2. Ability to provide termination on it’s Input (upstream) side to contain reflections; 3. Participation in the Permit chain logic such that the Repeater may have Data Sources on either side of the Repeater; 4. Simple, low-cost construction; 5. Ability to stack multiple Repeaters on a single DART data cable. 1.3 List Of Component Requirements The Repeater must provide the following minimal function set: Ability to receive and regenerate 32-bit data; Ability to receive and regenerate the Data Strobe; Ability to receive and regenerate the End of Record (EOR) and WAIT signals, with the provision that the End of Record signal is only driven if the ‘last’ module of a series of Data Sources is located upstream from (farther away from the Data Destination than) the Repeater. Participation in the Permit logic chain by passing the Permit In and Permit Out signals between input and output sides of the Repeater; in addition, the RS-485 drivers on the output side of the Repeater must only be active when the Permit token is active on the input side of the Repeater. The Permit logic of the Repeater must be able to be disabled such that the Repeater is always active, to allow the architecture in which multiple Repeaters are hooked up in series to drive a very long cable with no intervening Data Sources An ability to provide easy connection of a logic state analyzer to assist in debugging data transmission errors in the field; An ability to regenerate trigger strobes and trigger ID information to assist in the distribution of trigger signals over long data runs. 2 THEORY OF OPERATION AND OPERATING MODES 2.1 Basic Features & Operation (Including Block Diagram) A block diagram of the Repeater is shown here as Figure 3: Figure 3 a) Data and Data Strobe. The unit will receive the RS-485 data from the “upstream” side, latch it with the received data strobe, and regenerate a full- width data strobe using local logic. All data bits received are output simultaneously to the “downstream” side, and the output data strobe is timed with respect to the data such that setup and hold times are observed. The data strobe will be cleaned up upon input and will be used to fire a delay line circuit to regenerate a new data strobe on the “downstream” (output) side. The Repeater will create a 50 nanosecond wide output data strobe whose leading edge occurs no less than 10 nsec after the output data is stable. The timing relationship between the output data strobe and the output data is determined by a delay line and PAL combination within the module and is not user adjustable. A block diagram of the timing controller is shown below in Figure 4. Figure 4 b) Trigger Considerations In the DART data system a four-bit Trigger ID plus a Trigger Strobe is carried from the trigger system to all readout controllers via a 20-pin RS-485 cable. The top 10 pins of the cable carry the Trigger ID and the Trigger Strobe and the remaining 10 pins are unused. The 'first' readout controller is charged with taking the four-bit Trigger ID and placing the four-bit data value in to the header word (first data word of the block) so that the DC2 module may use it as an address. The DC2 has a switch pattern that is compared against the trigger ID. The DC2 whose pattern matches will receive the data, allowing for the data to be directed to different VME backplanes at the request of the trigger system. The Trigger Strobe must be carried to all readout controllers for it is the signal used to indicate that new data is present. In a general architecture where the Repeater is in the middle of the readout controllers, the Repeater must also regenerate the Trigger Strobe to insure that the Trigger Strobe arrives at all readout controllers in the system. To maintain signal integrity the Trigger Strobe is used to latch the Trigger ID bits and similar timing circuitry to that used for the Data Strobe insures the setup and hold time of the trigger information. A pair of 10-pin headers are used to support passing the Trigger ID and Trigger Strobe around over long RS-485 cables in addition to the data. In some applications the Trigger Strobe signal is NIM rather than RS-485, and so the Repeater provides, in addition to the RS-485 implementation, a NIM regenerator for NIM Trigger Strobe signals. Independent of the logic level used for the Trigger Strobe, the Trigger ID is always carried as an RS-485 signal. Data Sources also provide flow control signals that tell the trigger system that they are temporarily unable to accept new triggers. These signals, generically named ‘Trigger Holdoff’ signals, are typically NIM. The Repeater provides two NIM regenerators to allow simpler cabling of these ‘Trigger Holdoff’ signals, buffering from the Upstream to the Downstream side. c) End-of-Record (EOR) and WAIT The EOR signal, a pulse nominally 150 nsec wide, runs in the same direction as the data. This signal will be received and used to fire a digital pulse generator with a nominal duration of 150 nsec. Measurements of cable timing show that the EOR is always well apart in time from any data strobe and need not be correlated with any data values. A simple switch enables or disables the Repeater’s EOR driver. The switch is set to the ‘EOR Enabled’ position if the ‘last’ Data Source is Upstream of the Repeater. The switch is set to the ‘EOR Disabled’ position if the ‘last’ Data Source is Downstream of the Repeater. In the DART cable specification the ‘last’ data source is supposed to drive the EOR signal, and all other Data Sources should leave their EOR drivers in a high-impedance state. Two LED’s are connected to the EOR signal on the Upstream (input) and Downstream (output) sides of the Repeater, and flash when a pulse is seen. A flash on the Downstream side without a corresponding flash on the Upstream side signifies that a Data Source on the Downstream side has asserted EOR. A flash on both LED’s indicates that a Data Source on the Upstream side has asserted EOR, and that the Repeater has performed it’s required function. A flash on the Upstream side without a corresponding flash on the Downstream side is indicative of a failure within the Repeater itself. A reverse-direction signal named WAIT is also found on the cable as driven by the DC2. The driver for WAIT in the Repeater is always enabled. WAIT is asserted by the DC2 whenever the DC2 local FIFO is half-full. To minimize jerky, stop-start data transfer, the Repeater uses a digital delay to guarantee that the WAIT driven by the Repeater on the cable is the longer of 500 nsec or the duration of the WAIT signal as driven by the DC2. This will have no measureable effect upon bulk data transfer rates and will serve to smooth transfers at high rates where WAIT may be asserted often. d) Termination and Biasing Issues. All RS-485 inputs will be terminated using a 240 ohm resistor across the differential input, with 120 ohm bias resistors on each leg. The ‘plus’ input is biased to 3.3. volts and the ‘minus’ input is biased to ground, yielding a net matching impedance of 120 ohms. To eliminate confusion, here in Figure 5 is a sketch of the termination and bias technique: Figure 5 Termination resistors on the Downstream (output) side are assumed to be supplied externally. e) Permit Logic The Repeater monitors the state of the Permit pulse on both Upstream and Downstream sides. The unit pulses Downstream Permit Out (DPO) upon receipt of the Upstream Permit In (UPI) pulse. Similar logic will drive Upstream Permit Out (UPO) from Downstream Permit In (DPI). A single internal flip-flop maintains the state of the data drivers such that the data drivers are enabled whenever the Permit token is Upstream of the Repeater. A switch on the front panel labeled ‘First Upstream’ and ‘First Downstream’ controls the power-on (post-reset) state of the Permit flip-flop. All Permit signals are NIM, using standard LEMO connectors for I/O. A ‘psuedo-schematic’ for the Permit control logic is shown here as Figure 6: Figure 6 Upon receipt of the Upstream Permit In, the Repeater disables it’s data drivers and produces the Downstream Permit Out signal which tells the Data Sources on the Downstream side of the Repeater that they may send data. Conversely, upon receipt of Downstream Permit In, the Repeater turns it’s data drivers ON and passes the permit back to the Upstream Permit Out. The power-on reset condition of the Repeater is controlled by the ‘1st Upstream/1st Downstream’ switch. 2.2 Diagnostic Features An internal, recessed header is provided that yields access to the actual signal lines connected to the Repeater. A small diagnostic card with internal series resistors plugs in to the recessed header through a slot at front of the Repeater. This has the dual advantage that either oscilloscope or logic analyzer connections may be made, and that the excess capacitive load is only present when diagnosis is in progress. A card guide mounted to the Repeater circuit board insures correct mating of the diagnostic card, and a cover plate guards the diagnostic connector when not in use. A 120-pin DIN connector is used which insures correct orientation. The diagnostic connector provides access to all DART data signals plus the Trigger signals. Different plug-in diagnostic cards may be produced if necessary to support either logic analyzer or oscilloscope viewing of the signals on the DART cable. 4 INTERFACE SPECIFICATIONS 4.1. Front and Rear Panel I/O, Test & Monitoring 1. Two 50-pin, two 34-pin and two 10-pin ribbon cable headers for carrying data, control and trigger information, respectively, located on the rear panel. 2. LED's indicating state of DataStrobe, EOR, WAIT and power, located on the front panel. Separate sets of LED’s indicate the state of DataStrobe, EOR and WAIT for the cable downstream and upstream of the Repeater. Signal monitoring LED’s are pulse stretched for easy viewing of narrow signals. 3. LEMO connectors for NIM-level Permit In, Permit Out for both input and output sides of repeater (a total of four),located on the rear panel. 4. LEMO connectors for NIM-level Trigger In and Trigger Out, also located on the rear panel. 5. LEMO connectors for NIM-level Trigger Holdoff In and Trigger Holdoff Out (2 pairs), also located on the rear panel. 6. Internal (recessed) RESET pushbutton in parallel with front- panel RESET NIM input (LEMO). 7. Internal (hidden behind panel) 120-pin DIN Diagnostic Connector. These are sketched here in Figure 7: Figure 7 4.2.1.1.Connector Pin Configurations DART data signals are carried on a 50-pin cable and a 34-pin cable as RS-485 differential signals. All signals are sent with the ‘+’ side of the differential pair on the odd-numbered pin of the connector, and the matching ‘-’ side of the differential pair on the even-numbered pin directly across the connector. A third 20-pin connector is used for connection of trigger signals, with the same polarity scheme employed. The pinout of the 20-pin connector is designed to match the FSCC readout controller. Two of each connector exist on the Repeater, one for the Upstream (input) side, and one for the Downstream (output) side. Pinouts are as shown here in Figure 8: Figure 8 4.2.1.2.Signal Descriptions DART Data and strobes on ribbon cable are RS-485. Permit In and Permit Out are NIM. The Trigger ID is RS-485; the Trigger Strobe is supported in both RS-485 and NIM signal levels. 5 ELECTRICAL & MECHANICAL SPECIFICATIONS 5.1 Packaging & Physical Size . The Repeater will be a self-contained 19", 1U high AC powered box using a standard prefabricated rack-mount chassis. Diagnostic information is presented at the front panel; all user connections are made at the rear. 5.2 PC Board Construction Six layer FR4 with ground and power planes. 5.3 Power Requirements Estimated draw of 3 Amps at 5 volts provided by internal power supply driven from 110 VAC, for a total power draw of about 20 watts. 5.4 Cooling Requirements Convection through air vents in chassis box. 6 SAFETY FEATURES & QUALITY ASSURANCE PROCEDURES 6.1 Module Fusing & Transient Supression AC line power fused at 1/2 ampere, standard 3AG fuse in panel mount holder. Internal 5 volts fused at 5 amps via board-mount picofuse with single Transzorb on board. Multiple types of decoupling capacitors mounted across entire PC board for noise rejection. 7 EXAMPLE OF COMPONENT OPERATION WITHIN THE SYSTEM The Repeater will be used in those situations where data transmission is made difficult either by total cable length, multiple loads, or reflections caused by clumps of devices in the middle of the cable. In all these cases a Repeater may be used to break the physical cable in to smaller, more easily controlled chunks, where reflections are minimized and signal strength kept high.