University of Liverpool Nuclear Structure Computing Support Group Exogam Event Builder Specification

EDOC412


Exogam Event Builder Specification

This document is the result of discussions at GANIL between
Bruno Raine, Frederic Saillant (GANIL)
and John Cresswell (Univ. of Liverpool)

Version 1.0 July 1998

Introduction

The aim of this specification is to describe components of the proposed Event Builder for Exogam that would satisfy the requirements laid down in the Exogam Project Definition.

The essential requirement is that the system shall be able to process event data at upto 2Mbytes/second.

A second requirement is that the solution uses some of the technology and lessons learnt from the Eurogam/Euroball implementations. The user interface for setup and control of the data acquisition will use the MIDAS package. This places a requirement that the Event Builder needs to run a register server task for communication purposes. Transfer of formatted event data blocks to a Eurogam-style tape server implementation is also required.

The Eurogam Event Builder system consisted of several cpus in the VME crate, with each one allocated to specific tasks. This was largely due to the lack of performance available from cpus those few years ago, and to the simply nature of the VxWorks kernel. However, it may still be useful to split the functionality of the Event Builder system between more than one cpu.

The system is responsible for several functions. Not only is it responsible for the flow of data from the DT32 bus, through event formatting and outputting to the tape server, it has to provide parameter setup, control and monitoring functions.

Hardware components

The Event builder will consist of a VME crate containing the following modules ...

It was decided that for reasons of processing efficiency, that at least 2 cpus should be used. It is intended that the software task(s) involved in event checking and formatting should possibly be separated from those that supports external command access and the interrupt-driven input and output drivers.

However, it is intended that all software tasks should be able to run on any cpu in the system.

It is thought that one cpu should be sufficient to cope with event formatting. However, the software will be designed with the flexibility to add a further cpu for event processing if necessary.

Software components

The basic components of an Event Builder system are ...

Each of these components is described in the following sections.

Register Server

The Register Server is the prime communications path between the user and the system code. All operations applied to the crate, whether it be GO/STOP or reading statistics, exist in the form of "registers".

A register is normally conceived to be a bit of electronics (in a cpu chip for example) that stores an integer (or floating point) value. A Register Server "pseudo-register" is an extension of this idea. It may be a memory location that can be written and/or read, but it may be something more complicated like a function that can be executed and return a "condition code".

The Eurogam Register Server core has been ported to Lynxos. The addition of Exogam-specific registers will be necessary.

The Register Server code usually runs in only one cpu (the so-called "crate master") in the crate.

Internal task communications

Each task in the Event Builder will need to be able to receive and possibly send command messages. This requires a protocol definition.

For command transfer within the same cpu it is intended to use standard Posix message queues which GANIL already use.

For command transfer between tasks in different cpus a protocol will be defined, using a FIFO or mailbox interrupt.

For inter-task data sharing we will use the GANIL Equipment package. This allows blocks of memory to be allocated, and made accessible to tasks running in the same or different cpus.

Event data input

The D2VB module will provide the input data stream from the VXI crates via the DT32 bus. The module has 2MB of memory that can be sub-divided into many buffers. When one buffer fills, the controlling cpu can be sent an interrupt and the hardware will automatically switch to the next free buffer. With sufficent performance further down the line, this unit will not contribute any data acquisition deadtime.

Event data output

The crate interface to the tape server will be a PMC daughter board which connects to one of the cpus. This card could be either FDDI or Fast Ethernet. Both would be fast enough to cope with 2MB/sec of data. Fast Ethernet would certainly be cheaper, particularly for spy workstation interface cards, but its smaller packet size would place a greater load on systems. It is intended to do some tests to investigate whether Fast Ethernet is acceptable.

Formatted blocks of event data will be output as UDP packets in a format compatible with the tape server documentation.

Device Drivers

In a multi-tasking virtual memory environment like Lynxos, device drivers become very important. Interrupt routines cannot normally exist outside of the kernel, and device drivers provide a neat way of communicating between user-space tasks and I/O ports.

The CES enhancements to Lynxos provide a mechanism of connecting a routine in a user-space task to an interrupt. In this way a kernel-space device driver would not be necessary.

There are several device drivers required for the Event Builder. These include control of the DT2V and FDDI interfaces, and maybe command passing between cpus in crate.

Buffer queuing

The system needs to maintain a set of buffer queues to control the flow of data from the DT2V to the FDDI output interface.

GANIL already have a buffer management package which will be adequate for our purposes.

The buffer queues must be accessible by tasks in the same or different cpus.

Consideration needs to be given to tidying up the queues on stopping data acquisition (flushing partially filled event blocks), and on task termination (which should not normally occur).

Event Processing

A user space task will be responsible for receiving blocks of event data, processing them and outputting formatted event blocks to the FDDI interface driver.

This task could function using shared memory tables generated by register writes. These tables would provide initialisation data - e.g. lists of detectors and data words currently enabled.

It is important that the event checking code be provided with the current list of enabled parameters to test the validity of the event data stream.

Data Checking

The raw data blocks are checked to make sure they conform to the expected format generated by the VXI boards.

Further, each sub-event can be checked to make sure it is part of the same event, and that the VXI readout from each crate has not got out of synchronisation.

The data addresses can be checked to make sure they are unique and in the set of expected addresses. Checks can be made to ensure that any expected data is not missing, and that there is no unexpected data in the event.

The event data stream received from the VXI hardware via the DT32 bus is optimised for hardware readout and control. It has useful features for detecting if the hardware is not working properly. VXI data historically has been prone to containing a small level of data corruption. It is a useful service to the experimenter if these features are reported and removed.

We may also need to provide a mechanism to alter the action subsequent to finding a data error. The possible actions would be to discard the data word, discard all data words associated with the same detector, or discard the whole event.

Event Format

Event data from arrays such as Eurogam/Euroball are received as a mass of ADC words of varying types (Energies, TACs, patterns etc) There are two aims of a successful event format, The first is to write out the event data in a compact manner, so as not to waste tape capacity. The second is to associate the many data items with the originating detector. This makes it possible to use the symmetries involved in the array to make data access easier.

The Exogam segmented-clover detector with surrounding BGO is a very complicated detector, and some thought needs to be given to the way it is formatted.

It is intended to use the Euroball event format. The Euroball format specification is complex and contains a super-set of the formatting features required. Suitable subsets of it will be chosen. The exact details of the format will be fixed and published later so that all experimenters will have a clear idea how to access event data.

It may be possible for Exogam to have a fixed output event format. This would mean defining the format for every detector, and possible extra ancillary detectors in advance. This could be achieved by passing to the event processing code information to enable the set of input data words to be mapped into the defined event formats. Some thought needs to be given to this area, as it would provide for easy addition of ancillary detectors.

Controls

Historically, all control within the crate has been channelled through a single cpu running the Register Server task. Then, any commands can be applied in a systematic manner to all the necessary components in the crate.

For example, if the "GO" command were to be received, then the first check would be to see if the crate as a whole were in the right state to receive such a command. Assuming all was well, then the "GO" command would then be applied to the output controller, the event processing module and the input controller in sequence.

In this way, the Register Server, acts as the crate controller and always knows the general state of each component.

The register server contains a set of registers to provide system control. On the basic level there are register commands available for starting and stopping data acquisition. These will include code for ensuring each start is from a "clean" state. This means that, for example, any partially filled data buffers are cleared out on stopping.

In addition to commands for allowing the transmission of data to be separately controlled for each part of the event builder system, the status of each task in the system will be available (whether its waiting for data, sending data, running ...) via the register server. These will be useful diagnostic tools.

The default state will be for the system to be enabled with data transmission activated all the way through to the tape server. For example, if the tape server status window indicated that no data had been received, then the following checks could be made.

  1. Do not allow data transmission beyond DT2V driver. This should allow free running of the hardware, and checks can be made to see Fast Triggers are present. It also checks correct operation of the DT32 bus. Statistics for data blocks input to DT2V driver should be incrementing, but zero thereafter.

  2. If all is OK above then allow data through to event processing but nodo not allow data output to FDDI. This should provide further statistics about the quality of the hardware data blocks.

  3. If the quality looks bad, then there may be a hardware fault. If the quality looks good, and statistics are incrementing then the fault may be in the output data link. Further tests can then pinpoint the exact problem.

Statistics

Each component of the system will need to contribute to the available set of statistics. The D2VB handler will count every data block that comes from the DT32 bus. THe handler writing formatted event blocks to the tape server, will have a separate counter.

The event checking procedure will require many statistics for the various error types possible in the VXI data stream. These include corrupted data tokens, duplicate words etc.

Statistics will be programmed as 32bit counters, contained in shared memory locations in a memory object. Each statistic will be associated with its own "pseudo-register" in the register server. The available operations are read and clear (on write).

Additional Event Building Functionality

The modular design will allow the addition of extra components into the system to allow for data modification, controllable as options. Such components would need to be defined in advance as they would be hard-wired into the code, and in order to benchmark cpu performance.

For example, if online gain-matching is deemed necessary, then suitable code can be added between event checking and event formatting, prior to output to the tape server. This would be controlled via windows accessible as part of the graphical interface.