EDOC122

Centre d’Etudes Nucléaires de Bordeaux Gradignan

EUROGAM PROJECT

VxWorks in the Control System and Data Acquisition of the EUROGAM Detector

Edition 1.0 March, 15th 1991

P. KADIONIK, M.M. ALEONARD

CENBG-CNRS-IN2P3 France

(33) 56.89.18.00 1 General overview :

1.1 Presentation :

The EUROGAM multidetector array is built in a French-UK collaboration. This is a system of 70 Ge diodes, each of these detectors is surrounded by 10 BGO crystals. This array of detectors is used to detect gamma radiations emitted in a nuclear reaction and measure their energy and time correlation. The general scheme of a data acquisition system in nuclear physics is shown in figure 1 : A high energetic beam of particles is sent on a target. Radiations issued from the nuclear reaction are collected by the detectors (EUROGAM will detect gamma radiations). Pulses ouput from detectors are then shaped and digitized. General data processing is applied before on-line analysis and storage...

1.2 System overview :

The electronic units necessary to process the analog signals ouput from these detectors are software programmable. Cards have been designed in France in the new VXI standard (VME Extensions to Instrumentation). Analog outputs from all these detectors are digitized in the VXI crates. A fast on-line selection of these numeric data is realized by the Master Trigger module in order to select the raw events. A raw event is made up of all digitized data collected via hardware units from the various VXI crates after trigger validation. The raw events will be formatted after data reduction by the Event Builder processors (slow on-line filtering). The output data of this unit form the formatted events. An on-line control is supplied : the on-line histogramming of data counts versus their energy or time is built on-flight from the raw events in the Histogrammer crate with a hardware VME Direct Memory Increment unit. Formatted events issued from the Event Builder crate are saved in a list mode on Exabyte tapes for off-line analysis (further delayed in time). Some formatted events are sorted on-line and their histograms are built in the Sorter crate.

2 EUROGAM architecture :

The EUROGAM global architecture is shown in figure 2. It is a system distributed on an Ethernet network. All the CPU cards connected to Ethernet are MVME 147 cards. These are the Crate Controller for eac Crate (VME or VXI crate) connected to the network. The Ethernet network is used to download tasks and insure the communications between the Crate Controllers and the whole architecture. The DT32 bus (32 bits bus in ECL logic) designed for EUROGAM project and t optical link allow fast data transfers between crates.

2.1 VXI crates :

Each VXI crate contains :

- a Ressource Manager card based on VXI standard (MVME 147 card coupled with an interface to VXI designed in UK). This Ressource Manager is the Master in the VXI crate and has also an Ethernet link.

- Instrumentation cards shape pulses issued from the detectors and digitize them.

- a Readout Controller card (ROCO) designed in France is in charge of fast data transfer on DT32 bus (rate of 256k to 1M 32 bits words/s).

The Master Trigger card is located in the first of these VXI crates. It is responsible for the general control and sequential readings of all data coming from the Instrumentation cards towards each ROCO card (these form the raw event).

2.2 Histogrammer crate :

The role of this VME crate consists in the on-line histogram building of the current experiment. This histogram built in VME memory is used by workstations to check the experiment and control the detectors quality. The histogramming unit is software programmable to select specific parameters from choosen detectors and build their histogram via a Direct Memory Increment in a global dual-port memory (VME/VSB).

2.3 Event Builder crate :

The Event Builder basic functions is to format raw events and also filter them. A software selection is realized in order to filter raw events according to physical expectations and reduce the Event Builder output data rate. The input data to the Event Builder are coming via a DT32/HSM interface (HSM 8170 from CES). These data are processed by one of the MVME 165 CPU cards working in parallel mode on different sets of data. The reformatted events are then sent to a FIC 8232 card (CES) which transfers the formatted events through an optical link to the Sorter crate (6 Mbytes/s rate).

2.4 Sorter crate :

Among all the events coming in, only selected ones have special features for the physicists towards their experiment. The Sorter unit is able to select these specific events and enhance their features for control purposes. The Sorter unit has a hardware architecture similar to the Event Builder unit with MVME 165 CPU cards running in parallel mode in order to cope with the data rate.

2.5 Storage crate :

The Storage crate saves all formatted events on Exabyte tapes. Four Exabyte tapes will be used, each could work at a 512 kbytes/s rate. Histograms built in Sorter or Histogrammer crates may be saved on these tapes too.

2.6 General monitoring procedure :

Before starting the acquisition, all parameters must be set up : HV detectors power supplies, VXI instrumentation cards parameters... It represents thousand of parameters values to set up. In this distributed architecture, a general database reachable through the Ethernet network is used to set the local databases. The local databases are handled by the Crate Server software located in the Crate Controller cards. After the power-up phase, all the necessary set-up parameters values are sent to the Crate Server software. Start or stop orders are sent in the same way to the Trigger or other parts of the system (Event Builder,...). The data acquisition control is managed from workstations which get on-line collected histograms from the Histogrammer or Sorter units through the Ethernet network. These histograms (or spectra) are transferred via a Spectrum Server software.

2.7 Visualization :

Graphic workstations (SUN workstations) connected to the Ethernet network are used to set-up, start, control and stop data acquisition and allow to visualize the on-line histograms built in the Histogrammer and Sorter crates.

3 Software overview :

3.1 A distributed system :

The software as the architecture has a distributed structure. This system is based on Master/Slave relationship. Each crate is dedicated to a particular functionality in the acquisition line. MVME 147 CPU cards are nodes in the Ethernet network. These are the Crate Controller cards used as Crate Servers and are gateways for the other CPU cards in the same crate. Each Crate Server contains several software processes : a spectrum server, an error manager, a set-up software, control and diagnostic processes in relation with the local database. A general server software will route the procedures to the appropriate Crate Server(s) through the Ethernet network.

3.2 Choice of a Real Time kernel :

This acquisition system must minimize the dead times during the data acquisition. The data flow is almost continuous and each detector ”sees” on average 10000 events/s. The data management is based on use of a Real Time kernel. This Real Time kernel must be implemented in all CPU cards. It is to be performant and complete, able to support efficient network communications. It must be easily configurable since several CPU cards are used in our system and they are from various manufacturers. Besides, some of the application tasks are to be often reloaded depending on the evolution of the experiment. This must be done easily by the physicist. There are two aspects in Real Time applications in our project : an efficient Real Time kernel is mandatory in the Event Builder and Sorter crates. For control and set-up this aspect is less important but the communication aspects take the lead. TheVxWorks Real Time kernel has been choosen for these criteria and for its cross-development partnership with UNIX.

3.3 Communication procedures :

The VxWorks environment offers the UNIX environment facilities for the programmation and debugging. Software are mainly written in C. However assembly language 68xxx may be used in specific cases for efficiency reasons, or higher level languages like Fortran. Communications are essential in this architecture. The Remote Procedure Calls (RPC from SUN) are based on the Client/Server model. This is the general communication mechanism between master and slave CPU cards. A more general RPC library is written to be more suitable for specific applications. RPC will principaly ensure control operations and information exchanges between the different Crate Controller cards and the control workstations. Communications in the same crate will be possible by the VME or VXI backplane between the Crate Controller card (master) and the other CPU cards (slaves). This is a very interesting feature from the VxWorks kernel. However the efficiency of these communications are under evaluation for in-crate communications in a Real Time context.

3.4 Set-up and control procedures :

At power on each crate will be set in a given state : for example the VXI Crate Servers will be able to give a status and mapping of all the Instrumentation cards in their crate. In a second step, the set-up phase, these informations will be compared to the expected ones from initialization routines governed by the physicists. The set-up processes are to load all the necessary parameters (about 1500) in all the Instrumentation cards. The procedures are not time critical. Once the set-up phase is performed, the acquisition start-up is possible. At thi point the sequential ordering in time of execution of the start-up procedures is crucial for the goodness of the data collected : one task will be in charge of this start-up. When the acquisition is turned on, the Real Time performances of the software are requested in all processes dealing with the events coming out from the Instrumentation cards. The rate of events that could be handled by the Event Builder processors could be as big as 1M 32 bits words/s in the upgraded version of EUROGAM. In parallel to this events management, storage controls are to be implemented to check the quality of the experiment. Amoung the procedures, counting rate checks of the detectors insure that all the detectors and electronic performs well. These checkings can be done through the integration or analysis of histograms built on-line and on-flight.

3.5 Event Builder software :

The Event Builder software collects the raw events from several crates and builds a new formatted event with eventually new parameters for further storage. This filtering processes of the data provided by the High Speed Memory (HSM 8170 card) will be duplicated in several MVME 165 CPU cards. These filterings will be controlled by one arbitration task which will also direct the filtered events towards the optical link transfer process. The arbitration unit needs also have control of the data rate managed by the various CPU cards and error management.

3.6 Sorter software :

The organisation of these functions is equivalent in structure as the Event Builder one with arbitration task and parallel processes dealing with sequential data records. The sorter software is a further filtering of events to build histograms in one or two dimensions. Parameters for this filtering are often changed. These changes need reloading procedures easily implemented.

3.7 Test procedures :

A general tests software has been written to allow several basic tests of memories and registers on all developped cards. It is a simple user interface between the board designer and the VxWorks environment. Specific procedures are developped for testing the data path consistency too.

4 Conclusion :

The first phase of the EUROGAM project will be in operation for the end of summer 1991 in Daresbury (UK). The multidetector will have just 45 Ge and 450 BGO detectors. The upgraded phase is expected for the end of summer 1992 in Strasbourg (France) with a final configuration of 70 Ge and 700 BGO detectors. EUROGAM hardware and software structure will also be implemented on other multidetector arrays in design in France : DEMON, ICARE...

The software development for EUROGAM result from a collaboration of ingeniors working in several laboratories supported by IN2P3 in France and SERC in UK. These laboratories are CENBG (Bordeaux), CRN (Strasbourg), IPN (Orsay), CSNSM (Orsay) in France and Daresbury Laboratory, Oliver Lodge Laboratory in UK.