EDOC065

EUROGAM PROJECT

NSF DATA ACQUISITION SYSTEM Autofill Specification

Edition 2.0 January 1995

Nuclear Structure Software Support Group Liverpool University

1 INTRODUCTION

The philosophy of the new computer controlled autofill system is one which is intended to make it possible to ensure that the cryostats of a large variable number of (70) germanium detectors on the Eurogam array are filled at defined times and that the correct human intervention happens when necessary. For the large number of detectors foreseen, this will have to be in a way which allows sensible presentational sub-division into groups of detectors, e.g. grouping according to physical position in array.

The Autofill is a simple (semi-)feedback control system which purpose is to ensure that the detectors in the Eurogam array are cooled with liquid nitrogen. Filling of LN2 takes place at fixed programmable intervals and is stopped when exhaust nitrogen is detected. The filling operation is performed by activating solenoid valves through which the LN2 is allowed into the detector. Exhaust nitrogen is registered by a temperature sensor whose value is fed back to the computer. The software specified in this document will implement this control system and provide the operator with easy means to intervene in the fill operations.

In addition to filling the detectors at predefined times the autofill must detect malfunctions like overfilling (e.g a valve is stuck open), underfilling (the detector is not full when fill operation should have finished), give an audible alarm when any of these faults occur and also be able to detect if the power to a detector is turned off due to overheating (bias shutdown). The normal filling operations can be interrupted from the computers console terminal, or the computer can be overridden to allow manual filling via switches.

2 HARDWARE

The computer-crate is to be placed in a 39u high rack valve and sensor interface electronics and fans. A console terminal will be situated in the experiment area, and will provide user-access to the autofill system. The system may also have three other terminals connected, but interventions in the fill cycle will only be allowed from the terminal situated in the experiment area. The autofill system will be connected to ethernet and reports to the central Eurogam ’exception handler’ when faults occur.

The computer crate will contain the following items:

• MVME147 processor card with ethernet, four RS232 ports, SCSI and memory.

• MVME712M interface card for RS232, ethernet, SCSI and printer.

• XVME560 ADC cards (32 differential channels)

• XVME260 Relay output cards (32 channels)

• XVME240 TTL input/output cards (64 channels IO + 8 Interrupt + 8 output)

• Power supplies.

The sensor and solenoid valve interfaces are being designed and built at York University. They will contain all the necessary electronics to condition the signals from the sensors before they go to the ADC-cards and will also contain conditioning circuitry to convert the relay signals from the XVM260-cards so that they can drive the solenoid valves. The TTL-IO cards (XVME240) will sense the bias shutdown signal from the High Voltage Control, and thus avoid filling detectors which have had the power switched off. One XVME240 card will also read the state of the manual fill switches on the front of each manifold control unit. It will also have one channel reserved for a signal to the dewar inhibiting a fill of this when detector fills are going on.

These cards must be present for the system to work.

The interface units will contain LEDs to indicate ”vital signs”, and switches to allow manual override. They will be connected to the VMEcards with short lengths of ribbon cable. Connections to the solenoid valves require cables carrying mains voltages to be routed from the rear. These will be available in groups of eight, and six will be for normal connection to the primary valve of 6 detectors. One is intended to give control over a manifold shutoff valve to provide a fail-safe in case of other valves freezing. The 8th will be used to control the manifold output valve (which also is fitted with a sensor).

Additional requirement: The XVME cards should be configured to respond to addresses in the VMEbus Short IO address space and should respond both in user and supervisor modes. This will give 64 card addresses where each card occupies 1Kbyte of memory: 0000,0400,0800,......,F800,FC00.

NB! Care must be taken when using address 0x0000 as this may conflict with the VMEchip Global Control and Status Register (GCSR). If this address is used the GCSR must be programmed to no response.

3 SOFTWARE

The autofill control program should provide a user interface that makes it easy to get an overview of the state of the detectors, and also facilitate quick and safe intervention if the system needs attention from authorised personnel in the experimental area.

The development of the software will be done using VxWorks and SUN workstations and the design should take advantage of the operating systems multi-tasking facilities in order to obtain a modular and easily maintainable design. The code will be implemented using the C-language.

A common block of data (hereafter referred to as the database) should serve as communication between the tasks. The database will contain state variables and configuration data for all the detectors in the array. One proposes separate tasks to drive each XVME-card, and tasks for fill control, user interface and logging/supervision. High-level communication facilities like pipes would be rather inefficient in this application as large amounts of data are accessed by more than two tasks.

3.1 Access rights

Intervention in the filling cycle should only take place from a console terminal placed in the experiment area. Users logged on from the other three terminal ports can only view the state of the system. The same will apply to non-privileged users who log on at the console terminal. Access to the VxWorks shell must be restricted.

It is required that a login task is written to control access to both the shell and the autofill user program. The login must be password protected. The login task should be fully reentrant so that one can have copies of it serving the three other serial ports.

I propose grouping of user privileges as follows:

• limited access to autofill user program

• full access to autofill user program

• access to remote server(s)

• shell access

3.2 Recovery

The system must keep a backup of the detector states on disk. This could be on a local disk or on a remote file server (accessed via ethernet). Two backups will be kept, one updated after every change in the detector states, the other done at fixed intervals. The primary backup will only be done for the affected detector(s), whilst in the second case the complete database will be written onto the disk. On startup the system should first try to read configuration/state from the primary backup, if that fails try the secondary backup and as a last resort read the original data file as it was generated.

3.3 Configuration

The configuration of the detectors should be kept in a file for maximum flexibility.

Configuration parameters:

• physical position in array (1-72)

• exhaust sensor XVME560-card address and channel number

• detector valve XVME260-card address and channel number

• bias shutdown XVME240-card address and channel number

• manifold connection number

• sensor reference voltage (volts)

• maximum fill time (seconds)

• valve closure delay time (seconds)

• preset fill times

There will be a similar set of configuration parameters for each manifold. These will include:

• exhaust sensor XVME560-card address and channel number

• input valve XVME260-card address and channel number

• output valve XVME260-card address and channel number

• sensor reference voltage (volts)

• maximum fill time (seconds)

• valve closure delay time (seconds)

These values will be edited into a file initially and only the preset fill times can be changed without editing the file again.

It is suggested that the configuration is edited into a file as text using any available editor and that one allows pre-processing to facilitate use of symbols and comments. This text file will then be preprocessed, error checked and finally used as input for a program that generates a binary configuration/data-base file. With this approach one allows a free format for the source configuration combined with the advantage of knowing the file position for each detectors data. The latter will be used to limit backup to only the detectors which have actually had their states changed. This will save having to write back the whole database.

3.4 System timing

A means must be provided to measure the time in absolute terms using a predefined reference. All timing operations will then use absolute seconds from this reference (e.g when to start filling). This is analogous to the Julian time found in OS/9.

3.5 State variables detector

The system must maintain the following information for each detector • most recent ADC value (volts)

• time of last valve opening/closure

• time of next fill

• fill delay

• open/closed status for valve

• status: overfull, underfull, manifold error, filling, delayed, purging, avoided, bias shutdown.

3.6 State variables manifold

The system must maintain these variables for each manifold.

• most recent ADC value (volts)

• time of last valve opening/closure (seconds since reference)

• number of open detector valves

• number of overfull detectors (used for fill inhibit)

• open/closed status for both input and output(purge) valve

• status: overfull, underfull, purging.

3.7 Global state variables

• number of filling detectors

• overall system state (OK, CRASHED, FILL ERROR, FILLING, MANUAL FILL)

3.8 Fill cycle

The next fill time will for each detector be continuously compared to current time, and the valve opened if due. The valve is kept open until exhaust nitrogen is detected or the maximum fill time exceeded. The last condition is erroneous and the underfull error should be indicated. If nitrogen is detected after a fill is completed (allowing for closure delay) the overfull error must be indicated. If a detector is overfull the manifold input valve must be closed and further filling from that manifold will be aborted. The underfull and overfull errors are also possible when purging the manifold. The overfull error should inhibit further use of the manifold while it persistes.

Criteria for opening and closure of valves: Make use of manifold output valve sensor to ensure that the manifold is purged of air when detector filling starts. Thus a fill will include the following steps:

1. Open manifold input AND output valves

2. Wait until manifold full - or timeout

3. Open detector valve(s)

4. Close manifold output valve

5. Wait until detector(s) full - or timeout

6. Close detector valve(s)

7. Close manifold input valve (if not used by other detectors)

3.9 Error handling

Errors and time of errors must be recorded. Faults that need recording are e.g. overfull and underfull. During the development phase this facility will be used to record other types of errors for debugging purposes. All errors, including system errors (e.g. AD conversion timeout) must be reported to the Eurogam ’exception handler’. If the system crashes it must be ensured that all valves are closed (that is if the system error is of a nature that permits a controlled shutdown).

3.10 Manual fill

The system must detect the switching of the maunal fill key, and inhibit all fill actions when in manual mode. Ongoing fills will be aborted and any intervention from the console terminal must be made impossible.

4 USER REQUIREMENTS

The user must give his identifier and password correctly before he is admitted. The non-privileged user (hereunder also privileged users operating from non console terminals) can only view the system. The user interface must provide reliable means for operator intervention in the filling cycle. It must be ensured that no valves are operated on by mistake. During manual fill no privileged access will be permitted.

The autofill will be operated through a series of short commands which are prompted from a simple command interpreter. The keyboard input software must provide a timeout so that the system does not hang on, say, a half completed input line. Some commands (see below) should be able to execute on multiple detectors, in this case a further confirmation (YES/NO) is required. Typing errors should provoke meaningful error messages, not an error number. Abbreviated command names are allowed. A HELP command should be implemented to assist the operator.

There will be a set of auxiliary commands that can be executed in a so called ADMIN mode ( these will not be specified in detail here).

A command line will have the following syntax:

<COMMAND> {arg1} {arg2} {arg3} {arg4} {arg5}

Arguments may be omitted depending on the actual command.

An argument can have one of the following formats:

• A number in the range 1-72, a position identifier. This means that the action prescribed by the command will be executed on the given position only.

• A letter A-L, a manifold identifier. The command will be executed for all the positions on the specified manifold. Confirmation will be asked for.

• Range of detector numbers

• Sets of detectors (can be combined freely with ranges)

• The text string ”all”. The command will be executed for all detectors in the array. Confirmation will be asked for.

• A string on the format ”HH:MM”, where HH represents hours and MM minutes.

• A two character number giving seconds, minutes or hours.

• Short special purpose arguments.

The display should at any time indicate errors in the autofill, and by issuing a command the operator will be able to localise and investigate the error. A time of day clock should be always be displayed. The terminals graphical capacity should be exploited to ensure a clear and user-friendly screen layout.

Here is a detailed specification for the user commands:

4.1 SHOW

This command will be used to investigate the state of the autofill. It can be issued on three different levels.

• Top level. Overview of most important parameters for all detectors

• Manifold level. More detailed view of manifold and its detectors

• Position level. Detailed view of the given position

The operator can force displaying of another manifold or detector without explicitly issuing a new show command. This will be achieved by the use of single keystrokes (e.g arrows). Four different keys will be used to increment/ decrement detector/manifold ids respectively. A key-stroke to toggle between showing manifold and detector will also be provided.

Other non-privileged commands:

4.2 HELP

Online help, either general or specific for each command. General topics includes input line syntax, command execution modes and a guide to the display.

4.3 ADMIN

Switch to ADMIN mode.

4.4 NORMAL

ADMIN mode commmand to switch back to normal autofill mode

4.5 QUIT

Logout from the autofill system

PRIVILEGED COMMANDS can only be issued from the console by privileged users. These commands can be executed for multiple channels but confirmation is needed before the requested action takes place.

4.6 PRESET

Set the prescheduled fill times. This command sets upto 4 fill times. Options for adding to present fill times and clearing old ones are needed. By default the old fill times are written over.

The command will modify the next fill time parameter and set it to the first time encountered in the new list of fill times. Any time set by a previously issued FILL command will be lost.

4.7 FILL

Set an unscheduled fill. This command sets the next fill to the specified time; omitting the time argument will cause a fill to start immediately. If the specified fill time is beyond any of the prescheduled times, those will be ignored.

4.8 DELAY

Delay the next fill. This delays the next fill with the specified number of minutes. The display should indicate when a detector is delayed.

4.9 AVOID

Avoid specified detector(s). This command will stop all filling operations on the specified detectors(s) and also prevent alarms from being raised.

4.10 UNAVOID

Enable all fill operations (will also re-enable a detector which has been taken out through bias-shutdown). Essentially the opposite of AVOID.

4.11 CANCEL

This command will stop an ongoing fill or cancel a delayed fill. Error conditions are unaffected.

4.12 CLEAR

Clears errors and warnings.

5 TEST REQUIREMENTS

The tests will be carried out at two levels, power on test and monitoring test are integrated into the autofill control system, their purpose is to detect the faults. The auxiliary tests are meant to be used thereafter for further diagnosis and fault localisation. The purpose of the test is to locate the error to a particular card so this can be replaced, and possibly also decide which channels on the card are faulty.

5.1 Power on tests

• XVME560

– read identification pattern

– verify correct contents in status register after reset

– do a test conversion

• XVME260

– read identification pattern

– verify that output registers are cleared after reset

• XVME240

– read identification pattern

All power on test results will be logged, and the status of the test will be indicated by green (passed) or red LED (failed) on the front of the card. (not applicable for XVME260).

5.2 Monitoring test

This will consist in reporting errors from simple tests that will run as part of the task loops for the three XVME-tasks:

• XVME560 ADC conversion status

The RED LED will be lit to indicate the faulty card.

5.3 Auxiliary test

This paragraph contains some suggestions on test/faultfinding. It will NOT get a high priority , but if time permits at least some of the ideas will be implemented.

• One should immediately be able to test the crate for response from the cards. This will be a simple bus-error test that scans all the 64 1K address boundaries in the VME Short IO-space to check for presence of cards. This will detect any bus-errors by reporting the faulty card not present. A list of present cards and their type will be displayed.

• Show how a specified detector is connected to each XVME-card.

• Show the connections to a specified card.

• Show spare channels in the system.

• Explicit manipulation and viewing of IO-registers (this has low priority as it can be done using VxWorks shell).

• Explicit call of power on tests.

• Explicit AD conversion on a specified channel.

Those tools cold either be called from the VxWorks shell or as commands in the ADMIN mode. There will have to be a program to create the initial database from a text file.

Jomar Hønsi Liverpool, 10th of January 1995

E-mail: jh@ns.ph.liv.ac.uk Telephone: int + 51 794 3398