Exogam Autofill Control Specification (EDOC401)
John Cresswell
Nuclear Structure Software Systems Group, Department of Physics,
University of Liverpool, UK
Version 1.1, March 1998
Version 1.1 introduces an extra capability over the previous Eurogam
version. The Exogam Ge detectors will have a Pt100 temperature sensor
onboard. To make use of this an extra interface board will be added
and new software will need developing.
There will also be the capability to disable data acquisition during
fills if necessary.
This document is an updated version of the Eurogam Autofill
Specification document. There are several changes which relate to the
differences between Exogam and Eurogam, but also a re-working of some
aspects of the design.
However, there are no changes to the fundamental functionality of
the system. It is intended to keep intact all the basic design
characteristics that were agreed upon previously.
Any hardware and software changes are designed to provide improvements
in system performance, substantial reduction of cost together with minimising
the development effort.
The philosophy of the computer controlled autofill system is one which is
intended to make it possible to ensure that the cryostats of a large variable
number of germanium detectors on the Exogam array are filled at
defined times and that the correct human intervention happens when necessary.
The Autofill is a simple feedback control system,the purpose of which
is to ensure that the detectors in the Exogam array are cooled with liquid
nitrogen. Filling of LN2 takes place at fixed programmable intervals and is
stopped when exhaust nitrogen is detected. The filling operation is performed by
activating solenoid valves through which the LN2 is allowed into the detector.
Exhaust nitrogen is registered by a temperature sensor whose value is fed back
to the computer.
The software specified in this document will implement this control system and
provide the operator with easy means to intervene in the fill operations.
In addition to filling the detectors at predefined times the autofill must
detect malfunctions like overfilling (e.g a valve is stuck open), underfilling
(the detector is not full when fill operation should have finished), give an
audible alarm when any of these faults occur and also be able to detect if
the power to a detector is turned off due to overheating (bias
shutdown).
The detector temperature will also be used to detect detectors warming
up, and hence try a fill. If the fill does not lower the temperature,
then more drastic action is necessary.
The normal filling operations can be interrupted from the computers console
terminal, or the computer can be overridden to allow manual filling via switches.
The computer system will be PC-based and may be either a standard base
unit
or (perhaps better for the array racking) an industrial rack-mounted chassis.
The computer interface consists of sets of 3 ISA I/O cards,
interfacing to the valve and sensor interface electronics.
Each set of cards is capable of supporting 24 detector fill channels.
A PC monitor may be situated in the experiment area for local access,
and the system will be remotely accessible via the LAN.
The autofill system will report to the main data acquisition
exception handler when faults occur.
A fourth interface card will be used to interface to the detector temperature
sensor. This will take the form
of a second ADC card to interface to 16 channel RTD interface boards.
The computer base unit will contain the following set of cards:
- 32 differential channel ADC card (Computerboards CIO DAS 6402/12)
- 32 channel SPST electromechanical relay output card (Keithley PIO-32OUT)
- 32 channel TTL digital I/O card (Computerboards CIO DIO48)
- 8 channel ADC card (Computerboards CIO DAS 1602/12) + 2 CIO-EXP-RTD cards
The sensor and solenoid valve interfaces are designed and built at York
University.
This part of the system is described in EDOC408.
They will contain all the necessary electronics to condition the
signals from the sensors before they go to the ADC card(s) and will also contain
conditioning circuitry to convert the output relay signals so that they can drive the solenoid valves.
The TTL IO card(s) will sense the bias shutdown signal from the
High Voltage Control, and thus avoid filling detectors which have had the
power switched off. One channel of the TTL digital input card will
also read the state of the
manual fill switches on the front of the manifold control unit.
It will also have one or more channels reserved for signals to be used
for inhibiting (whilst detectors are filling) data acquisition or
a dewar fill.
These cards must be present for the system to work.
The interface unit contains LEDs to indicate "vital signs", and switches
to allow manual override. It connects to the IO cards with short
lengths of ribbon cable. Connections to the solenoid valves require cables
carrying mains voltages to be routed from the rear. These will be available
in groups of eight, and six will be for normal connection to the primary
valve of 6 detectors. One is intended to give control over a manifold shutoff
valve to provide a fail-safe in case of other valves freezing. The 8th will be
used to control the manifold output valve (which also is fitted with a sensor).
Additional requirement: The IO cards will be configured to
respond to specific available addresses in the PC IO address space.
Allocation for more than one set of cards will be defined
in case the need arises.
It is intended to use the Linux operating system which provides
a familiar Unix-style environment. It has the capability of easy
configuration of access security, reliability and technical features
that make IO ports easily accessible for our code.
The autofill control program provides a user interface that makes it
easy to get an overview of the state of the detectors, and also facilitates
quick and safe intervention if the system needs attention from authorised
personnel in the experimental area.
A common block of data (hereafter referred to as the database)
serves as communication between the tasks. The database contains state
variables and configuration data for all the detectors in the array.
There are separate tasks to drive each IO card, and tasks for fill
control, user interface and logging/supervision.
For Eurogam,
intervention in the filling cycle was defined to only take place
from a console terminal placed in the experiment area near the array.
Users logged on from elsewhere could only view the
state of the system. The same applied to non-privileged users who logged on
at the console terminal.
For Exogam, access to the array will be limited, and a revised login
capability definition may be required.
The system access constraints will be configurable at the network
access and userid level as deemed necessary at the time.
The system must keep a
backup of the detector states on disk. This will be on a local disk and/or on
a remote file server (accessed via ethernet).
Two backups will be kept,
one updated after every change in the detector states, the other done
at fixed intervals. The primary backup will only be done for the
affected detector(s), whilst in the second case the complete database
will be written onto the disk. On startup the system should first try
to read configuration/state from the primary backup, if that fails try
the secondary backup and as a last resort read the original data file as
it was generated.
The configuration of the detectors is kept in a file
for maximum flexibility.
Configuration parameters:
- physical position in array (1 -"N")
- exhaust sensor ADC card address and channel number
- detector valve Relay output card address and channel number
- bias shutdown digital input card address and channel number
- detector temperature sensor card address and channel number
- manifold connection number
- sensor reference voltage (volts)
- maximum fill time (seconds)
- valve closure delay time (seconds)
- preset fill times
There will be a similar set of configuration parameters for each
manifold. These will include:
- exhaust sensor ADC card address and channel number
- input valve digital input card address and channel number
- output valve Relay output card address and channel number
- sensor reference voltage (volts)
- maximum fill time (seconds)
- valve closure delay time (seconds)
These values will be edited into a file initially and only the preset fill times
can be changed without editing the file again.
The configuration is edited into a file as text using
any available editor. This text file is then preprocessed, error
checked and finally used as input for a program that generates a binary
configuration/data-base file. With this approach one allows a free format for
the source configuration combined with the advantage of knowing the file
position for each detectors data. The latter will be used to limit backup
to only the detectors which have actually had their states changed. This will
save having to write back the whole database.
The system maintains the following information for each detector
- most recent ADC value (volts)
- most recent temerature value (degK)
- time of last valve opening/closure
- time of next fill
- fill delay
- open/closed status for valve
- status: overfull, underfull, manifold error, filling, delayed, purging,
avoided, bias shutdown.
The system maintains the following information for each manifold
- most recent ADC value (volts)
- time of last valve opening/closure (seconds since reference)
- number of open detector valves
- number of overfull detectors (used for fill inhibit)
- open/closed status for both input and output(purge) valve
- status: overfull, underfull, purging.
- number of filling detectors
- overall system state (OK, CRASHED, FILL ERROR, FILLING, MANUAL FILL)
The next fill time for each detector is continuously
compared to current time, and the valve opened if due. The valve is kept
open until exhaust nitrogen is detected or the maximum fill time exceeded.
If the last condition is erroneous the underfull error is
indicated. If nitrogen is detected after a fill is completed (allowing for
closure delay) the overfull error is indicated. If a detector is
overfull the manifold input valve is closed and further filling from that
manifold is aborted.
The underfull and overfull errors are also possible when purging the manifold.
The overfull error inhibits further use of the manifold while it persists.
Criteria for opening and closure of valves:
The manifold output valve sensor is monitored to ensure that the
manifold is purged
of air when detector filling starts. Thus a fill will include the
following steps:
- Open manifold input AND output valves
- Wait until manifold full - or timeout
- Open detector valve(s)
- Close manifold output valve
- Wait until detector(s) full - or timeout
- Close detector valve(s)
- Close manifold input valve (if not used by other detectors)
Errors and time of errors is recorded. Faults that
need recording are e.g. overfull and underfull.
All errors, including system errors (e.g. AD conversion timeout)
are reported to the data acquisition system exception handler.
The system is designed such that in the event of software failure,
valves are closed (that is if the system error is
of a nature that permits a controlled shutdown).
The system detects the switching of the maunal fill key, and inhibits all
fill actions when in manual mode. Ongoing fills will be aborted and any
software controlled intervention is inhibited.
The user must give his identifier and password correctly
before he is admitted. The non-privileged user (also privileged
users operating from non priviledged access points) can only view the system.
The user interface provides reliable means for operator intervention in
the filling cycle. It is ensured that no valves are operated on by mistake.
During manual fill no privileged access will be permitted.
The original Eurogam autofill system had a VT100 style interface.
The Exogam version will have a graphical interface in the style of the
MIDAS Tcl/Tk acquisition system.
The following refers to the VT100 style interface.
Equivalents will be provided graphically.
The autofill is operated through a series of short commands which are
prompted from a simple command interpreter. The keyboard input software
provides a timeout so that the system does not hang on, say, a half completed
input line.
Some commands (see below) can execute on multiple detectors,
in this case a further confirmation (YES/NO) is required. Typing errors
provoke meaningful error messages, not an error number.
Abbreviated command names are allowed.
There will be a set of auxiliary commands that can be executed in a
so called ADMIN mode ( these will not be specified in detail here).
A command line will have the following syntax:
<COMMAND> {arg1} {arg2} {arg3} {arg4} {arg5}
Arguments may be omitted depending on the actual command.
An argument can have one of the following formats:
- A number in the range 1-N, a position identifier. This means that the
action prescribed by the command will be executed on the given position
only.
- A letter A-D, a manifold identifier. The command will be executed for all
the positions on the specified manifold. Confirmation will be asked for.
- Range of detector numbers
- Sets of detectors (can be combined freely with ranges)
- The text string "all". The command will be executed for all detectors
in the array. Confirmation will be asked for.
- A string on the format "HH:MM", where HH represents hours and MM
minutes.
- A two character number giving seconds, minutes or hours.
- Short special purpose arguments.
The display will indicate errors in the autofill, and by issuing
a command the operator will be able to localise and investigate the error.
A time of day clock will always be displayed.
Here is a detailed specification for the user commands:
This command will be used to investigate the state of the autofill. It can
be issued on three different levels.
- Top level. Overview of most important parameters for all detectors
- Manifold level. More detailed view of manifold and its detectors
- Position level. Detailed view of the given position
The operator can force displaying of another manifold or detector without
explicitly issuing a new show command. This will be achieved by the use of
single keystrokes (e.g arrows). Four different keys will be used to increment/
decrement detector/manifold ids respectively. A key-stroke to toggle between
showing manifold and detector will also be provided.
Other non-privileged commands:
Online help, either general or specific for each command.
General topics includes input line syntax, command execution modes and a
guide to the display.
Switch to ADMIN mode.
ADMIN mode commmand to switch back to normal autofill mode
Logout from the autofill system
PRIVILEGED COMMANDS can only be issued from the console by privileged
users. These commands can be executed for multiple channels but confirmation
is needed before the requested action takes place.
Set the prescheduled fill times. This command sets
upto 4 fill times. Options for adding to present fill times and
clearing old ones are needed. By default the old fill times are written over.
The command will
modify the next fill time parameter and set it to the first time
encountered in the new list of fill times. Any time set by a previously issued
FILL command will be lost.
Set an unscheduled fill. This command sets the next fill
to the specified time; omitting the time argument will cause a fill to start
immediately. If the specified fill time is beyond any of the prescheduled
times, those will be ignored.
Delay the next fill. This delays the next fill with the
specified number of minutes. The display should indicate when a detector is
delayed.
Avoid specified detector(s).
This command will stop all filling operations on the specified detectors(s)
and also prevent alarms from being raised.
Enable all fill operations (will also re-enable a detector
which has been taken out through bias-shutdown). Essentially the opposite of
AVOID.
This command will stop an ongoing fill or cancel a
delayed fill. Error conditions are unaffected.
Clears errors and warnings.
The tests will be carried out at two levels, power on test and
monitoring test are integrated into the autofill control system,
their purpose is to detect the faults. The auxiliary tests are meant
to be used thereafter for further diagnosis and fault localisation. The purpose
of the test is to locate the error to a particular card so this can be replaced,
and possibly also decide which channels on the card are faulty.
- ADC card
- read identification pattern
- verify correct contents in status register after reset
- do a test conversion
- Relay output card
- read identification pattern
- verify that output registers are cleared after reset
- Digital input card
- read identification pattern
All power on test results will be logged.
This consists of reporting errors from simple tests that run as part
of the task loops for the four IO card tasks,
e.g. the ADC conversion status.
- One should immediately be able to test the crate for response from the cards.
This will be a simple bus-error test that scans for presence of
cards. This will
detect any bus-errors by reporting the faulty card not present. A list
of present cards and their type will be displayed.
- Show how a specified detector is connected to each IO card.
- Show the connections to a specified card.
- Show spare channels in the system.
John Cresswell, March 1998