The following instructions assume that you are familiar
with Linux and are just an indication of the tasks required for the MIDAS
installation. It is NOT expected that someone who has never used Linux before
would be able to perform the software installation without additional assistance.
In particular you WILL require superuser (root) access to your Linux server in order to perform a complete initial
installation of the software.
These instructions have been tested using Scientific Linux (RedHat Enterprise from CERN)
with the 2.6 Linux kernel.
The embedded Linux system is loaded into the Xilinx Virtex PPC processor core by one of a number of hardware specific methods. The Linux kernel then takes over and NFS mounts the Linux root filesystem from server and mounts any other
file systems required and runs a sequence of scripts which load the MIDAS Data Acquisition components such as the
MIDAS servers and the Data Acquisition program.
The version of Linux used by the Virtex processor is a part of the standard Linux core. All code used is covered by the GPL and hence is available free of any license charge.
For information on how the Linux system is built see Linux for the Xilinx Virtex4/5 FPGAs.
The current release of Linux for the Xilinx Virtex PPC core is 2.6.31.
The instructions and examples which follow use the file pathnames which are used at Daresbury. While it is possible to install the software at other locations (no code has inbuilt absolute pathnames which cannot be overridden) you will have to understand the installation in greater detail so that your installation is consistent.
The hardware configuration assumed is a Linux workstation having 2 ethernet interfaces. The first ethernet interface is used for communication with the external network (this is in principle optional). The second ethernet interface (IP address 192.168.0.1) is used for communication with the FEE Virtex Linux system (192.168.0.12).
Throughout these instructions the host name of the Virtex Linux system is nnvtx4b. You can change this to whatever you wish but you MUST be consistent and replace nnvtx4b by your name whereever you see it in the instructions and examples.
Download and install the root file system for the Virtex Linux in the directory /MIDAS/XilinxLinux/ppc_4xx/rfs.
You must install the tar file as user=root in order that the correct file ownership and permissions for the Linux file system are preserved.
The file system will be the directory structure /MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b.
Install the MIDAS distribution for Virtex Linux in the directory /MIDAS/linux-ppc_4xx
This contains MIDAS device drivers for Linux, MIDAS servers and common programs.
Install any application packges required
The Linux/Unix server needs to know the name and IP address of the Virtex Linux system.
If the Virtex Linux system is being connected to a "private" network then
the Virtex Linux system can have an IP address like 192.168.0.12.
Edit the file /etc/hosts and add (for each Virtex processor) a line like
192.168.0.12 nnvtx4b
All boot parameters and initial configuration options are supplied by a DHCP server.
Install the DHCP service
Linux installations may have a DHCP server included although it may not be enabled and configured by default.
If not it can easily be installed using the "Add\Remove Software" tool.
sample configuration file for DHCP: /etc/dhcpd.conf
default-lease-time 3600;
ddns-update-style ad-hoc;
max-lease-time 14400;
subnet 192.168.0.0 netmask 255.255.255.0 {
host nnvtx4b {
hardware ethernet 00:04:a3:25:a5:ca;
fixed-address 192.168.0.12;
server-name "192.168.0.1";
next-server 192.168.0.1;
option root-path "/MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b,tcp";
option host-name "nnvtx4b";
filename "xxxx.bin";
}
}
Start the dhcpd server. Note that this will have to be restarted manually in order to activate any changes to the dhcpd.conf file.
Enable the NFS service
If NFS had not previously been running in the Linux server it will be necessary to start it.
This normally happens automatically when the file /etc/exports exists
The Linux/Unix server must also SHARE the file systems accessed via NFS with Linux.
edit the file /etc/exports
for each Virtex Linux processor add
/MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b nnvtx4b(rw,sync,nohide,no_root_squash)
/MIDAS nnvtx4b(rw,sync,nohide)
restart the NFS service to activate the change and/or start the NFS service
The Virtex Linux system needs to know the IP address of your server and how to mount the MIDAS application (as distinct from its root file system).
You will need to review the files /etc/hosts and /etc/fstab in the Virtex Linux file system. These are the files
/MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b/etc/hosts and /MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b/etc/fstab as seen by your server. If you use the default installation configuration these should already be correct. These path names may of course need to be modified to be compatible with your installation as determined by the DHCP parameters.
hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.0.1 myserver loghost egmsg
You may have to modify the supplied file to use the IP address of your server.
fstab
none /proc proc defaults 0 0
none /dev/shm tmpfs size=64M 0 0
none /dev/pts devpts gid=5,mode=620 0 0
myserver:/MIDAS /MIDAS nfs soft,intr,timeo=20
The supplied file will normally not require to be changed.
Time of day and date for the Virtex Linux system.
It is probable that the Virtex system does not have hardware to preserve time-of-day and date. If required these can be set on boot either from a network time standard or from the local Linux server.
Review the file /etc/sysconfig/setclock in the Virtex Linux file system. To use the local Linux server this should contain
/usr/bin/rdate -s myserver
You will then need the time-stream service in the server.
If needed install xinetd. Then in the file /etc/xinetd.d/time-stream ensure disable=no.
Now you will need to connect a simple line-mode terminal to the terminal port of the Virtex system or (better) use a terminal
emulator program running in a workstation or PC.
A VERY simple program is provided by the MIDAS session. (See later).
You will be pleased to hear that Virtex Linux should now boot!!.
You will expect to see output like this following. NOTE that initial messages (loading the Linux kernel into RAM) are hardware dependent and that messages after the line Starting MIDAS Data Acquisition for nnvtx4b
will be dependent on the particular application being loaded.
Obtained EEPROM configuration data in get_mac_addr
Obtained good MAC address from EEPROM data
loaded at: 00400000 0051B59C
board data at: 00519520 0051959C
relocated to: 0040505C 004050D8
zimage at: 00405FBD 005180B4
avail ram: 0051C000 04000000
Linux/PPC load: console=ttyUL0 ip=dhcp root=/dev/nfs rw
Uncompressing Linux...done.
Finished loading now booting the kernel
[ 0.000000] Linux version 2.6.25-xlnx (nf@nnlxb.dl.ac.uk) (gcc version 4.0.0 (DENX ELDK 4.1 4.0.0)) #55 Wed Sep 10 09:08:48 BST 208
[ 0.000000] Xilinx Generic PowerPC board support package (Xilinx ML403) (Virtex-4 FX)
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0 -> 16384
[ 0.000000] Normal 16384 -> 16384
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[1] active PFN ranges
[ 0.000000] 0: 0 -> 16384
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16256
[ 0.000000] Kernel command line: console=ttyUL0 ip=dhcp root=/dev/nfs rw
[ 0.000000] Xilinx INTC #0 at 0x81800000 mapped to 0xFDFFF000
[ 0.000000] PID hash table entries: 256 (order: 8, 1024 bytes)
[ 0.000224] Console: colour dummy device 80x25
[ 0.000862] Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.001788] Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.019611] Memory: 62464k available (1728k kernel code, 596k data, 104k init, 0k highmem)
[ 0.112565] Mount-cache hash table entries: 512
[ 0.119058] net_namespace: 152 bytes
[ 0.121826] NET: Registered protocol family 16
[ 0.125830] Registering device uartlite:0
[ 0.126891] Registering device xsysace:0
[ 0.127953] Registering device xpraysnucam:0
[ 0.129199] Fixup MAC address for xilinx_lltemac:0
[ 0.129247] Registering device xilinx_lltemac:0
[ 0.130309] Registering device xilinx_iic:0
[ 0.170841] NET: Registered protocol family 2
[ 0.204707] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.207060] TCP established hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.207430] TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.207634] TCP: Hash tables configured (established 2048 bind 2048)
[ 0.207672] TCP reno registered
[ 0.226051] io scheduler noop registered
[ 0.226114] io scheduler anticipatory registered (default)
[ 0.226147] io scheduler deadline registered
[ 0.226374] io scheduler cfq registered
[ 0.299698] uartlite.0: ttyUL0 at MMIO 0x84000003 (irq = 6) is a uartlite
[ 0.299776] console [ttyUL0] enabled
[ 2.662862] brd: module loaded
[ 2.680202] tun: Universal TUN/TAP device driver, 1.6
[ 2.744030] tun: (C) 1999-2004 Max Krasnyansky
[ 2.817335] xilinx_lltemac xilinx_lltemac.0: MAC address is now 0: a:35: 1:8b:ff
[ 2.904044] xilinx_lltemac xilinx_lltemac.0: XLlTemac: using DMA mode.
[ 2.984152] XLlTemac: Dma base address: phy: 0x84600100, virt: 0xc5008100
[ 3.064058] XLlTemac: buffer descriptor size: 32768 (0x8000)
[ 3.132026] XLlTemac: Allocating DMA descriptors with kmalloc<6>XLlTemac: (buffer_descriptor_init) phy: 0x3c68000, virt: 0xc3c68000, size: 0x8000
[ 3.297495] XTemac: No PHY detected. Assuming a PHY at address 0
[ 3.364133] eth0: Dropping NETIF_F_SG since no checksum feature.
[ 3.438774] xilinx_lltemac xilinx_lltemac.0: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xC5004000, irq=3
[ 3.552625] mice: PS/2 mouse device common for all mice
[ 3.612037] i2c /dev entries driver
[ 3.658928] xilinx_iic.0 #0 at 0x81600000 mapped to 0xC5020000, irq=5
[ 3.737263] TCP cubic registered
[ 3.768261] NET: Registered protocol family 1
[ 3.820242] NET: Registered protocol family 17
[ 3.876716] RPC: Registered udp transport module.
[ 3.928077] RPC: Registered tcp transport module.
[ 4.493084] eth0: XLlTemac: Options: 0x3fa
[ 4.524092] eth0: XLlTemac: allocating interrupt 1 for dma mode tx.
[ 4.600067] eth0: XLlTemac: allocating interrupt 2 for dma mode rx.
[ 6.683294] eth0: XLlTemac: speed set to 1000Mb/s
[ 6.724043] eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
[ 6.800032] eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
[ 7.888086] Sending DHCP requests ., OK
[ 7.932081] IP-Config: Got DHCP answer from 192.168.0.1, my address is 192.168.0.12
[ 8.012763] IP-Config: Complete:
[ 8.048029] device=eth0, addr=192.168.0.12, mask=255.255.255.0, gw=192.168.0.1,
[ 8.144025] host=nnvtx4b, domain=dl.ac.uk, nis-domain=nuclear.physics,
[ 8.228033] bootserver=192.168.0.1, rootserver=192.168.0.1, rootpath=/MyMIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b
[ 8.362573] Looking up port of RPC 100003/2 on 192.168.0.1
[ 8.445584] Looking up port of RPC 100005/1 on 192.168.0.1
[ 8.535248] VFS: Mounted root (nfs filesystem).
[ 8.572419] Freeing unused kernel memory: 104k[ 8.884341] eth0: XLlTemac: PHY Link carrier lost.
[ 10.934120] eth0: XLlTemac: speed set to 1000Mb/s
[ 10.972014] eth0: XLlTemac: PHY Link carrier restored.
INIT: version 2.86 booting
Starting sysinit...
Welcome to DENX and STFC Daresbury Embedded Linux Environment
Press 'I' to enter interactive startup.
Setting clock (utc): Wed Nov 3 14:25:54 GMT 2010 [ OK ]
Building the cache [ OK ]
Setting hostname nnvtx4b: [ OK ]
Mounting local filesystems: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Finishing sysinit...
INIT: Entering runlevel: 3
Entering non-interactive startup
FATAL: Module ipv6 not found.
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: [ OK ]
FATAL: Module ipv6 not found.
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Starting rpcbind: [ OK ]
Mounting NFS filesystems: [ OK ]
Mounting other filesystems: [ OK ]
Starting xinetd: [ OK ]
Starting midas: Starting MIDAS Data Acquisition for nnvtx4b
[ 44.123301] Trying to free nonexistent resource <0000000081840000-00000000818fffff>
[ 44.196149] XPRAYSNUCAM: xpraysnucam: mem region start 0x81840000 for 0xc0000 mapped at 0xc5080000
[ 44.304116] XPRAYSNUCAM: xpraysnucam: driver assigned major number 254
System identified is CPU ppc; Platform is unix; OS is Linux and Version is 2.6.25-xlnx
Environment selected is CPU ppc; Platform unix; OS Linux and Operating System linux-ppc_4xx
MIDASBASE = /MIDAS and MIDAS_LIBRARY = /MIDAS/TclHttpd/linux-ppc_4xx
package limit is not available: can't find package limit
Running with default file descriptor limit
Message logger not contacted.
Spectrum Access Server installed on port 10230
Executive Program Release 1
Executive task now ready for work
Executive task started - PID= 504
NUCAM2 Data Acquisition Program Release 2.030309 starting
package setuid is not available: can't find package setuid
Running as user 0 group 0
Creating NetVars
Output buffer length = 65504; format option = 3; transfer option = 3
NetVars created and initialised
Data Acquisition task has PID 506
Statistics thread starting
Statistics thread created
Heartbeat thread [ 71.604340] XPRAYSNUCAM: xpraysnucam_open:
starting
Heartbeat thread created
Installing signal handlers
Done
PORGAMRAYS Initialise complete
Data Acquisition without tape server
NUCAM2 Data Acquisition now all ready to start
SIGBUS, SIGSEGV and SIGPIPE traps setup
[ OK ]
DENX ELDK version 4.2 build 2008-02-20
Linux 2.6.25-xlnx on a ppc
nnvtx4b login: /debug user "debug" password ".3su5+=lox+f"
httpd started on port 8015
custom startup for nnvtx4b
/NetVarService
/SigTaskService
XPRAYSNUCAM Access package 1.0
/XPRAYSNUCAMAccessServer
/SpectrumService
Completed custom startup from /MIDAS/config/TclHttpd/nnvtx4b/startup.tcl
After the initial kernel boot Virtex Linux mounts the root file system and follows the normal Linux startup procedure.
The MIDAS data acquisition startup is invoked by the file (seen by the Virtex system) /etc/rc3.d/S90MIDAS which is the file
(as seen by the Linux/Unix server) /MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b/etc/rc3.d/S90MIDAS. This invokes the file
/MIDAS/linux-ppc_4xx/startup/nnvtx4b which loads the required Data Acquisition application.
If you have installed the software at a different location then pathnames in the script examples which follow may need to be changed. Note however that the scripts are executed by the Virtex Linux system and so the pathnames are as seen by Virtex Linux and NOT by the Linux server. If you mount for Linux the MIDAS software as /MIDAS (see the fstab example file which follows) then no changes will be needed regards of the location on the Linux server.
If /MIDAS is a symbolic link on your Linux server then remember that you must export the file system to which the symbolic link points. This is normal NFS practice.
Generic MIDAS startup script /MIDAS/XilinxLinux/ppc_4xx/rfs/nnvtx4b/etc/rc3.d/S90MIDAS
#! /bin/bash
HOSTNAME=`/bin/hostname`
echo "Starting MIDAS Data Acquisition for $HOSTNAME"
export processor=ppc_4xx
export PATH=/MIDAS/linux-ppc_4xx/bin:${PATH}
export LD_LIBRARY_PATH=/MIDAS/linux-ppc_4xx/lib:${LD_LIBRARY_PATH}
[ -x /MIDAS/linux-ppc_4xx/startup/$HOSTNAME ] && /MIDAS/linux-ppc_4xx/startup/$HOSTNAME
sleep 10
Sample application specific start up script (file /MIDAS/linux-ppc_4xx/startup/nnvtx4b)
#!/bin/bash
#
UM=`umask`
umask 0
#
cd /MIDAS/linux-ppc_4xx/drivers/xpraysnucam/module
./load
sleep 5
MIDAS_MEMSAS_PORT=0
export MIDAS_MEMSAS_PORT
/MIDAS/TclHttpd/linux-ppc_4xx/TclHttpd-server&
sleep 10
#
# define netints and access registers
#
netstring TS_Server "192.168.0.1"
netint Output_BufferSize 64
netint Format_Option 3
netint Xfer_Option 3
netint Xfer_Overlap 1
netint Xfer_Priority 1
cd /MIDAS/Data_Acq/bin/linux-ppc_4xx
./ExecV1
nice ./PorgamRaysExecV1
#
umask $UM
#
Several configuration files are used by the standard startup scripts. In most cases these are supplied in the various distributions and the versions there should be considered as the definitive source.
Sample start up script for TclHttpd (file /MIDAS/config/startup/nnvtx4b/startup.tcl)
puts "custom startup for nnvtx4b"
package require NetVarServer
package require SigTaskServer
package require SpectrumServer
package require XPRAYSNUCAMAccessServer
For more information on options for the TclHttpd server see
here.
The MIDAS GUI session provides 2 extras which can be used in conjunction with the Virtex Linux systems.
1) A SIMPLE terminal emulator (See MIDAS Main Menu; Extras; Terminal).
First connect the serial terminal output from the Virtex system to either the serial comm port on the workstation (port /dev/ttyS0) or via a serial convertor to a USB port (/dev/ttyUSB0).
Linux does not by default permit normal user programs direct access to I/O ports. The MIDAS terminal program attempts to change this using the command
sudo -n chmod 666 ....
To enable this you need to add permission in the file /etc/sudoers. Add a line something like
npg localhost=NOPASSWD: /bin/chmod
to add permission for a local ID npg.
Note sudo requires that the file /etc/sudoers has the permissions 440 when used.
2) If software (Linux kernel) is loaded into the Virtex FPGA via JTAG using a Xilinx USB programmer then some system changes are needed in order to get the Xilinx program which does this (xmd) to run.
a) Comment out the following lines from /etc/rc.local (these are added by the Xilinx software install).
#/lib/modules/.../kernel/drivers/misc/install_windrvr6/windrvr6 no
#/lib/modules/.../kernel/drivers/misc/install_xpc4drvr
b) copy libusb-driver.so into /opt/Xilinx/10.1 (already done for libUSB 1.0.6 which needs package libusb-devel in order to build)
c) add the line
setenv LD_PRELOAD /opt/Xilinx/10.1/libusb-driver.so
to the head of /opt/Xilinx/10.1/ISE.csh
d) If needed install udev and/or enable it
add the file /etc/udev/rules.d/xusbdfwu.rules (obtain from Xilinx)
# version 0003
SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="0008", MODE="666"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="0007", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusbdfwu.hex -D $TEMPNODE"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="0009", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xup.hex -D $TEMPNODE"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="000d", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_emb.hex -D $TEMPNODE"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="000f", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xlp.hex -D $TEMPNODE"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="0013", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xp2.hex -D $TEMPNODE"
BUS=="usb", ACTION=="add", SYSFS{idVendor}=="03fd", SYSFS{idProduct}=="0015", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xse.hex -D $TEMPNODE"
and the file /etc/udev/rules.d/libusb-driver.rules
ACTION=="add", BUS=="usb", SYSFS{idVendor}=="03fd", MODE="666"
then restart udev
e) enable the yum repository by in /etc/yum.repos.d/atrpms.repo changing the line enabled=0 to enabled=1
f) install the package fxload.x86_64 which is on the repository and is used by xusbdfwu.rules by the command
yum install fxload.x86_64
disconnect the Xilinx programmer from the USB bus (if connected) and then reconnect.
return to MIDAS download home page
