VME_UNIVERSE: VMEbus Driver and Tools for Linux

Contents:

Abstract
Installation
Loading and Unloading the Module
Utility Programs
Writing Applications Using the vme_universe Application Program Interface (API)
Master Windows
Slave Windows
Direct Memory Access (DMA)
Interrupts
Using Module Parameters
Accessing the VMEbus Using a Kernel Module
Reserving RAM at Boot Time
Debugging Tips
Changes Since Version 2.x
License

Abstract

This document describes installation, configuration and usage of the vme_universe project for devices using the Tundra Universe II VMEbus-PCI bridge. The vme_universe project provides a loadable Linux device driver module, an API library for accessing the VMEbus and a set of utilities for quick access to the VMEbus. The utilities also serve as C code examples for programmatically accessing the VMEbus.

This document assumes that you have some knowledge of the Linux operating system, C programming, and the VMEbus protocol.

Installation

Building

Warning: Linux kernel source code must be installed to build the driver module.

The driver, library and all utilities are supplied as source code. To use the vme_universe driver and tools, they must first be compiled (built) into executable code and installed. The next code listing illustrates how to compile and install the vme_universe project code.

Code listing 1: Compiling and Installing the VME_UNIVERSE Project

// From the vme_universe base directory execute:
sh$ make
sh# make install

Verifying the Installation

If the project is built and installed correctly, you should have the following files are on your system:

Code listing 2: Verify the Installation

// This is the driver module.  Make sure you use `uname -r`, not 'uname -r'.
sh$ ls /lib/modules/`uname -r`/kernel/drivers/vme/
vme_universe.o
// This is the library
sh$ ls /usr/lib/libvme*
/usr/lib/libvme.so    /usr/lib/libvme.so.3   /usr/lib/libvme.so.3.3 
// These are the header files
sh$ ls /usr/include/vme
universe.h  vmi_api.h  vme.h  vmivme.h
// These are the utilities
sh$ ls /usr/bin/vme*
/usr/bin/vme_acquire_bus           /usr/bin/vme_poke
/usr/bin/vme_catch_interrupt       /usr/bin/vme_release_bus
/usr/bin/vme_dma_read              /usr/bin/vme_rmw
/usr/bin/vme_dma_write             /usr/bin/vme_slave_peek
/usr/bin/vme_endian                /usr/bin/vme_slave_poke
/usr/bin/vme_generate_interrupt    /usr/bin/vme_sysreset
/usr/bin/vme_peek

Loading and Unloading the Module

Use the modprobe command to load the driver module by entering modprobe vme_universe.

Use the lsmod command command to verify that the module loaded successfully. When you enter lsmod, the module name vme_universe should appear in the output.

To unload the module, enter modprobe -r vme_universe.

Utility Programs

Utilities

The following VMEbus utilities are supplied to enable quick access to the VMEbus.

vme_acquire_bus
vme_catch_interrupt
vme_dma_read
vme_dma_write
vme_endian
vme_generate_interrupt
vme_peek
vme_poke
vme_release_bus
vme_rmw
vme_slave_peek
vme_slave_poke
vme_sysreset

Utilities as Examples

Full source code for all of the utilities listed above are available in the test directory. The test utilities source code can be used as example code for writing applications using the vme_universe API.

Writing Applications Using the vme_universe Application Program Interface (API)

Functions

The vme_universe project provides a shared object library for use in applications that access the VMEbus. The following is a list of functions provided by this library:

vme_acquire_bus_ownership
vme_dma_buffer_create
vme_dma_buffer_map
vme_dma_buffer_phys_addr
vme_dma_buffer_release
vme_dma_buffer_unmap
vme_dma_read
vme_dma_write
vme_get_arbitration_mode
vme_get_arbitration_timeout
vme_get_bus_ownership
vme_get_bus_release_mode
vme_get_bus_request_level
vme_get_bus_request_mode
vme_get_bus_timeout
vme_get_endian_conversion_bypass
vme_get_master_endian_conversion
vme_get_max_retry
vme_get_posted_write_count
vme_get_slave_endian_conversion
vme_init
vme_interrupt_attach
vme_interrupt_generate
vme_interrupt_release
vme_location_monitor_create
vme_location_monitor_release
vme_master_window_create
vme_master_window_map
vme_master_window_phys_addr
vme_master_window_release
vme_master_window_translate
vme_master_window_unmap
vme_read_modify_write
vme_register_image_create
vme_register_image_release
vme_release_bus_ownership
vme_set_arbitration_mode
vme_set_arbitration_timeout
vme_set_bus_release_mode
vme_set_bus_request_level
vme_set_bus_request_mode
vme_set_bus_timeout
vme_set_endian_conversion_bypass
vme_set_master_endian_conversion
vme_set_max_retry
vme_set_posted_write_count
vme_set_slave_endian_conversion
vme_slave_window_create
vme_slave_window_map
vme_slave_window_phys_addr
vme_slave_window_release
vme_slave_window_unmap
vme_sysreset
vme_term

Compiling and Linking the Application

To use the API functions listed above, your code must include the header files vme/vme.h and vme/vme_api.h. The application must also be linked with the libvme.so library by including the switch -lvme to the compile command. The next code listing is a small example application that initializes the VMEbus interface.

Code listing 3: A Simple VMEbus Application

// Here is a simple piece of code to try compiling and running.
// We'll call it hello_vme.c

#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>

int main()
{
	vme_bus_handle_t bus_handle;

	if (vme_init(&bus_handle)) {
		perror("Cannot initialize the VMEbus");
		return -1;
	}

	printf("We're ready to start accessing the bus now!\n");

	vme_term(bus_handle);
	return 0;
}


// To compile the application run:
sh$ gcc hello_vme.c -o hello_vme -lvme
// If you have not already done so, load the driver.
sh# modprobe vme_universe
// Now let's run our code.
sh$ ./hello_vme
We're ready to start accessing the bus now!

Master Windows

Master Windows enable access to other boards across the VMEbus.

Functions

The following functions are used when accessing the VMEbus using Master Windows:

vme_master_window_create
vme_master_window_map
vme_master_window_phys_addr
vme_master_window_release
vme_master_window_translate
vme_master_window_unmap
vme_read_modify_write

Examples

The next example uses the Master Window API functions to read data from the VMEbus.

Code listing 4: peek.c

// I wrote this example to access a VMIVME-5588DMA reflected memory board addressed
// at VMEbus address 0x08000000 in A32 space.
// You should change VME_ADDRESS, ADDRESS_MODIFIER, and NBYTES to access a board
// installed in your chassis


#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>

#define VME_ADDRESS       0x08000000
#define ADDRESS_MODIFIER  VME_A32SD
#define NBYTES            0x40

int main()
{
	vme_bus_handle_t bus_handle;
	vme_master_handle_t window_handle;
	uint8_t *ptr;
	int ii;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_master_window_create(bus_handle, &window_handle,
				     VME_ADDRESS, ADDRESS_MODIFIER, NBYTES,
				     VME_CTL_PWEN, NULL)) {
		perror("Error creating the window");
		vme_term(bus_handle);
		return -1;
	}

	ptr = vme_master_window_map(bus_handle, window_handle, 0);
	if (!ptr) {
		perror("Error mapping the window");
		vme_master_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	/* Print the VMEbus data */
	for (ii = 0; ii < NBYTES; ++ii, ++ptr) {
		printf("%.2x ", *ptr);
		/* Add a newline every 16 bytes */
		if (!((ii + 1) % 0x10))
			printf("\n");
	}
	printf("\n");

	if (vme_master_window_unmap(bus_handle, window_handle)) {
		perror("Error unmapping the window");
		vme_master_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	if (vme_master_window_release(bus_handle, window_handle)) {
		perror("Error releasing the window");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}



// To compile the application run:
sh$ gcc peek.c -o peek -lvme
// And run it
sh$ ./peek
ff 4b 70 ff 01 e0 ff 00 ff ff ff ff ff ff ff ff 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 ff ff 00 00 ff ff 00 00 ff ff 00 00 
ff ff ff 0f ff ff ff 0f ff ff ff 0f ff ff ff 0f

Note: For performance reasons, using the flag VME_CTL_PWEN to vme_master_window_create is recommended. This flag turns on write posting.

vme_master_window_create creates a VMEbus window that enables access to the VMEbus address, VME_ADDRESS, in the address space determined by ADDRESS_MODIFIER. The window is at least NBYTES large. Note that vme_master_window_map returns a pointer. To read data from the VMEbus, dereference the returned pointer as seen in the code printf("%.2x ", *ptr);.

Note: If peek prints all FF's, this may indicate a bus error. Bus errors occur when no board on the VMEbus answers the access at the provided address and address modifier. If this occurs, ensure the addressing of your target board and the VME_ADDRESS and ADDRESS_MODIFIER are correct. See Debugging tips to learn more about how to debug bus errors.

Here is an example of how to turn the front panel LED on and off on the VMIVME-5588DMA board. If you have a board in your chassis that has an LED, modify the addresses and offsets to turn the LED on and off on your board.

Turning an LED on and off is the embedded systems equivalent of "hello world".

Code listing 5: toggle_led.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>

#define VME_ADDRESS       0x08000000
#define ADDRESS_MODIFIER  VME_A32SD
#define NBYTES            0x40

int main()
{
	vme_bus_handle_t bus_handle;
	vme_master_handle_t window_handle;
	uint8_t *ptr;
	uint8_t *csr;		/* The CSR register controls the LED */
	const int offset = 0x5;	/* The CSR register is offset 5 bytes from
				   VME_ADDRESS */
	const int bit = 7;	/* Bit 7 controls the LED */

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_master_window_create(bus_handle, &window_handle,
				     VME_ADDRESS, ADDRESS_MODIFIER, NBYTES,
				     VME_CTL_PWEN, NULL)) {
		perror("Error creating the window");
		vme_term(bus_handle);
		return -1;
	}

	ptr = vme_master_window_map(bus_handle, window_handle, 0);
	if (!ptr) {
		perror("Error mapping the window");
		vme_master_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	/* Toggle the LED */
	csr = ptr + offset;
	if (*csr & (1 << bit)) {
		*csr &= ~(1 << bit);
		printf("The LED should now be off\n");
	} else {
		*csr |= (1 << bit);
		printf("The LED should now be on\n");
	}

	if (vme_master_window_unmap(bus_handle, window_handle)) {
		perror("Error unmapping the window");
		vme_master_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	if (vme_master_window_release(bus_handle, window_handle)) {
		perror("Error releasing the window");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc toggle_led.c -o toggle_led -lvme

toggle_led reads the current state of the LED by dereferencing the csr pointer and, depending on the current state, writes either 1 or 0 to the register LED bit to turn the LED on or off. The lines *csr &= ~(1 << bit); and *csr |= (1 << bit); actually make two accesses to the VMEbus, a read cycle and a write cycle. (This is more apparent if you read *csr |= (1 << bit); as *csr = *csr | (1 << bit);).

See the vme_peek and vme_poke source code for more examples of writing an application that accesses the VMEbus using the Master Window interface.

Advanced Functions

The Tundra Universe VMEbus-PCI bridge chip Master Window interface works by creating a one-to-one mapping of PCI bus address space to VMEbus address space. Sometimes it is useful to know which PCI address is used to access a VMEbus window. The vme_master_window_phys_addr function returns the base PCI address of a window.

Because the amount of available PCI address space is limited (4GB on a 32-bit machine, some of which is taken up by physical RAM, peripheral devices, etc.) it is possible you will need access to more VMEbus memory than can be mapped to the PCI bus at one time. A possible workaround is to create a window, access it, and then release the window so PCI address space is available to create a new window shifted to the next range of addresses. Although this solution works, it incurs a lot of overhead. The vme_master_window_translate function is the recommended workaround to this issue. vme_master_window_translate shifts the one-to-one mapping of PCI space to VMEbus space by reusing the range of PCI address space already assigned to a window to access a different range of VMEbus addresses.

VMEbus read-modify-write cycles can be generated using the vme_read_modify_write function. For an example on the use of this function, refer to the source code for the vme_rmw utility.

Slave Windows

Slave Windows can be thought of as shared memory on the VMEbus. A Slave Window is used to expose a specific region of the PCI bus address space to the VMEbus. Most often this is done to expose a portion of the local RAM to the VMEbus so that it can be accessed by other boards in your VME chassis, but any device with a PCI bus address can be exposed to the VMEbus.

The vme_slave_window_create function opens a Slave Window to the VMEbus. If the physical address is not specified, this function attempts to allocate RAM to share with the VMEbus at the specified VMEbus address. If a physical address is specified, that PCI address is mapped to the specified VMEbus address.

Functions

The following functions are used when accessing VMEbus Slave Windows:

vme_slave_window_create
vme_slave_window_map
vme_slave_window_phys_addr
vme_slave_window_release
vme_slave_window_unmap

Examples

Here is an example demonstrating how to create and access a VMEbus Slave Window:

Code listing 6: slave_peek.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>

#define VME_ADDRESS       0x00040000
#define ADDRESS_SPACE     VME_A24
#define NBYTES            0x10

int main()
{
	vme_bus_handle_t bus_handle;
	vme_slave_handle_t window_handle;
	uint8_t *ptr;
	int ii;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_slave_window_create(bus_handle, &window_handle,
				    VME_ADDRESS, ADDRESS_SPACE, NBYTES,
				    VME_CTL_PWEN | VME_CTL_PREN, NULL)) {
		perror("Error creating the window");
		vme_term(bus_handle);
		return -1;
	}

	ptr = vme_slave_window_map(bus_handle, window_handle, 0);
	if (!ptr) {
		perror("Error mapping the window");
		vme_slave_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	/* Print the data */
	for (ii = 0; ii < NBYTES; ++ii, ++ptr) {
		printf("%.2x ", *ptr);
		/* Add a newline every 16 bytes */
		if (!((ii + 1) % 0x10))
			printf("\n");
	}
	printf("\n");

	if (vme_slave_window_unmap(bus_handle, window_handle)) {
		perror("Error unmapping the window");
		vme_slave_window_release(bus_handle, window_handle);
		vme_term(bus_handle);
		return -1;
	}

	if (vme_slave_window_release(bus_handle, window_handle)) {
		perror("Error releasing the window");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc slave_peek.c -o slave_peek -lvme 
// And run it
sh$ ./slave_peek
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Note: For performance reasons, it is recommended that you use the VME_CTL_PWEN and VME_CTL_PREN flags to vme_slave_window_create. These flags turn on write posting and read prefetch respectively.

Warning: When NULL is specified for the phys_addr parameter of vme_slave_window_create the driver attempts to allocate contiguous physical memory. The amount of physically contiguous RAM available for allocation is limited, and will vary depending on the specific system. Since the amount depends on the specific system, a request for large amounts of RAM using vme_slave_window_create will sometimes fail.

To guarantee that memory will be available for the Slave Window, reserve memory at boot time and pass the address of the physical memory as the phys_addr parameter of vme_slave_window_create. The next code listing illustrates this concept:

Code listing 7: Using RAM Reserved at Boot Time for a Slave Window

// Assume that we have reserved the top 2MB of memory on a 64MB machine.
// The base address of our reserved memory begins at 0x3e00000 so here is the
// call we would make to use that memory as slave ram.

#define MB  1024 * 1024
	if(vme_slave_window_create(bus_handle, &window_handle,
	                           VME_ADDRESS, ADDRESS_SPACE, 2 * MB,
	                           VME_CTL_PWEN | VME_CTL_PREN, 62 * MB)) {
		perror("Error creating the window");
		vme_term(bus_handle);
		return -1;
	}

Warning: In older kernel versions, specifying mem=62M ensured that the base address of the reserved memory would be located at memory address 0x3e00000 (1024 * 1024 * 62). This cannot be assumed with newer kernel versions. See /proc/iomem to determine where the top of the kernel's physical RAM is located.

See the vme_slave_peek and vme_slave_poke source code for more examples of how to write applications that use the Slave Window interface.

Direct Memory Access (DMA)

The Tundra Universe II chip can initiate DMA transfers to access the VMEbus.

Functions

The following functions are used when accessing the VMEbus using the DMA engine:

vme_dma_buffer_create
vme_dma_buffer_map
vme_dma_buffer_phys_addr
vme_dma_buffer_release
vme_dma_buffer_unmap
vme_dma_read
vme_dma_write

Examples

In this example, the peek.c program has been modified to transfer data using DMA:

Code listing 8: dma_peek.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>

#define VME_ADDRESS       0x08000000
#define ADDRESS_MODIFIER  VME_A32SD
#define NBYTES            0x40

int main()
{
	vme_bus_handle_t bus_handle;
	vme_dma_handle_t dma_handle;
	uint8_t *ptr;
	int ii;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_dma_buffer_create(bus_handle, &dma_handle, NBYTES, 0, NULL)) {
		perror("Error creating the buffer");
		vme_term(bus_handle);
		return -1;
	}

	ptr = vme_dma_buffer_map(bus_handle, dma_handle, 0);
	if (!ptr) {
		perror("Error mapping the buffer");
		vme_dma_buffer_release(bus_handle, dma_handle);
		vme_term(bus_handle);
		return -1;
	}

	/* Transfer the data */
	if (vme_dma_read(bus_handle, dma_handle, 0, VME_ADDRESS,
			 ADDRESS_MODIFIER, NBYTES, 0)) {
		perror("Error reading data");
		vme_dma_buffer_unmap(bus_handle, dma_handle);
		vme_dma_buffer_release(bus_handle, dma_handle);
		vme_term(bus_handle);
		return -1;
	}

	/* Print the VMEbus data */
	for (ii = 0; ii < NBYTES; ++ii, ++ptr) {
		printf("%.2x ", *ptr);
		/* Add a newline every 16 bytes */
		if (!((ii + 1) % 0x10))
			printf("\n");
	}
	printf("\n");

	if (vme_dma_buffer_unmap(bus_handle, dma_handle)) {
		perror("Error unmapping the buffer");
		vme_dma_buffer_release(bus_handle, dma_handle);
		vme_term(bus_handle);
		return -1;
	}

	if (vme_dma_buffer_release(bus_handle, dma_handle)) {
		perror("Error releasing the buffer");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc dma_peek.c -o dma_peek -lvme 
// And run it
sh$ ./dma_peek
ff 70 4b ff 00 ff e0 01 ff ff ff ff ff ff ff ff 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 ff ff ff 00 ff ff ff 00 ff ff ff 
0f ff ff ff 0f ff ff ff 0f ff ff ff 0f ff ff ff

// But wait! There seems to be some sort of byte swapping issue.
// Notice that the output of our peek program was:
ff 4b 70 ff 01 e0 ff 00 ff ff ff ff ff ff ff ff
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 ff ff 00 00 ff ff 00 00 ff ff 00 00
ff ff ff 0f ff ff ff 0f ff ff ff 0f ff ff ff 0f

The peek function dereferenced the pointer and transferred data across the VMEbus one byte at a time. The dma_peek application printed the data from the local memory buffer one byte at a time by dereferencing a pointer, however, the data was transferred across the VMEbus 32 bits at a time because the default data width for vme_dma_read with address modifier VME_A32SD is 32 bits. This inconsistancy in data widths is what caused the byte-swapping issue. To correct the problem, use a data width flag in the vme_dma_read function call as illustrated in the next code listing.

Code listing 9: Specifying the Data Width to vme_dma_read

// So change the vme_dma_read command to

	if(vme_dma_read(bus_handle, dma_handle, 0, VME_ADDRESS,
       	                ADDRESS_MODIFIER, NBYTES, VME_DMA_DW_8))

sh$ gcc dma_peek.c -o dma_peek -lvme 
sh$ ./dma_peek
ff 4b 70 ff 01 e0 ff 00 ff ff ff ff ff ff ff ff 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 ff ff 00 00 ff ff 00 00 ff ff 00 00 
ff ff ff 0f ff ff ff 0f ff ff ff 0f ff ff ff 0f

// Thats more like it!

Note: Not all data widths are available for all address modifiers. For example, the data width flag VME_DMA_DW_64 cannot be used with address modifier VME_A32SD, because the Tundra Universe II chip forces multiplex block transfers any time the data switch is set to 64-bits. Conversely, if the address modifier is VME_A32MBLT, the data width must be 64-bit. 8-bit data width transfers are not available for block transfer address modifiers.

Warning: When NULL is specified for the phys_addr parameter of vme_dma_buffer_create the driver attempts to allocate contiguous physical memory. The amount of physically contiguous RAM available for allocation is limited, and will vary depending on the specific system. Since the amount depends on the specific system, a request for large amounts of RAM using vme_dma_buffer_create will sometimes fail.

To guarantee that memory will be available for the DMA buffer, reserve memory at boot time and pass the address of the physical memory as the phys_addr parameter of vme_dma_buffer_create. Here is an example using this concept:

Code listing 10: Using RAM Reserved at Boot Time for a DMA Buffer

// Assume that we have reserved the top 2MB of memory on a 64MB machine.
// The base address of our reserved memory begins at 0x3e00000 so here is the
// call we would make to use that memory as a dma buffer.

#define MB  1024 * 1024
	if(vme_dma_buffer_create(bus_handle, &dma_handle, 2 * MB, 0,
       	                         62 * MB)) {
		perror("Error creating the buffer");
		vme_term(bus_handle);
		return -1;
	}

Warning: In older kernel versions, specifying mem=62M ensured that the base address of the reserved memory would be located at memory address 0x3e00000 (1024 * 1024 * 62). This cannot be assumed with newer kernel versions. See /proc/iomem to determine where the top of the kernel's physical RAM is located.

See the vme_dma_read and vme_dma_write source code for more examples of how to write applications that use VMEbus DMA.

Interrupts

The vme_universe project enables the generation of interrupts on the VMEbus, as well as notification of when VMEbus interrupts and other events relating to the VMEbus occur.

Functions

The following functions are used to handle VMEbus interrupts:

vme_interrupt_attach
vme_interrupt_generate
vme_interrupt_release

VMEbus Interrupts

There are two methods for being notified when an interrupt occurs:

The first method is to make a function call, vme_interrupt_attach, that blocks until the interrupt occurs. Here is an example using the vme_interrupt_attach function call to wait for an interrupt:

Code listing 11: get_interrupt.c (Method 1)



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <signal.h>
#include <stdio.h>

#define LEVEL             VME_INTERRUPT_VIRQ3
#define VECTOR            0x10


int main()
{
	vme_bus_handle_t bus_handle;
	vme_interrupt_handle_t interrupt_handle;
	int data;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_interrupt_attach(bus_handle, &interrupt_handle, LEVEL,
				 VECTOR, VME_INTERRUPT_BLOCKING, &data)) {
		perror("Error attaching to the interrupt");
		vme_term(bus_handle);
		return -1;
	}

	/* For VMEbus interrupts, the returned data is (level << 8) & vector */
	printf("VMEbus interrupt occured on level %d, vector 0x%x\n",
	       data >> 8, data & 0xff);

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc get_interrupt.c -o get_interrupt -lvme 
// And run it
sh$ ./get_interrupt
VMEbus interrupt occured on level 3, vector 0x10

The vme_interrupt_attach function call returns an integer data parameter containing the level and vector of the interrupt.

The second interrupt notification method uses Posix real-time signals. In this example we request that the driver send a signal to the process in response to the interrupt.

Code listing 12: get_interrupt.c (Method 2)



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <signal.h>
#include <stdio.h>

#define LEVEL             VME_INTERRUPT_VIRQ3
#define VECTOR            0x10

int done = 0;


void handler(int sig, siginfo_t * siginfo, void *extra)
{
	/* For VMEbus interrupts, the returned data is (level << 8) & vector */
	printf("VMEbus interrupt occured on level %d, vector 0x%x\n",
	       siginfo->si_value.sival_int >> 8,
	       siginfo->si_value.sival_int & 0xff);

	done = 1;
}


int main()
{
	vme_bus_handle_t bus_handle;
	vme_interrupt_handle_t interrupt_handle;
	struct sigevent event;
	struct sigaction action;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	/* Set up sigevent struct */
	event.sigev_signo = SIGIO;
	event.sigev_notify = SIGEV_SIGNAL;
	event.sigev_value.sival_int = 0;

	/* Install the signal handler */
	sigemptyset(&action.sa_mask);
	action.sa_sigaction = handler;
	action.sa_flags = SA_SIGINFO;

	if (sigaction(SIGIO, &action, NULL)) {
		perror("Error installing signal handler");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_interrupt_attach(bus_handle, &interrupt_handle, LEVEL,
				 VECTOR, VME_INTERRUPT_SIGEVENT, &event)) {
		perror("Error attaching to the interrupt");
		vme_term(bus_handle);
		return -1;
	}

	/* Loop here until we get the signal */
	while (!done) ;

	if (vme_interrupt_release(bus_handle, interrupt_handle)) {
		perror("Error releasing the interrupt");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc get_interrupt.c -o get_interrupt -lvme 
// And run it
sh$ ./get_interrupt
VMEbus interrupt occured on level 3, vector 0x10

Important: Note that this method of using get_interrupt.c makes a call to vme_interrupt_release, while the first method did not make this call.

Mailbox Interrupts

Mailbox interrupts are handled similarly to the way VMEbus interrupts are handled. Here is an example using mailbox interrupts:

Code listing 13: get_mailbox.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>


#define LEVEL             VME_INTERRUPT_MBOX3
#define VME_ADDRESS       0x0
#define ADDRESS_SPACE     VME_A16


int main()
{
	vme_bus_handle_t bus_handle;
	vme_vrai_handle_t vrai_handle;
	vme_interrupt_handle_t interrupt_handle;
	int data;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}
	if (vme_register_image_create(bus_handle, &vrai_handle, VME_ADDRESS,
					  ADDRESS_SPACE, 0)) {
		perror("Error creating VMEbus register image");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_interrupt_attach(bus_handle, &interrupt_handle, LEVEL,
				     0, VME_INTERRUPT_BLOCKING, &data)) {
		perror("Error attaching to the interrupt");
		vme_register_image_release(bus_handle, vrai_handle);
		vme_term(bus_handle);
		return -1;
	}

	printf("Mailbox interrupt occurred, data is 0x%x\n", data);

	if (vme_register_image_release(bus_handle, vrai_handle)) {
		perror("Error releasing VMEbus register image");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


To trigger mailbox interrupts, the register space of the Tundra Universe II chip must be exposed to the VMEbus. The vme_register_image_create function call exposes the Tundra Universe II's registers to the VMEbus at address 0x0 in A16 space.

Warning: Use a different address if a board is already mapped to address 0x0 in A16 space.

To trigger the mailbox interrupt, a value must be written to the Tundra Universe II's mailbox register. There are four mailboxes [0-3] with address offsets of 0x348, 0x34C, 0x350 and 0x354, respectively. Since we are using mailbox 3 (0x354) in get_mailbox, we must use address offset 0x354 to trigger the interrupt. To do this, we will use another VMIC single board computer in the same chassis and write to the register using the vme_poke utility.

Code listing 14: Triggering the Mailbox Interrupt

// First let's verify that we can access the Tundra device.
sh$ vme_peek -aVME_A16S -A0x0 -dVME_D16
10e3  // 10e3 is Tundra's vendor id
// Now write to the mailbox register to trigger the interrupt.
sh$ vme_poke -aVME_A16S -A0x354 a5

This should cause the get_mailbox application to receive an interrupt.

Location Monitor Interrupts

Location monitor interrupts are handled in the same way as VMEbus interrupts. Here is an example that illustrates handling Location Monitor interrupts:

Code listing 15: get_lm.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <stdio.h>


#define LEVEL             VME_INTERRUPT_LM2
#define VME_ADDRESS       0x08000000
#define ADDRESS_SPACE     VME_A32


int main()
{
	vme_bus_handle_t bus_handle;
	vme_lm_handle_t lm_handle;
	vme_interrupt_handle_t interrupt_handle;
	int data;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}
	if (vme_location_monitor_create(bus_handle, &lm_handle, VME_ADDRESS,
					ADDRESS_SPACE, 0, 0)) {
		perror("Error creating VMEbus location monitor image");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_interrupt_attach(bus_handle, &interrupt_handle, LEVEL, 0,
				 VME_INTERRUPT_BLOCKING, &data)) {
		perror("Error attaching to the interrupt");
		vme_location_monitor_release(bus_handle, lm_handle);
		vme_term(bus_handle);
		return -1;
	}

	printf("Location monitor interrupt\n");

	if (vme_location_monitor_release(bus_handle, lm_handle)) {
		perror("Error releasing VMEbus location monitor image");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc get_lm.c -o get_lm -lvme
// And run it as a background task
sh$ ./get_lm &
// Now access the location monitor window to cause an interrupt
sh$ vme_peek -aVME_A32SD -A0x08000010
Location monitor interrupt
00

The location monitor image area covers 0x1000 bytes, starting at the base address supplied to the vme_location_monitor_create function call. Address bits 3 and 4 determine which of the four location monitor interrupts of VME_INTERRUPT_LM[0-3] are triggered.

Here is another Location Monitor interrupt example using the vme_catch_interrupt utility to demonstrate the mapping of location monitor interrupts. For this example we will use the location_monitor module parameter to create the location monitor image. Module parameters are discussed in greater detail later in this document.

Code listing 16: Location Monitors

// If the module is already loaded, then we will need to unload it
sh# modprobe -r vme_universe
// Load the module with the location_monitor parameter specifying a location
// monitor image at address 0x08000000 in A32 space.
sh# modprobe vme_universe location_monitor=0x80F20000,0x08000000
// Verify that we created the location monitor image by looking at the proc page.
sh$ cat /proc/vme/ctl

comm=0x847

master control=0x80d00000

miscellaneous control=0x35060000

location monitor:
  control=0x80f20000
  base=0x8000000

// Yep, our location monitor image is there
// Now we'll start handlers for all of the location monitor interrupts
sh$ vme_catch_interrupt -lVME_INTERRUPT_LM0 -fVME_INTERRUPT_SIGEVENT &
sh$ vme_catch_interrupt -lVME_INTERRUPT_LM1 -fVME_INTERRUPT_SIGEVENT &
sh$ vme_catch_interrupt -lVME_INTERRUPT_LM2 -fVME_INTERRUPT_SIGEVENT &
sh$ vme_catch_interrupt -lVME_INTERRUPT_LM3 -fVME_INTERRUPT_SIGEVENT &
// Now we'll trigger the location monitor interrupts in order.
sh$ vme_peek -aVME_A32SD -A0x08000000
00
Location monitor interrupt level 0
sh$ vme_peek -aVME_A32SD -A0x08000008
00
Location monitor interrupt level 1
sh$ vme_peek -aVME_A32SD -A0x08000010
00
Location monitor interrupt level 2
sh$ vme_peek -aVME_A32SD -A0x08000018
00
Location monitor interrupt level 3

Warning: Even though there is a VMIVME-5588DMA card at location A32SD 0x08000000 in the example shown, the output was always 00 because the Tundra Universe II chip generates DTACK and terminates the bus cycle when an attempt is made to access the addresses of its own location monitor image. This results in the Location Monitor image masking its VMEbus address range from the user.

Bus Error Interrupts

Bus error interrupts are handled similarly to all other interrupts. The data parameter of vme_interrupt_attach returns the address of the bus error for blocking calls and, for signals, the value of the si_value.sival_int element of the siginfo struct returns the address at which the bus error occurred.

An example of handling bus error interrupts is left as an exercise for the reader.

Note: The returned address is always an even value, even if an odd value address is accessed. This is a VMEbus addressing artifact and not an error in the driver's operation.

Generating VMEbus Interrupts

The Tundra Universe II chip can be used to generate interrupts on the VMEbus. Here is an example:

Code listing 17: gen_interrupt.c



#include <vme/vme.h>
#include <vme/vme_api.h>
#include <signal.h>
#include <stdio.h>

#define LEVEL             VME_INTERRUPT_VIRQ3
#define VECTOR            0x10


int main()
{
	vme_bus_handle_t bus_handle;

	if (vme_init(&bus_handle)) {
		perror("Error initializing the VMEbus");
		return -1;
	}

	if (vme_interrupt_generate(bus_handle, LEVEL, VECTOR)) {
		perror("Error generating the interrupt");
		vme_term(bus_handle);
		return -1;
	}

	if (vme_term(bus_handle)) {
		perror("Error terminating");
		return -1;
	}

	return 0;
}


// To compile the application run:
sh$ gcc gen_interrupt.c -o gen_interrupt -lvme

Note: Currently, the driver for the Tundra Universe II device can only generate interrupts with even vectors. An error is returned if an odd vector is passed to the vme_interrupt_generate function call.

Using Module Parameters

The following section lists the module parameters for the vme_universe driver and a description of how to use them.

Parameters

The default behavior of the VMEbus bridge device can be changed by modifying specific vme_universe driver parameters. In most cases, the driver's default register values are sufficient and these parameters do not need to be modified.

comm

The comm register specifies a value to the comm register for VMIC single board computers. The register is specified below:

Register 1: Comm

Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
ReservedVME_ENBYPASSReservedWTDSYS
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
ReservedBERRIBTOVBTOABLESECMEC
Bit 11: VME_EN (VMEbus enable) - Must be set to enable access to the VMEbus. 0=Disabled, 1=Enabled.
Bit 10: BYPASS (VMEbus endian conversion bypass) - Bypasses the VMEbus endian conversion hardware. 0=Do not bypass endian conversion, 1=Bypass endian conversion.
Bit 8: WTDSYS - Map watchdog timeout to cause VMEbus sysfail. 0=Disabled, 1=Enabled.
Bit 6: BERRI (Auxiliary bus error interrupt enable) 0=Disabled, 1=Enabled.
Bits 5 through 4: BTOV (Auxiliary bus error timeout value) 00=16 us, 01=64 us, 10=256 us, 11=1 ms.
Bit 3: BTO (Auxiliary bus error timeout timer enable) 0=Disabled, 1=Enabled.
Bit 2: ABLE (Auxiliary bus error logic enable) 0=Disabled, 1=Enabled.
Bit 1: SEC (Slave endian conversion enable) 0=Disabled, 1=Enabled.
Bit 0: MEC (Master endian conversion enable) 0=Disabled, 1=Enabled.

master_control

The master_control parameter sets the value of the Tundra Universe II master control register. This register is specified below:

Register 2: Master Control

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
MAXRTRYPWON
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
VRLVRMVRELVOWNReserved
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
ReservedPABSReserved
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
BUS_NO
Bits 31 through 28: MAXRTRY - Maximum number of retries. 0000=Retry forever, 0000-1111=Multiple of 64; number of retries before the PCI master interface signals an error.
Bits 27 through 24: PWON (Posted write transfer count) - Transfer count at which the PCI slave channel posted write FIFO gives up the VME master interface. 0000=128 bytes, 0001=256 bytes, 0010=512 bytes, 0011=1024 bytes, 0100=2048 bytes, 0101=4096 bytes, 1111=Early release of BBSY.
Bits 23 through 22: VRL (VMEbus request level) 00=Level 0, 01=Level 1, 10=Level 2, 11=Level 3.
Bit 21: VRM (VMEbus request mode) 0=Demand, 1=Fair.
Bit 20: VREL (VMEbus release mode) 0=Release when done, 1=Release on request.
Bit 19: VOWN (VMEbus ownership bit) 0=VMEbus not captured, 1=Capture and hold the VMEbus.
Bits 13 through 12: PABS (PCI aligned burst size) - Controls the PCI address boundary at which the Universe II breaks up a PCI transaction in the VME slave channel and the DMA channel. This field also determines when the PCI master module as part of the VME slave channel will request the PCI bus. 00=32 bytes, 01=64 bytes, 10=128 bytes.
Bits 7 through 0: BUS_NO (PCI bus number)

miscellaneous_control

The miscellaneous_control parameter sets the value of the Tundra Universe II miscellaneous control register. This register is specified below:

Register 3: Miscellaneous Control

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
VBTOReservedVARBVARBTO
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
SW_LRSTSW_SRSTReservedBIENGBIReserved
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
Reserved
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
Reserved
Bits 31 through 28: VBTO (VMEbus timeout) 0000=Disabled, 0001=16 us, 0010=32 us, 0011=64 us, 0100=128 us, 0101=256 us, 0110=512 us, 0111=1024 us.
Bit 26: VARB (VMEbus arbitration mode) 0=Round-robin, 1=Priority.
Bits 25 through 24: VARBTO (VMEbus arbitration timeout) 00=Disable timeout, 01=16 us, 10=256 us.
Bit 23: SW_LRST (Software PCI reset) 0=No effect, 1=Initiates LRST# when the module is loaded.
Bit 22: SW_SRST (Software VMEbus reset) 0=No effect, 1=Initiates VMEbus sysreset when the module is loaded.
Bit 20: BI (Bi-mode) 0=Universe II is not in bi-mode, 1=Universe II is in bi-mode.
Bit 19: ENGBI (Global bi-mode initiator) 0=Assertion of VIRQ1 is ignored, 1=Assertion if VIRQ1 puts the Universe in bi-mode.

system_controller

The system_controller parameter specifies the setting for the Universe II VMEbus System Controller state. A value of 0 specifies the Universe II is not system controller. A value of 1 specifies the Universe II is system controller. Any other value leaves the system controller state at the default state for the Universe II.

filter

The filter parameter sets the value of the Tundra Universe II specific register. This register is specified below:

Register 4: Filter

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
Reserved
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
Reserved
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
ReservedDS0/DS1ASDTKFLTRReservedMASTt11READt27
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
ReservedPOSt28ReservedPREt28
Bit 14: DS0/DS1 (Data strobe filter) 0=Disabled, 1=Enabled.
Bit 13: AS (Address strobe filter) 0=Disabled, 1=Enabled.
Bit 12: DTKFLTR (VME DTACK* inactive filter) 0=Slower but better filter, 1=Faster but poorer filter.
Bit 10: MASTt11 (VME master parameter t11 control) - DS* high time during BLT's and MBLT's. 0=Default, 1=Faster.
Bits 9 through 8: READt27 (VME master parameter t27 control) - Delay time before DS* negation after read. 00=Default, 01=Faster, 10=No delay.
Bit 2: POSt28 (VME slave parameter t28 control) - Time between DS* and DTACK* for posted write. 0=Default, 1=Faster.
Bit 0: PREt28 (VME slave parameter t28 control) - Time between DS* and DTACK* for prefetch read. 0=Default, 1=Faster.

pci_lo_bound

The pci_lo_bound parameter specifies the lowest PCI address that can be used for a VMEbus Master Window mapping.

pci_hi_bound

The pci_hi_bound parameter specifies the highest PCI address that can be used for a VMEbus Master Window mapping.

vrai

The vrai parameter specifies the VMEbus register access image, which exposes the Tundra Universe II's registers to the VMEbus. The vrai parameter is specified as: vrai=control,address

The control value of the vrai parameter refers to the image control register value. This register is specified as:

Register 5: VRAI (VMEbus register access image control)

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
ENReserved
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
PGMSUPERReservedVAS
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
Reserved
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
Reserved
Bit 31: EN (Image enable) 0=Disabled, 1=Enabled.
Bits 23 through 22: PGM (Program/Data AM code) 01=Data, 10=Program, 11=Both.
Bits 21 through 20: SUPER (Supervisor/User AM code) 01=Non-privledged, 10=Supervisor, 11=Both.
Bits 17 through 16: VAS (VMEbus address space) 00=A16, 01=A24, 10=A32.

The address value of the vrai parameter sets the VMEbus address to which the registers will be mapped.

location_monitor

The location_monitor parameter specifies a location monitor image, which enables the use of location monitor interrupts. The location_monitor parameter is specified as: location_monitor=control,address

The control value of the location_monitor parameter refers to the image control register value. This register is specified below:

Register 6: Location Monitor Control

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
ENReserved
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
PGMSUPERReservedVAS
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
Reserved
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
Reserved
Bit 31: EN (Image enable) 0=Disabled, 1=Enabled.
Bits 23 through 22: PGM (Program/Data AM code) 01=Data, 10=Program, 11=Both.
Bits 21 through 20: SUPER (Supervisor/User AM code) 01=Non-privledged, 10=Supervisor, 11=Both.
Bits 17 through 16: VAS (VMEbus address space) 00=A16, 01=A24, 10=A32.

The address value of the location_monitor parameter sets the VMEbus address to which the registers will be mapped.

master_window[0-7]

The master_window parameters specify any of the 8 VMEbus Master Windows. The master_window parameter is specified as: master_window<number>=control,size,VMEbus address,[PCI address]; Substitute the window number for <number>.

The control value of the master_window parameter refers to the image control register. This register is specified below:

Register 7: Master Window Control

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
ENPWENReserved
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
VDWReservedVAS
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
ReservedPGMReservedSUPERReservedVCT
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
ReservedLAS
Bit 31: EN (Image enable) 0=Disabled, 1=Enabled.
Bit 30: PWEN (Posted write enable) 0=Disabled, 1=Enabled.
Bits 23 through 22: VDW (VMEbus maximum data width) 00=8 bits, 01=16 bits, 10=32 bits, 11=64 bits.
Bits 18 through 16: VAS (VMEbus address space) 000=A16, 001=A24, 010=A32.
Bit 14: PGM (Program/Data AM code) 0=Data, 1=Program.
Bit 12: SUPER (Supervisor/User AM code) 0=Non-privledged, 1=Supervisor.
Bit 8: VCT (VMEbus cycle type) 0=No VMEbus block transfers, 1=single VMEbus block transfers.
Bit 0: LAS (PCI bus memory space) 0=PCI bus memory space, 1=PCI bus I/O space.

The size value of the master_window parameter sets the minimum size of the window.

The VMEbus address value of the master_window parameter sets the VMEbus address of the window.

The PCI address value of the master_window parameter sets the local PCI address of the window and is an optional parameter. If this parameter is not specified, the driver selects a PCI address. If a PCI address is specified for this parameter, ensure that it does not conflict with any other devices in the system. See the /proc/iomem file to determine the addresses of other devices.

slave_window[0-7]

The slave_window parameters specify any of the 8 VMEbus Slave Windows. The slave_window parameter is specified as: slave_window<number>=control,size,VMEbus address,[PCI address]; Substitute the window number for <number>.

The control value of the slave_window parameter refers to the image control register. This register is specified below:

Register 8: Slave Window Control

Bit 31Bit 30Bit 29Bit 28Bit 27Bit 26Bit 25Bit 24
ENPWENPRENReserved
Bit 23Bit 22Bit 21Bit 20Bit 19Bit 18Bit 17Bit 16
PGMSUPERReservedVAS
Bit 15Bit 14Bit 13Bit 12Bit 11Bit 10Bit 9Bit 8
Reserved
Bit 7Bit 6Bit 5Bit 4Bit 3Bit 2Bit 1Bit 0
LD64ENLLRMWReservedLAS
Bit 31: EN (Image enable) 0=Disabled, 1=Enabled.
Bit 30: PWEN (Posted write enable) 0=Disabled, 1=Enabled.
Bit 29: PREN (Prefetch read enable) 0=Disabled, 1=Enabled.
Bits 23 through 22: PGM (Program/Data AM code) 01=Data, 10=Program, 11=Both.
Bits 21 through 20: SUPER (Supervisor/User AM code) 01=Non-privledged, 10=Supervisor, 11=Both.
Bits 17 through 16: VAS (VMEbus address space) 00=A16, 01=A24, 10=A32.
Bit 7: LD64EN (Enable 64-bit PCI bit transactions) 0=Disabled, 1=Enabled.
Bit 6: LLRMW (Enable PCI bus lock on VMEbus RMW) 0=Disabled, 1=Enabled.
Bits 1 through 0: LAS (PCI bus address space) 00=PCI bus memory space, 01=PCI bus I/O space, 10=PCI bus configuration space.

The size value of the slave_window parameter sets the minimum size of the window.

The VMEbus address value of the slave_window parameter sets the VMEbus address of the window.

The PCI address value of the slave_window parameter sets the local PCI address of the window and is an optional parameter. If this parameter is not specified, the driver will attempt to allocate RAM for the Slave Window.

How to Use Module Parameters

To use module parameters, add them to the /etc/modules.conf file like this:

Code listing 18: Using Module Parameters

#We'll specify the comm register to turn slave endian conversion off
#and a 0x10000 byte Slave Window in A24 space at VMEbus address 0x0.
options vme_universe comm=0x845 slave_window0=0xE0F10000,0x10000,0x0

Warning: Module parameters can be passed in using insmod, but insmod does not read parameters from /etc/modules.conf.

Module parameters are required only when unusual settings are to be specified. The driver's default values are usually sufficient. Most features of the Tundra Universe II device can be programmed using API functions.

Accessing the VMEbus Using a Kernel Module

All of the API functions we have already discussed can also be accessed by other kernel modules. In addition, the following functions can be used with kernel modules:

vme_interrupt_asserted
vme_interrupt_clear
vme_interrupt_disable
vme_interrupt_enable
vme_interrupt_irq
vme_interrupt_vector

Here is an example module that interfaces with the VMEbus using the vme_universe driver:

Code listing 19: vme_module.c



#include <linux/modversions.h>
#include <linux/config.h>
#ifdef CONFIG_SMP
#define __SMP__
#endif

#include <linux/module.h>
#include <asm/uaccess.h>
#include "vme/vme.h"
#include "vme/vme_api.h"


#define MODULE_NAME  "vme_example"
MODULE_AUTHOR("VMIC <www.vmic.com>");
MODULE_DESCRIPTION("VME module example");
#ifdef MODULE_LICENSE
MODULE_LICENSE("Dual BSD/GPL");
#endif

int vme_open(struct inode *inode, struct file *file_ptr);
int vme_close(struct inode *inode, struct file *file_ptr);
ssize_t vme_read(struct file *file_ptr, char *buffer, size_t size,
		 loff_t * off);
ssize_t vme_write(struct file *file_ptr, const char *buffer, size_t size,
		  loff_t * off);

static struct file_operations file_ops = {
	open:vme_open,
	release:vme_close,
	read:vme_read,
	write:vme_write,
};


static vme_bus_handle_t bus;
static vme_master_handle_t window;
static vme_interrupt_handle_t interrupt;
static char *ptr;
static int irq, major;


#define VME_ADDRESS          0x08000000
#define ADDRESS_MODIFIER     VME_A32SD
#define NBYTES               0x40
#define LEVEL                VME_INTERRUPT_VIRQ3
#define VECTOR               0x10


/*============================================================================
 * Hook to the open file operation
 */
int vme_open(struct inode *inode, struct file *file_ptr)
{
	MOD_INC_USE_COUNT;
	return 0;
}


/*============================================================================
 * Hook to the close file operation
 */
int vme_close(struct inode *inode, struct file *file_ptr)
{
	MOD_DEC_USE_COUNT;
	return 0;
}


/*============================================================================
 * Hook to the read file operation
 */
ssize_t vme_read(struct file * file_ptr, char *buffer, size_t size,
		 loff_t * off)
{
	int nbytes, bytes_rem;

	/* Calculate how many bytes we can read (all|some|none)?
	 */
	nbytes = (size > NBYTES - *off) ? NBYTES - *off : size;
	if (0 >= nbytes)
		return 0;

	/* Transfer the data
	 */
	bytes_rem = copy_to_user(buffer, ptr + *off, nbytes);
	if (bytes_rem)
		return -EFAULT;

	nbytes -= bytes_rem;

	/* Update the file position
	 */
	*off += nbytes;

	/* Return the actual number of bytes transferred
	 */
	return nbytes;
}


/*============================================================================
 * Hook to the write file operation
 */
ssize_t vme_write(struct file * file_ptr, const char *buffer, size_t size,
		  loff_t * off)
{
	int nbytes, bytes_rem;

	/* Calculate how many bytes we can read (all|some|none)?
	 */
	nbytes = (size > NBYTES - *off) ? NBYTES - *off : size;
	if (0 >= nbytes)
		return 0;

	/* Transfer the data
	 */
	bytes_rem = copy_from_user(ptr + *off, buffer, nbytes);
	if (bytes_rem)
		return -EFAULT;

	nbytes -= bytes_rem;

	/* Update the file position
	 */
	*off += nbytes;

	/* Return the actual number of bytes transferred
	 */
	return nbytes;
}


/*============================================================================
 * Interrupt service routine
 */
void vme_isr(int irq, void *dev_id, struct pt_regs *regs)
{
	int vector;

	/* Is the interrupt on our level? If vme_interrupt_asserted returns
	   true, then the interrupt was on the level that we reserved and we
	   should do something clever.
	 */
	if (vme_interrupt_asserted(bus, interrupt)) {
		vme_interrupt_vector(bus, interrupt, &vector);
		printk(KERN_NOTICE MODULE_NAME ":Interrupt vector 0x%x\n",
		       vector);

		/* Clear the interrupt
		 */
		vme_interrupt_clear(bus, interrupt);
	}
}


/*===========================================================================
 * Driver module initialization
 */
int init_module(void)
{
	if (vme_init(&bus)) {
		printk(KERN_ERR MODULE_NAME ": Failed vme_init\n");
		return -1;
	}

	if (vme_master_window_create(bus, &window, VME_ADDRESS,
				     ADDRESS_MODIFIER, NBYTES, VME_CTL_PWEN,
				     NULL)) {
		printk(KERN_ERR MODULE_NAME
		       ":Failed vme_master_window_create\n");
		goto error_create;
	}

	ptr = vme_master_window_map(bus, window, 0);
	if (NULL == ptr) {
		printk(KERN_ERR MODULE_NAME ":Failed vme_master_window_map\n");
		goto error_map;
	}

	/* Get the IRQ of the VMEbus bridge device
	 */
	if (vme_interrupt_irq(bus, &irq)) {
		printk(KERN_ERR MODULE_NAME ":Failed vme_interrupt_irq\n");
		goto error_irq;
	}

	if (request_irq(irq, vme_isr, SA_SHIRQ, MODULE_NAME, &file_ops)) {
		printk(KERN_ERR MODULE_NAME ":Failed reqest_irq\n");
		goto error_irq;
	}

	/* The VME_INTERRUPT_RESERVE flags tells the VMEbus bridge driver to not
	   do anything with interrupts on the level we specify here (not just
	   level/vector, but the entire level). Therefore, be sure to always do
	   the request_irq before vme_interrupt_attach, otherwise you will have
	   a race condition where noone handles the interrupt, and your system
	   could lock up.
	 */
	if (vme_interrupt_attach(bus, &interrupt, LEVEL, VECTOR,
				 VME_INTERRUPT_RESERVE, NULL)) {
		printk(KERN_ERR MODULE_NAME ":Failed vme_interrupt_attach\n");
		goto error_interrupt_attach;
	}

	/* We let the kernel pick a character device major number for us.
	 */
	major = register_chrdev(0, MODULE_NAME, &file_ops);
	if ((-EBUSY == major) || (-EINVAL == major)) {
		printk(KERN_ERR MODULE_NAME ":Failed register_chrdev\n");
		goto error_register;
	}

	return 0;

      error_register:
	vme_interrupt_release(bus, interrupt);

      error_interrupt_attach:
	free_irq(irq, &file_ops);

      error_irq:
	vme_master_window_unmap(bus, window);

      error_map:
	vme_master_window_release(bus, window);

      error_create:
	vme_term(bus);

	return -1;
}


/*============================================================================
 * Driver module exit routine
 */
void cleanup_module(void)
{
	unregister_chrdev(major, MODULE_NAME);
	vme_interrupt_release(bus, interrupt);
	free_irq(irq, &file_ops);
	vme_master_window_unmap(bus, window);
	vme_master_window_release(bus, window);
	vme_term(bus);
}


And here is the Makefile to compile the module:

Code listing 20: Makefile



KERNELREV:=$(shell uname -r)
KERNELSRC:=/lib/modules/$(KERNELREV)/build

MODNAME:=vme_example.o
MODDIR:=/lib/modules/$(KERNELREV)/kernel/drivers/vme

CFLAGS = -D__KERNEL__ -DMODULE -I$(KERNELSRC)/include -O -Wall

all: $(MODNAME)

install: all
	install -m0644 -oroot -groot -D $(MODNAME) $(MODDIR)/$(MODNAME)
	/sbin/depmod

uninstall:
	rm -f $(MODDIR)/$(MODNAME)

$(MODNAME): vme_module.o
	$(LD) -r $^ -o $@

clean:
	rm -f *.o *~


This module implements read and write interfaces to the VMIVME-5588DMA board used in an earlier example, and also handles VMEbus interrupts on level 3.

Note that the calls used to create and map a VMEbus Master Window are identical to the calls used in the earlier peek example. The pointer returned by vme_master_window_map is then used to access the VMEbus in the read and write functions.

The vme_interrupt_attach function is called with the flag VME_INTERRUPT_RESERVE, which informs the vme_universe driver that another module will handle interrupts on the given VMEbus level. This is accomplished by attaching an interrupt handler to the same IRQ as the vme_universe driver (i.e. a shared PCI interrupt). Any time an interrupt occurs on the VMEbus, both the ISR for the vme_universe driver and the ISR for our module will be executed. The vme_interrupt_asserted function must then be called to determine if the interrupt occured on the VMEbus level that we reserved. vme_interrupt_asserted returns a non-zero value if the interrupt is on the reserved level, or 0 if it is not on our reserved level.

Note the comment in the ISR that we should do something clever here. Obviously a print statement is not clever. Try replacing it with something more interesting!

Important: Although a vector has been passed into the vme_interrupt_attach function, when VME_INTERRUPT_RESERVE is used, that entire VMEbus interrupt level is reserved regardless of the vector requested. Your code should be prepared to deal with unexpected interrupt vectors.

So let's do something with this module.

Code listing 21: Using the Kernel Module

sh# make install
sh# modprobe vme_example
sh$ lsmod
Module                  Size  Used by    Not tainted
vme_example             2784   0 (unused)
vme_universe           69248   1 [vme_example]

// Notice that our module vme_example is loaded.
// Also notice that it has been registered as a user of the vme_universe driver.
sh$ cat /proc/interrupts
 CPU0       
  0:    1240357          XT-PIC  timer
  1:       2543          XT-PIC  keyboard
  2:          0          XT-PIC  cascade
  8:          1          XT-PIC  rtc
 10:          0          XT-PIC  vme_universe, vme_example
 11:         74          XT-PIC  DC21143 (eth0)
 12:         20          XT-PIC  PS/2 Mouse
 14:      13529          XT-PIC  ide0
 15:         97          XT-PIC  ide1
NMI:          0 
LOC:          0 
ERR:          0
MIS:          0

// Notice here that the drivers vme_example and vme_universe are sharing an IRQ.
// That is what we want.

// Because we let the kernel choose the major number for our driver, we need to
// find out what that number is and create a file in the /dev/ directory for it.
sh$ grep vme /proc/devices
221 vme_universe
254 vme_example

sh# mknod --mode=666 /dev/vme_example c 254 0
sh$ ls -l /dev/vme_example
crw-rw-rw-    1 root     root     254,   0 Jul 13 00:17 /dev/vme_example

// Now we should be able to dump the VMEbus registers mapped by our module
// I'll use the command 'od' (octal dump) to do it
sh$ od -Ax -t x1 /dev/vme_example
000000 ff 70 4b ff 00 ff e0 01 ff ff ff ff ff ff ff ff
000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000020 00 00 00 00 00 ff ff ff 00 ff ff ff 00 ff ff ff
000030 0f ff ff ff 0f ff ff ff 0f ff ff ff 0f ff ff ff
000040

// Hmmm! There's that byte swapping thing again!

The data is swapped because the copy_to_user function in vme_read moves the data as longwords, but we printed the data as bytes. This is the same problem we saw in the DMA section.

Note: This section is not intended to be a tutorial on writing Linux kernel modules. I suggest Linux Device Drivers, 2nd edition by Alessandro Rubini and Jonathan Corbet as a good place to start learning how to write Linux kernel modules.

Reserving RAM at Boot Time

As noted in previous sections, it is sometimes necessary to reserve RAM at boot time for slave RAM or DMA buffers.

For example, to reserve 2MBytes of a total 64MBytes of RAM while using LILO, add the line append="mem=62M" in /etc/lilo.conf file and run the lilo command.

Code listing 22: Reserving RAM at Boot Time Using LILO

image=/boot/vmlinuz-2.4.18-3smp
      label=Linux
      root=/dev/hda1
      initrd=/boot/initrd-2.4.18-3.img
      append="mem=62M"

Important: If LILO is not run, the changes will not take effect.

If you are using GRUB, add mem=62M to the kernel boot line in the /boot.grub/menu.lst file, as illustrated below.

Code listing 23: Reserving RAM at Boot Time Using GRUB

title Linux (2.4.18-3smp)
	root (hd0,0)
	kernel /boot/vmlinuz-2.4.18-3smp ro root=/dev/hda1 mem=62M
	initrd /boot/initrd-2.4.18-3.img

In older kernel versions, specifying mem=62M ensured that the base address of the reserved memory would be located at memory address 0x3e00000 (1024 * 1024 * 62). This cannot be assumed with newer kernel versions. See /proc/iomem to determine where the top of the kernel's physical RAM is located.

Here is the resulting /proc/iomem output on my machine. The details will vary from system to system.

Code listing 24: /proc/iomem

00000000-0009fbff : System RAM
0009fc00-0009ffff : reserved
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000f0000-000fffff : System ROM
00100000-03e603ff : System RAM
  00100000-002342b0 : Kernel code
  002342b1-00306b3f : Kernel data
e0000000-e7ffffff : PCI Bus #01
  e0000000-e3ffffff : S3 Inc. Trio 64 3D
e8000000-ebffffff : Intel Corp. 440BX/ZX - 82443BX/ZX Host bridge
ed000000-ed0003ff : Digital Equipment Corporation DECchip 21142/43
ed001000-ed001fff : Tundra Semiconductor Corp. CA91C042 [Universe]

Since the last entry for System RAM is at 0x3e603ff, my reserved memory starts at 0x3e60400. To be safe, it is often better to use page aligned memory 4kb on x86), so I would use 0x3e61000 as my base address, meaning that I do not have the full 2MB. It might be smart to reserve a little extra memory when you do this.

There is one more thing to worry about. The vme_universe driver inquires with the kernel to determine which memory regions can be used for the placement of VMEbus windows. Since the kernel is not aware of the reserved memory, it may allow the vme_universe driver to create a VMEbus window that conflicts with the specified reserved RAM. Thus, the vme_universe driver needs to know about the reserved RAM. This can be done by passing the pci_lo_bound module parameter to the vme_universe driver. Since the machine in our example has 64MB of RAM we need to call the vme_universe driver with the parameter pci_lo_bound=0x04000000. This is how we would add the parameter to the /etc/modules.conf file:

Code listing 25: /etc/modules.conf

options vme_universe pci_lo_bound=0x04000000

Debugging Tips

Proc Pages

The following files contain useful information for debugging VMEbus accesses.

/proc/iomem - Displays the memory allocation for your system. When Master Windows are created, an entry is made for them in this file. For example:

08000000-080fffff : VMEbus master window 0

/proc/interrupts - Displays the assignment of drivers to IRQ's, as well as a count of how many interrupts have ocurred on a particular IRQ. When the vme_universe driver is loaded, an entry such as the following will be displayed:

10: 0 XT-PIC vme_universe

/proc/vme/ctl, /proc/vme/interrupt, /proc/vme/master and /proc/vme/slave - These proc pages are registered by the vme_universe driver and can be used to retrieve information about the status of the driver.

Utilities

The utility commands described previously make excellent debug tools. Use them to access a board on the VMEbus, to test if interrupts are being asserted, or to assert them, and to test endian conversion hardware.

Debugging Bus Errors

A common case in which these utilities can be useful is testing for bus errors.

If the data read from the bus is all "F's", a bus error may exist. Use the vme_catch_interrupt utility to determine if this is the case:

Code listing 26: Checking for Bus Errors

// First start the interrupt handler utility to watch for bus errors
// Be sure to put it in the background
sh$ vme_catch_interrupt -lVME_INTERRUPT_BERR &
// Then we are going to try to read from the VMEbus using vme_peek
sh$ vme_peek -aVME_A24SD -A0x8000
BERR: address = 0x8000
ff

In this example, a bus error occurred at VMEbus address 0x8000. The most likely cause is that no board is addressed at 0x8000 with address modifier A24SD. Recheck the addressing of the target board and ensure the correct address is being used.

Compiling with Debug

To compile the vme_universe driver with debug functionality, add -DDEBUG to CFLAGS in the Makefile. When the driver is installed, it will print messages to the /var/log/messages file. The contents of this file can be viewed with the dmesg command.

Changes Since Version 2.x

Major changes have been made in the functionality of the VMEbus driver for Linux. The following is a summary of the most important changes:

  • Dynamic window allocation - Windows can now be allocated using API calls. Previously, the only way to allocate windows was to set them up in the /etc/modules.conf file. Additionally, the physical address no longer needs to be specified by the user; the driver can now choose it (which is the recommended method). However, if you prefer, you can still choose the physical address and configure the windows in the /etc/modules.conf file.
  • Inter-module interface - The VMEbus can now be accessed through the driver from other kernel modules.
  • SMP support.
  • A completely new API to support some of the new features.

Interfacing with Legacy Code

The 2.x API is still supported through a legacy library. Existing applications written with earlier versions will work without modification (famous last words!), but must be recompiled before they can be used.

Installing the Legacy Library

To install the legacy library, switch to the legacy directory of the vme_universe project and run make install.

Compiling Applications

To compile applications using the VME legacy library, link with the libraries liblegacy_vme.a and libvme.so by including the compiler switches -llegacy_vme -lvme.

You will receive warnings that the header files in the directory /usr/include/vmic are deprecated. These warnings are normal and can be ignored.

License

All of the source code supplied with the vme_universe project is covered under the BSD license.



Copyright 2004 VMIC. Questions, Comments, Corrections? Email support.embeddedsystems@gefanuc.com.