Mylex Disk Array Controllers (aka RAID Controllers): Use in SuSE Linux

Knowledgebase

Request:

You want to operate a hardware RAID (Redundant Array of Independent Disks) using a Mylex controller of the DAC960 family.

Status of DAC960 Support

Author Leonard N. Zubkoff declared the actual (April 1999) release of his DAC960 driver to be the first "production release". Obtain the most recent status of this driver from his Linux homepage http://www.dandelion.com/Linux/.

The DAC 960 driver is not yet contained in the "official" Linux kernel 2.2.5. However, SuSE included it into the kernel sources shipping with SuSE Linux (package lx_suse) from version 6.1 on. It is available also in the installation system used for installing SuSE Linux.

The authoritative reference documentation is the file /usr/src/linux/Documentation/README.DAC960 (Note: in the recent SuSE Linux kernel sources this file is shifted into the directory /usr/src/linux/Documentation).

This SDB article is based on this README. Its purpose is

to inform about thesupported Hardware,
to explain the new device files of the DAC960 driver,
to sketch the procedure of installing SuSE Linux onto a DAC960 RAID,
to point out some highlights of the DAC960 operation under Linux.

Supported Hardware

=========     Start Quote README.DAC960   =============

eXtremeRAID 1100 (DAC1164P)
            3 Wide Ultra-2/LVD SCSI channels
            233MHz StrongARM SA 110 Processor
            64 Bit PCI (backward compatible with 
                        32 Bit PCI slots)
            16MB/32MB/64MB Parity SDRAM Memory 
            with Battery Backup

AcceleRAID 250 (DAC960PTL1)
            Uses onboard Symbios SCSI chips on 
            certain motherboards.
            Also includes one onboard Wide Ultra-2/LVD 
            SCSI Channel
            66MHz Intel i960RD RISC Processor
            4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory

AcceleRAID 200 (DAC960PTL0)
            Uses onboard Symbios SCSI chips on 
            certain motherboards
            Includes no onboard SCSI Channels
            66MHz Intel i960RD RISC Processor
            4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory

AcceleRAID 150 (DAC960PRL)
            Uses onboard Symbios SCSI chips on 
            certain motherboards
            Also includes one onboard Wide Ultra-2/LVD 
            SCSI Channel
            33MHz Intel i960RP RISC Processor
            4MB Parity EDO Memory

DAC960PJ    1/2/3 Wide Ultra SCSI-3 Channels
            66MHz Intel i960RD RISC Processor
            4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory

DAC960PG    1/2/3 Wide Ultra SCSI-3 Channels
            33MHz Intel i960RP RISC Processor
            4MB/8MB ECC EDO Memory

DAC960PU    1/2/3 Wide Ultra SCSI-3 Channels
            Intel i960CF RISC Processor
            4MB/8MB EDRAM or 2MB/4MB/8MB/16MB/32MB 
            DRAM Memory

DAC960PD    1/2/3 Wide Fast SCSI-2 Channels
            Intel i960CF RISC Processor
            4MB/8MB EDRAM or 2MB/4MB/8MB/16MB/32MB 
            DRAM Memory

DAC960PL    1/2/3 Wide Fast SCSI-2 Channels
            Intel i960 RISC Processor
            2MB/4MB/8MB/16MB/32MB DRAM Memory

For the eXtremeRAID 1100, firmware version 
5.06-0-52 or above is required.

For the AcceleRAID 250, 200, and 150, firmware 
version 4.06-0-57 or above is required.

For the DAC960PJ and DAC960PG, firmware version 
4.06-0-00 or above is required.

For the DAC960PU, DAC960PD, and DAC960PL, 
firmware version 3.51-0-04 or above is required.

[ ... omissions  ... ]

Please note that not all SCSI disk drives are 
suitable for use with DAC960 controllers 

[ ... omissions  ... ]

Mylex makes available a hard disk compatibility 
list by FTP at ftp://ftp.mylex.com/pub/dac960/diskcomp.html

========     End Quote README.DAC960   =============

DAC960 RAID Devices in Linux

The DAC960 hides the actual physical setup of a RAID (hard disks involved, RAID operation mode etc.) from the operating system. Instead, the controller presents so-called logical drives (Mylex term: system drives) to the operating system. These logical drives can be used just like the familiar "physical" drives (creating of partitions, file systems and swapspace, mounting etc.)

The device files for DAC 960 system drives reside in the separate directory /dev/rd and have "descriptive" names. Examples:

/dev/rd/c0d1 Controller 0, system drive 1
/dev/rd/c1d0 Controller 1, system drive 0 (if a second DAC960 is present in the system).
/dev/rd/c0d0p3 Controller 0, system drive 0, partition 3.

Note that the count of controllers und system drives starts with 0.

Attention! Each system drive may be divided into up to seven (7) partitions. Take this somewhat low limit into account at an early stage when designing the RAID and the partition layout!

If more partitions are needed (say, for a big application), you need to create correspondingly more system drives. (Note: fdisk won't keep you from creating further partitions on one system drive, like c0d0p8. But there is no device file which could provide access to such a partition. Result: the disk space occupied by such partitions remains unusable).

Device numbers. Device files are available for 8 DAC960 controllers, 32 system drives per controller and 7 partitions per system drive. The major numbers characterize the controller: 48 for controller 0 up to 55 for controller 7. The minor numbers describe the system drive and partition: a section of eight minor numbers referring to each single system drive. The following ls -l output shows typical examples:

                   maj  min

brw-------   ...   48,   0 Apr 13 08:49 c0d0
brw-------   ...   48,   1 Apr 13 08:49 c0d0p1
...
brw-------   ...   48,   7 Apr 13 08:49 c0d0p7
brw-------   ...   48,   8 Apr 13 08:49 c0d1
brw-------   ...   48,   9 Apr 13 08:49 c0d1p1
...
brw-------   ...   48,  16 Apr 13 08:49 c0d2
brw-------   ...   48,  17 Apr 13 08:49 c0d2p1
...
brw-------   ...   49,  16 Apr 13 08:49 c1d2
brw-------   ...   49,  17 Apr 13 08:49 c1d2p1

Installation of SuSE Linux (6.1 and newer) onto a DAC960 RAID Array

Preparation: Configure the RAID and the logical disks (system drives) . You cannot yet do this from within Linux: it is necessary instead to use the DACCF (Disk Array controller Config Facility) shipped with the DAC960.
Begin the installation of SuSE Linux as usual by booting from the installation floppy or the first CD and proceed until the main menu of linuxrc.
Load the driver module for the DAC960. It is available in the menu Kernel Modules/SCSI Modules like the other modules for SCSI hostadapters.

After loading the module, immediately check the protocol window on whether the controller(s) and system drive(s) present have been detected. If they are, the system drive(s) are available from now on by the names
```
 /dev/rd/c0d0, /dev/rd/c0d1 usw. 
```
as described above.
There are no further particularities during the rest of the installation. The system drives can be used like ordinary hard disks for partitioning, creating and mounting filesystems/ swap space.
LILO will work also when installed on a system drive, provided the system drive is bootable at all. To achieve this, adjust the settings in your system BIOS and the DAC960 BIOS (accessible by pressing Alt-M during system startup just after the first DAC960 messages are displayed).

If the boot configuration is correct, Linux is able to run from a DAC960 RAID alone without the presence of further (auxiliary) hard disks.

DAC960 RAID Operation under Linux: Some Highlights

User Interface of the driver

The DAC960 driver logs to the syslog. In SuSE Linux the default location for these logs are /var/log/messages and the Alt-F10 system console /dev/tty10.

NEW: the user interface located in /proc/rd:

/proc/rd/status. Contains a short (self-descripting ;-) status info consisting of one of the texts OK or ALERT.
/proc/rd/c? One subdirectory for each controller, containing two detailed status info files and one "command file":
- ```
/proc/rd/c?/current_status
```
- ```
/proc/rd/c?/initial_status
```
- ```
/proc/rd/c?/user_command
```
Details can be found in the file README.DAC960 .

Management of RAIDs and Error Recovery on the Running System

One of the crucial objectives when deploying a RAID is to avoid system downtime caused by a hard disk failure. Correspondingly, it is important to be able to perform the procedures needed to replace a defective hard disk without being forced to shut the system down.

Of course, for this being possible at all, it is necessary that

the defective disk was redundant, i.e.,
- was part of a RAID which actually did operate in a redundant mode (e.g. RAID 1, RAID 4, RAID 5),
- or had been configured as a standby disk (Mylex Term: Hot Spare) and did not contain live data at all.
and the hardware itself is designed to permit changing of hard disks on the running system (Hot Swap).

The DAC960 controller and L. N. Zubkoff's driver allow the following operations on a RAID without shutting the system down and booting into the maintenance utility DACCF :

Automatic "killing" of a defective hard disk (i.e., taking it offline) (DAC960).
Actual replacement of hard disks (Hot Swap), provided the disk itself is hot-swappable (DAC960).
Inclusion of a Hot Spare disk into a RAID operating in redundant mode and rebuild, as automatic reaction to a disk failure in this RAID (DAC960).
Manual killing of a hard disk, e.g. preparing for a disk exchange or removal (driver command).
Manual driver command to make a disk replacing a killed one take over the latter's former function: namely
- Hot spare state, or
- inclusion into a RAID operating in redundant mode. This means in particular rebuilding the disk.
Manual rebuild of a single drive belonging to a RAID operating in redundant mode (driver command).
Manual consistency check of a system drive (driver command).

If a rebuild or a check is in progress, messages about the progress are logged each minute to the syslog. The actual state is seen also in /proc/rd/c?/current_status.

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.