Installing a 2-Node NCS Cluster With an Inexpensive SCSI SAN

Articles and Tips: article

Chris Greyson
Instructional Designer
Novell Education

01 Feb 2002

NetWare 6 includes a 2-node Novell Clustering Services (NCS) license. If you're not currently clustering your NetWare servers, this license gives you a great opportunity to configure clustering in a lab for experimenting with and demonstrating clustering features and benefits.

A key component of a clustering solution is the Storage Area Network (SAN). The SAN provides the hard disk space necessary to run the cluster and store shared resources for applications and services.

However, because most companies are facing tight budgets, investing $5,000 to $10,000 in a "SCSI in a box" SAN cluster solution or contracting with a hardware vendor for a SAN Fibre Channel system (which can run $50,000 or more) to road test a 2-node cluster is just not possible.

While developing the new Upgrading to NetWare 6 course (Course #3000), the instructional developers recognized this challenge and provided training for network administrators that focuses on implementing a non-production 2-node NCS cluster on a SCSI hardware configuration for under $500.

The following are general instructions for installing and testing this configuration, along with troubleshooting tips and advice gathered from our own lab and classroom experiences, and from several Novell clustering experts.

You should already have NetWare 6 installed on both servers before starting any of the following procedures.

Hardware Requirements and Tips

The following lists the hardware required for your $500 SAN configuration, along with some hardware recommendations:

SCSI Adapter Cards.

You will need two Adaptec 2940 PCI SCSI adapter cards (one for each cluster server). All our development and testing efforts in Education involved exclusive use of the Adaptec 2940 card and the driver provided with NetWare 6.

You can try using another type of SCSI adapter card (no guarantees), but you need to make sure the card and NetWare .HAM driver are capable of being multi-initiated. This means that the two SCSI adapter cards can exist on the same SCSI bus, sharing the same storage device.

To find out if the SCSI card and driver are capable of multi-initiation, check the card and .HAM driver specifications. This information is normally listed on the manufacturer's Web site. If not, call the manufacturer for the information.

SCSI Hard Drive.

You will also need an external SCSI hard drive in an external SCSI enclosure that has two external SCSI connectors. The size of the hard drive is not critical (you can purchase a used 2 gigabyte hard drive for under $50).

Note: Make sure that termination on the hard drive is disabled, or data will not flow across the entire length of the bus, and clustering will not work properly.

A SCSI bus is all the wires and cables that transmit the SCSI data. In this case, the SCSI bus includes the two SCSI cables and any wires associated with the adapter cards, the hard drive, and the enclosure.

Because the SCSI adapter cards are at the ends of the 2-node bus, you also need to make sure that the adapter cards are terminated. If you have another SCSI device attached to the adapter card (such as a CD-ROM drive), you will need to terminate that device instead of the adapter card.

SCSI Cables.

Finally, you will need two SCSI cables to attach the adapter cards to the SCSI enclosure. In our experience, the SCSI cables often proved to be the weakest link in our SAN configuration. We highly recommend using quality SCSI cables with the same specifications (especially the same impedance). Whenever possible, use the same type and length from the same manufacturer.

Install the SAN Hardware

Once you have the proper hardware, you are ready to install and connect the SAN to your two NetWare 6 servers by performing the following steps:

Turn off the two servers, and seat the Adaptec 2940 adapter cards in expansion slots.

Make sure you seat the cards properly, and that you use a screw to lock down the cards in position. Because the SCSI cables are heavy duty, they tend to unseat the SCSI adapter cards when you are attaching the cables. (This problem caused us several hours of troubleshooting in the classroom and lab before we discovered what was happening.)
Plug in the SCSI hard drive and enclosure to a power source. Make sure the hard drive is turned off.
Connect the SCSI cables to the hard drive enclosure and the SCSI adapter cards. Make sure the cables are firmly attached with clips or screws, or you might experience problems when testing the SAN.
Check the SCSI hard drive and SCSI adapter card settings by using the Adaptec SCSI Utility (Ctrl-A). To do this, perform the following steps.
- Turn on the SCSI hard drive; then turn on the first server.
- When the server detects the Adaptec 2940 adapter card and displays the card name on the screen, press Ctrl+A immediately to start the Adaptec SCSISelect utility.

You should see two options available: Configure/View Host Adapter Settings and SCSI Utilities (the wording may vary with the version of SCSISelect).

Select the SCSI Utilities option.

After a few moments, SCSISelect displays a list of SCSI ID numbers. You should see an ID number for the SCSI adapter card and an ID number for the SCSI hard drive. SCSI ID numbers are assigned to SCSI devices and adapter cards by the manufacturer.

The higher the ID number, the higher the priority of the SCSI device or card. SCSI hard drives are normally assigned an ID number of 0, giving them the highest priority. SCSI cards are given a low priority number such as 6 or 7.

Note: You need to make sure each SCSI hard drive and adapter card have a different ID number. Also, make sure the SCSI hard drive has a higher priority number than the SCSI adapter cards.

When configuring a 2-node SCSI cluster for NCS, consider using ID 7 for one card and ID 6 for the other card. Even if they are available, avoid using higher SCSI ID numbers (8-15).

Also make sure you set the ID number for the SCSI hard drive to 2 or 3. Leaving the ID set at 0 can cause the hard drive to be used as the bootable drive for your server.

For example, we connected a used SCSI hard drive to one of the servers in our lab. When booting the server, it launched NetWare 5.1 instead of NetWare 6. We later discovered that NetWare 5.1 was installed on the SCSI hard drive, and that the server was booting from the hard drive because the ID was set to 0.

From this information, make any necessary changes to the SCSI ID numbers. You can use SCSISelect to set the SCSI ID number for the adapter card. However, for external SCSI devices such as a hard drive, the SCSI ID usually is set with a switch on the back of the device.

If the enclosure and the hard drive were not pre-assembled by the manufacturer, you might need to set the number by configuring a jumper on the hard drive.

Check the configuration settings for the SCSI adapter card, and make sure that the SCSI hard is not configured as the boot drive.
Record the SCSI ID numbers for the adapter card and hard drive, and then save any changes and exit the SCSISelect utility. You should see a message requesting you to reboot the server.
Leave the first server in that state, and then turn on the second server.
Use the SCSISelect utility with the second server to check the SCSI ID number of the SCSI adapter card. If the ID number of the adapter card is the same as the card number in the first server, change the ID number of the adapter card in the second server.
When you finish, save any changes and exit the SCSISelect utility. Both servers should now display a reboot message.

Check the Drivers for the SCSI Card and Hard Drive

Now that the SCSI hard drive is connected to both servers and running, you can configure the SAN for NCS. To do this, perform the following steps:

Press any key on both servers to reboot the servers.

As each server starts (and before NetWare 6 loads), you should see the name of the SCSI adapter card and the name of the SCSI hard drive displayed on the server screen.
After NetWare 6 loads (and the GUI interface is displayed), make sure a driver for the SCSI adapter card and a driver for the SCSI hard drive are installed on the servers by doing the following on each server:
- Start the NetWare Configuration utility by entering the NWCONFIG utility at the server console prompt. In the Configuration Options dialog, select Driver Options. In the Driver Options dialog, select the Configure Disk And Storage Device Drivers option.
- Press Tab and scroll through the list of selected disk drivers. Choose from the following:

If you see SCSIHD listed for the hard drive and a driver listed for your SCSI card, exit the NWCONFIG utility by pressing Esc until you see an Exit the Install message; then select Yes and start at Step 3.

If you do not see SCSIHD or a driver listed for your SCSI card, press the Tab key and select the Select an additional driver option. Select the driver from the list; select Yes or No (depending on your need to select another driver); then press Tab and scroll through the disk drivers list. You should see SCSIHD and the driver for your SCSI card listed with a "currently loaded" status.

Exit the NWCONFIG utility by pressing the Esc key until an Exit the Install message is displayed; then select Yes.
At the console prompt for each server, enter LIST DEVICES <Enter> (you may need to enter the command SCAN FOR NEW DEVICES first).

After a few moments, you see the SCSI hard drive listed with a device ID number (such as 0x0005 or 0x000C) and a name (such as FUJITSU MAE3091LP).

The device ID number might be different on each server, but the hard drive name should be the same.

Note: If you see "Unbound Device" instead of the SCSI hard drive name, the .HAM driver for the adapter card has been loaded, but the .CDM driver for the SCSI hard drive has not been loaded.

Note: To correct this situation, use the File Browser utility in the graphical console to copy the SCSIHD.CDM and SCSIHD.DDI files from C:\NWSERVER\DRIVERS to C:\NWSERVER, load the .HAM and .CDM drivers, enter SCAN FOR NEW DEVICES, and then enter LIST DEVICES again at the console prompt.

Note: The SCSI hard drive name is listed instead of "Unbound Device."

Record the name of the SCSI hard drive and the device ID number for each server.

Initialize the SCSI Hard Drive

If you are using a SCSI hard drive that contains partitions and data, you need to back up the data and initialize the drive to remove the partitions in preparation for installing NCS.

SCSI hardware manufacturers (such as Adaptec) indicate that you must perform a low-level format if the drive was previously connected to a different SCSI card. You can perform a low-level format of the hard drive by using a SCSI utility.

You can perform a high-level format of the hard drive using ConsoleOne (or DOS), but this only clears the partition table; it does not remove partitions. A high-level format may cause problems later, such as a server abending when you create an NSS volume.

In experimenting with several types of used SCSI hard drives, we have found very few instances that required a low-level format. If you want to use ConsoleOne to perform a high-level format, do the following:

Start the ConsoleOne utility from one of the servers, and log in as Administrator to the tree where the server objects are located.
Select the server object; then select Properties.
List the devices connected to the server by selecting the Media > Devices option.
Select the device ID number of your SCSI hard drive. The name of the SCSI hard drive is displayed.
Select Initialize Hard Disk. A message indicates that all partitions will be deleted.
Initialize the SCSI hard drive and delete all partitions by selecting Yes.

(Conditional) If you see a warning indicating a problem with removing NSS pools or traditional volumes, select OK.
When the process is complete, check settings in the Media > Devices list for the SCSI hard drive device.

The capacity, free space, and unpartitioned space are all equal and include most space on the hard drive. In addition, there should be no partitions listed in the Partitions drop-down list.
Close the Properties dialog by selecting Cancel.
Close ConsoleOne by selecting File > Exit.

With the SCSI hard drive connected to servers and initialized, and the SCSI drivers installed on both servers, your SAN is ready for an NCS installation.

Installing NCS with a SAN

Instructions for installing NCS 1.6 with NetWare 6 are available at: http://www.novell.com/documentation/lg/ncs6p/index.html

The following are some guidelines to help you during installation of NCS 1.6:

NCS installation gives you a choice of creating a new cluster, upgrading a cluster, or adding a node (server) to the cluster.

Because you are setting up the cluster for the first time, select Create New Cluster, and make sure that you select the Skip The File Copy option.

When you install NetWare 6, all the NLMs for clustering are automatically installed to the server. When you install NCS 1.6, you can skip copying these files.

About halfway through NCS installation, you select each NetWare server you want to add to the cluster. The installation program accesses each server and then adds the server name and IP address to a list.

When you press Next, watch the server screens. You should see several cluster NLMs loaded on the servers. At this point, Alt+Esc to bring up the Logger screen on each server to check for any error messages (you can also press Crtl+Esc to bring up the Available Screens menu).

After entering an IP address for the cluster object, you should see an NCS Shared Media Selection screen. If you do not, then the NCS installation program did not find the shared SCSI hard drive. At this point, you need to cancel the installation, and retrace your steps.

Check all cabling and connections to make sure everything is secure. Check for duplicate SCSI ID numbers, and make sure you can see the SCSI hard drive listed when you enter the List Devices command at the server console prompt.

Post-Installation SCSI BIOS Settings

If you are experiencing problems with SAN after the NCS installation, try using the SCSISelect utility to reconfigure the following BIOS settings of your SCSI adapter card.

BIOS Setting	Suggested Configuration
Maximum Sync Transfer Rate	Set both SCSI adapter cards to a common transfer speed. If one card is transferring data at a faster rate than the other, you might experience problems when running the cluster.
Advanced "Speed Up" Options	Disable advanced options designed to increase the speed of transferring data. These include Wide, Ultra, and Send Start Unit Command.
Extended BIOS Translation for DOS Drive	Turn off (or disable) drive translation. This function can be fatal to NetWare, and can possibly cause a server ab-end.

For more information on configuring SCSI devices, see what is posted at http://www.paralan.com.

This information is taken from the "Upgrading to NetWare 6" manual, Course number #3000. See http://novell.com.education to find out about enrolling in this course.

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.