NetWare 4.0 Performance Tuning and Optimization: Part 2
Articles and Tips: article
Senior Consultant
Systems Research
01 Jun 1993
This AppNote discusses the use of the NetWare 4.0 auto-tuning capabilities and related setable parameters to enhance the performance of NetWare servers. Part 1 outlined the performance enhancements included in NetWare 4.0. Part 3 will describe the NetWare 4.0 setable parameters related to the cache, the LAN channel, and the disk channel.
Other AppNotes in This Series May 93 "NetWare 4.0 Performance Tuning and Optimization: Part 1" Oct 93 "NetWare 4.x Performance Tuning and Optimization: Part 3" Mar 93 "Using Production Workload Characteristics to Validate LAN Performance Evaluation Studies" May 92 "Identifying Test Workloads in LAN Performance Evaluation Studies"
Tuning the NetWare 4.0 Operating System
Performance tuning of NetWare networks is exciting, yet risky. If you're able to make noticeable improvements, you're a hero for a day. But if nothing you do helps - or worse, if anything you do hurts - you're in the doghouse.
Tuning is difficult because minor modifications (whether positive or negative) are not always perceptible. And major modifications shouldn't be made without thorough testing in a lab setting or pilot system. In fact, true tuning can only be performed in a controlled environment - usually a lab. This is due to the highly random and variable nature of production workloads. An incorrect parameter setting can look good for hours or days after you make the change, but it will actually hurt performance in the long run. Good settings can also be made to look bad if an unusually heavy production workload is placed on the network immediately following your changes.
So, the first step in the tuning process requires you to create a test workload with the same volume and pattern characteristics that your user community will generate during periods of peak production. This repeatable test workload allows you to measure the baseline performance of your system. After you make a thoughtful change, you can measure your system's performance again to ascertain any change in performance - positive or negative. This controlled process is the only sure way to know whether your tuning parameters are effective.
Next, you need an understanding of your server's system components and how they interact in your production environment. Some study of your production workload also helps. You should be familiar with the workload characteristics of your production environment. (These important volume and pattern considerations are outlined in "Using Production Workload Characteristics to Validate LAN Performance Evaluation Studies" in the March 1993 NetWare Application Notes.)
Once you have a representative test workload and some experience with your workload's characteristics, you're ready to begin testing.
This AppNote will provide you with a better understanding of several of NetWare 4.0's setable parameters. Knowing which parameters affect performance and what level of control (both positive and negative) they have over NetWare's performance will help you prioritize your tuning options.
How Auto-Tuning Works
Like NetWare 3.x, NetWare 4.0 performs several auto-tuning functions that automatically tune NetWare's configuration for changes in the production workload. In every case, auto-tuning increases the number of resources available to the operating system. These increases occur when there is an imbalance between service requests and available resources. When the server detects a need for additional bandwidth in any of the auto-tuning parameters, the parameter is increased.
However, because resources are limited, and because these parameters are never decreased, these automatic increases must be constrained. Constraints are also necessary to soften the allocation impact of brief utilization peaks. For instance, a large NDIR search could easily cause NetWare to permanently allocate most of the available cache memory for directory cache buffers if there were no limits placed on the allocation.
Auto-tuning constraints are imposed by using the SET console command at the server. Most of NetWare's setable parameters specify wait times, maximums, and minimums for allocating a particular resource. Since these terms are used frequently in the following discussion of NetWare's auto-tuning processes, I'll define them here:
Wait time is the length of time the operating system must wait, since the last allocation of new resource, before allocating a new resource unit.
Maximum is the upper limit (quantity) of resource allocation the operating system may not exceed.
Minimum is thenumber of resources before which the operating system may freely allocate new resources without any wait time. Once the minimum is allocated, the need for new resources must last longer than the prescribed wait time before NetWare may allocate each new unit of resource.
Tuning the Auto-Tuning Parameters
In NetWare 4.0, the auto-tuning facility impacts three operating system parameters:
Service Processes
Directory Caching
Physical Packet Receive Buffers
Service Processes
In NetWare, service processes are threads of execution that act as hosts to incoming service requests. NetWare 4.0 is capable of allocating up to 100 service processes. The following setable parameters control the allocation of service processes.
|
setableparameter = default setting |
min .. max |
|
Set maximum service processes = 40 |
5 .. 100 |
|
Set new serviceprocess wait time = 2.2 sec |
0.3 .. 20 |
Minimum Service Processes. A setable parameter for the minimum number of service processes is not available at the console. However, a minimum does exist and is set to 10. This means that after NetWare starts out with one service process, the operating system may freely allocate up to 10 service processes before using the wait time parameter.
Monitoring Service Processes. After installing a server, allow it to run for several hours or days. This break-in period will allow the server's resources to settle in while servicing the normal peaks and valleys of your production workload. You can use MONITOR.NLM to examine the number of service processes your server has allocated. NetWare reports this statistic on the main screen (see Figure 1).
Figure 1: NetWare displays the current number of service processes in the MONITOR utility's main screen.
In many cases, the number of service processes will remain below 10. In most cases the number will remain well below 40. This is common, since NetWare 4.0 is designed to service up to 1,000 users. Provided your server hardware platform is optimized for the number of users it is servicing, NetWare should perform beautifully.
When to Increase the Maximum. There are several exceptional cases where you would want to increase the maximum number of service processes. These include abnormally high levels of traffic or numbers of users. Servers servicing 250 to 1,000 clients may need to increase the maximum setting for additional service processes. Other exceptions can come from workloads generated by imaging and videomedia. These types of workloads are very different from the patterns generated by traditional file service-based applications. In these cases, the maximum setable parameter may need to be increased.
Whenever you increase the maximum, we recommend that your server's number of service processes remain two or three below the maximum. You may need to increase the maximum to guarantee this excess capacity.
Directory Caching
When NetWare 3.x was released in 1989, the majority of NetWare servers and their managed data spaces were smaller than today's average. Their associated directory spaces required less space as well. That made it possible to cache and hash the directory tables in NetWare 2.x and in previous versions.
However, since the release of NetWare 3.x, directory spaces are servicing entries for upwards of 500,000 files. The amount of memory required to cache the directory entries for 500,000 files is at minimum 64MB - an impossibility in many cases.
NetWare 3.x and 4.x both allocate less cache for the directories and use a least-recently-used (LRU) algorithm to discard unused directory cache blocks to make room for newly read or written blocks of data. This method of caching only the most recently used directory blocks works well for two reasons:
The file access patterns of most user communities remain somewhat isolated to SYS:\PUBLIC, common database files, user directories, and other common data sets. This means that a very few number of directory blocks are frequently used, most of which can be cached.
Due to the commonality of data shared in many user communities, many of the directory blocks are shared as well.
SET Parameters. The following setable parameters control the allocation of directory cache buffers.
|
setableparameter = default setting |
min..max |
|
Set maximumdirectory cache buffers = 500 |
20 ..4000 |
|
Set minimumdirectory cache buffers = 20 |
10 ..2000 |
|
Set directorycache allocation wait time = 2.2 sec |
0.5 ..120 |
Each directory cache buffer measures 4KB and can hold up to 32 directory entries. Each additional name space loaded also requires one directory entry for every file.
NetWare uses the LRU algorithm (described above) to determine whether additional buffers can be allocated when the need arises. If less than 2.2 seconds (the default setting) have elapsed since the least recently used buffer was last used, a new buffer is allocated.
When to Change the Settings. In many cases, the parameters related to the allocation of directory cache blocks do not need to be changed for either large or small data sets. However, there is one notable exception based on application type.
If the majority of the workload generated by the user community is file-oriented (as opposed to record-oriented), you may need to examine their file search patterns. File-oriented applications such as word-processing, spreadsheets, and other document-based tools allow for a great deal of directory browsing and searches. If this functionality is regarded as a high priority and the data set is very large, the minimum and maximum number of directory cache buffers may need to be increased.
If the maximum number of buffers is not reached under normal use, you can feel safe in doing nothing. Otherwise you may need to increase the minimums and maximums and possibly add additional RAM to your server for this specific purpose.
For instance, in a legal office, where document search and retrieval is a common occurrence, you could increase the minimum and maximum in such a way that would allow NetWare to permanently allocate 32MB of RAM for directory caching. Of course, each environment is different and may require more or less memory than this example.
If you decide to change these parameters, remember that the minimum number is not the number of cache buffers initially allocated. It is the number of buffers NetWare may freely allocate without any wait time. So you may not see the effect of your minimum setting until long after booting the server.
Packet Receive Buffers
Receive buffers are used to store incoming packets from each of the networks attached to a NetWare server. Each buffer is equal in size to the maximum physical receive packet size. In most cases, this is 1514 bytes for Ethernet segments and 4202 bytes for Token-Rings.
Larger packet sizes are possible on Token-Ring, but they are not supported by the DOS Requester v1.01 initially released with NetWare 4.0. Some developers, such as Fluent, are using the IPX API to transmit packets as large as 8KB.
SET Parameters. The following setable parameters govern the number, size, and allocation wait time for receive buffers.
|
setableparameter = default setting |
min .. max |
|
Set maximumpacket receive buffers = 100 |
50 ..2000 |
|
Set minimumpacket receive buffers = 10 |
10 ..1000 |
|
Set new packetreceive buffer wait time = 0.1 sec |
0.1 ..20 |
|
Set maximumphysical packet receive size = 1514 |
618 .. 24682 |
NetWare allocates new receive buffers when it runs out- in other words, when incoming requests are present with no free buffers. After the minimum is reached, NetWare waits 0.1 seconds (the default setting) to determine whether the need can be satisfied with existing buffers. If not, NetWare allocates a new buffer.
A good rule of thumb for the maximum packet receive buffers setting is approximately one receive buffer for each client. So, if the server won't be servicing more than 100 clients, the default of 100 should be plenty.
Monitoring Receive Buffers. Of all of the resources affected by auto-tuning, the number of receive buffers is potentially the most volatile. Before proceeding with any tuning of the receive buffer setable parameters, we recommend that you use a protocol analyzer to identify:
Peak periods of workload
The number of clients employing Packet Burst and how often it is used
The number of packets in each burst (window size)
Packet Burst, a new protocol built into the DOS Requester on top of IPX, allows a stream of up to 64KB of packets to be sent from one network address to another. If the path between those two addresses passes through the server, using it as a router, or if one of those addresses belongs to the server, the previous rule of thumb does not apply.
In these cases, where Packet Burst is employed frequently, the minimum and maximum settings will need to be watched carefully. You can monitor the number of packet receive buffers in MONITOR.NLM's main screen (see Figure 2).
Figure 2: NetWare displays the current number of packet receive buffers in the MONITOR utility's main screen.
When to Change the Settings. If your server allocates the maximum number of buffers, you run the risk of the server's LAN channel reaching an overflow condition in which it ignores incoming packets until additional receive buffers are available. When this happens, those packets' clients resend their packets. Nothing is lost, but each client pays a penalty in response time.
This problem is exacerbated when the server is routing between two or more networks with a significant differential in speed. In this case, it is possible for a packet stream to arrive on the "fast net" destined for an address on the "slow net" and occupy valuable receive buffers for a significant period of time. In these types of configurations, the minimum and maximums may need to be increased greatly.
To better handle these types of situations, Novell has increased the priority of routing requests in NetWare 4.0. This change should reduce the number of receive buffers required to service a given workload, when compared to the number of buffers required in NetWare 3.x.
Summary
This AppNote has discussed NetWare 4.0's auto-tuning capabilities and the SET parameters related to service processes, directory caching, and packet receive buffers. Part 3 of this series will detail the NetWare 4.0 setable parameters related to the cache, the LAN channel, and the disk channel.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.