Memory Fragmentation and Segmentation
Articles and Tips:
01 Apr 1998
Editor's Note: "Technically Speaking" answers your technical questions, focusing on issues that affect network administrators. To submit a question for a future column, please send an e-mail message to nwc-editors@novell.com, or fax the question to 1-801-228-4576.
I was recently asked to solve the following problems:
My company's server has plenty of memory, but NetWare 4 displays Short Term Memory Alloc messages. What's wrong?
My company's server has 128 MB of RAM, but NetWare 4 reports only 64 MB of RAM. What's wrong?
These errors are caused by two problems: The first error is caused by memory fragmentation, and the second error is caused by memory segmentation. This article describes how memory fragmentation and memory segmentation occur and explains how you can resolve both problems on a NetWare 4 server. (Because the network operating system component of intraNetWare is NetWare 4.11, this article also applies to intraNetWare.)
MEMORY FRAGMENTATION
Memory fragmentation eventually occurs on all NetWare 4 servers. Depending on the way you manage a server and the NetWare Loadable Modules (NLMs) you run, memory fragmentation can occur daily or occasionally, over a period of years.
The most common cause of memory fragmentation is loading and unloading a scheduled NLM, such as a backup NLM. However, other automated NLMs can also cause memory fragmentation. For example, Novell's FTP server can cause memory fragmentation because the FTP server automatically loads when a request is received and then unloads when the request times out.
Memory fragmentation can also be caused by NLMs that are unloaded and then reloaded as part of another process. For example, a backup NLM could schedule the unloading of a database NLM during the backup process. The backup NLM would then reload the database NLM when this process was completed.
Since a database NLM is designed to be loaded and left running, this NLM makes permanent memory pool allocations, which are not returned to system memory when the NLM is unloaded. When the database NLM is reloaded, it may not reuse its permanent memory pool allocation and may, therefore, leave gaps in memory. As a result, memory fragmentation may occur.
Although memory fragmentation can cause several errors, it most often results in Short Term Memory Alloc messages at the server console. These messages indicate that small memory resources are not available to the requesting process.
SOLUTIONS FOR MEMORY FRAGMENTATION
To resolve memory fragmentation, you should first ensure that the following command is included in the STARTUP.NCF filebeforeyou load disk drivers or name space drivers:
SET RESERVED BUFFERS BELOW 16 MEG = 300
By setting this parameter to 300, you allocate the largest memory pool possible in low memory to be used for short-term memory allocations. As a result, NetWare 4 does not need to allocate high memory to NLMs that make short-term memory allocations.
If changing this parameter does not resolve memory fragmentation, you must down the server and restart it. If the server frequently experiences severe memory fragmentation, you should identify which NLMs are being loaded and unloaded and determine how you can leave these NLMs loaded all the time.
MEMORY SEGMENTATION
Memory segmentation occurs when system memory is presented to NetWare 4 as two or more noncontiguous memory blocks. Although several factors can cause this condition, the result is always the same: The NetWare Cache Memory Allocator cannot use all of the installed RAM. Depending on the cause, NetWare 4 may or may not see all of the installed RAM.
If the NetWare Cache Memory Allocator cannot use all of the installed RAM, the server may display error messages. Most frequently, the server reports that the NetWare Cache Memory Allocator is out of available memory.
SOLUTIONS FOR MEMORY SEGMENTATION
The solutions used to resolve memory segmentation on NetWare 3 servers do not work on NetWare 4 servers. NetWare 3 is based on a multipool memory model and doesn't allocate memory for the NetWare Cache Memory Allocator until the first volume is mounted. As a result, you can prevent disk drivers from loading in the STARTUP.NCF file, and you can use the REGISTER MEMORY command before loading disk drivers and mounting volumes in the AUTOEXEC.NCF file. NetWare 3 can then see all of the available memory before allocating memory for the NetWare Cache Memory Allocator.
Unlike NetWare 3, NetWare 4, is based on a single-pool, flat-memory model. When NetWare 4 is initialized, it immediately allocates memory for the NetWare Cache Memory Allocator. As a result, NetWare 4 can allocate only the memory that is physically available at the time. Once NetWare 4 allocates memory for the NetWare Cache Memory Allocator, NetWare 4 cannot dynamically reallocate this memory.
If you use the REGISTER MEMORY command to resolve memory segmentation, NetWare 4 cannot use the additional memory it sees for internal processes. NetWare 4 can use this additional memory only for file cache buffers.
To resolve memory segmentation, you should first ensure that you have not imposed false memory limitations on the server. Loading a DOS memory manager (HIMEM.SYS or EMM386.EXE, for example) in the CONFIG.SYS file is one of the most common causes of memory segmentation. You should also ensure that you are not loading a CONFIG.SYS file on the server's boot disk or boot partition.
Next, you should run the computer's Setup program to ensure that you have not set any special memory options. If you do not understand the memory options, you can contact the computer manufacturer to ensure that you are using the proper settings for a NetWare 4 server.
Next, you should install the latest NetWare 4 patches and updates. One of the most important patches is the Loader patch, which is a static patch that is applied to the SERVER.EXE file. This patch changes the way NetWare 4 looks at system memory. (You can download intraNetWare Support Pack 4.0, which contains the latest NetWare 4 patches and updates, from Novell's Support Connection World-Wide Web site at http://support.novell.com.)
Unfortunately, the Loader patch does not resolve memory segmentation on some computers. And on some computers, this patch prevents NetWare 4 from running, either causing an ABEND when the SERVER.EXE file is loaded or locking the computer as soon as this file starts to run.
The Loader patch may also have conflicts with some PCI plug-and-play adapters. These adapters set their memory address buffers when they power on and cannot reset these buffers when NetWare 4 loads. As a result, these adapters may fail if you install the Loader patch.
If the Loader patch causes these kinds of problems, you must replace the modified SERVER.EXE file with the original file (which is renamed SERVER.OLD when you install the patch).
If you have not imposed false memory limitations, if you have selected the proper memory options, and if you have installed the latest patches and updates and NetWare 4 still does not see all of the installed memory, the problem is a physical limitation of the computer. This problem can occur if NetWare 4 is running on a clone PCI and ISA bus computer.
At this point, you should contact the computer manufacturer to see if this manufacturer has a system BIOS upgrade or a flash update that allows NetWare 4 to see all of the installed RAM. Because some computers were not designed specifically to be used as servers, the system BIOS may implement a RAM addressing limiter to ensure compatibility with Windows 95, Windows 3.x, and DOS. The limiter allows the computer to have more RAM but reports only 16 MB of RAM to NetWare 4. In some PCI and ISA computers, for example, the limiter reports a maximum of 64 MB of RAM to NetWare 4.
The computer may also be physically limited because it has a 16-bit bus master device. Such devices can address only a limited amount of RAM because their embedded processor has a fixed number of address lines available. Trying to force 16-bit bus master devices to address RAM beyond their physical limitation eventually causes RAM corruption, which leads to ABENDs and data corruption.
Although you may not have installed a 16-bit bus master device, the computer may have an embedded 16-bit bus master device. For example, many lower cost PCI and ISA computers that were not explicitly designed to be servers use 16-bit PCI-to-ISA bridge devices. These bridges are inherently bus master devices. And because the computer has an embedded 16-bit bus master device, the computer cannot directly address memory above 16 MB.
Some device manufacturers are providing drivers for their 16-bit bus master devices that allow for double buffering. These drivers can reduce memory segmentation because they perform double memory moves for each I/O operation. Because these drivers do not remove the computer's physical limitation, however, a highly utilized server may run out of reserved buffers below 16 MB. Once again, RAM corruption may occur.
To permanently resolve memory segmentation, you must use computers that are specifically designed to be used as servers. These computers have a true 32-bit bus design, which means that they use EISA or Micro Channel primary buses and can also have PCI local buses through 32-bit PCI bridge devices.
Some computer manufacturers have produced lower cost PCI and ISA computers that use a 32-bit PCI bridge device. Before purchasing such a computer, you should compare it to a true 32-bit computer. You may find that many lower cost PCI and ISA computers use 16-bit embedded LAN and disk adapters. EISA and PCI computers, on the other hand, use 32-bit embedded adapters.
When configuring a computer to be used as a server, you should use 32-bit LAN and disk adapters. If you do install a 16-bit ISA, Micro Channel, or PCI adapter, you should ensure that it is not a bus master adapter. (For more information about how memory segmentation affects NetWare 4, download the Memory Segmentation technical information document from Novell's Support Connection site at http://support.novell.com. To find this document, search for document number 2908018.)
CONCLUSION
Memory fragmentation occurs with all operating systems, not just NetWare 4. If memory fragmentation begins to affect system performance or data integrity, you must reboot the server.
Memory segmentation, on the other hand, is caused by the physical limitations of the computer. (Such physical limitations also affect all operating systems, not just NetWare 4.) Either the computer is limited through its inherent design or through the use of devices that prevent NetWare 4 from directly addressing all of the available memory. The best solution is to use a computer that does not have these physical limitations.
Mickey Applebaum has worked with NetWare for 14 years. Mickey currently provides technical support on the Internet for The Tech Forums Inc. (http://theforums.com). You can reach Mickey atmickey@homein.salt-lake-city.ut.us.
NetWare Connection,April 1998, pp.41-42
* Originally published in Novell Connection Magazine
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.