Novell is now a part of Micro Focus

Memory Utilization and APIs: A Comparison of NetWare 4.x and NetWare 3.12

Articles and Tips: article

KEVIN BURNETT
Developer Support Engineer
Developer Support

01 Dec 1996


Discusses three memory utilization issues: NetWare 3.12 memory architecture (a brief description), NetWare 4.10 memory architecture, and the APIs available to both platforms.

Introduction

Memory utilization on a NetWare server has always been an issue for the NLM developer. This article will discuss the following three memory utilization issues: NetWare v3.12 memory architecture (a brief description), NetWare v4.10 memory architecture, and the APIs available to both platforms.

NetWare 3.12 memory management is somewhat CPU efficient; however, the memory allocation algorithms can produce inefficiencies. NetWare 3.12 has three main pools of memory and three sub-pools. The three main memory pools include:

  • System cache buffers

  • Permanent

  • Alloc memory

Note: Memory that is not used by the NetWare operating system or for DOS (if loaded) is given to these memory pools.

NetWare 3.12 allocates minimal memory to the permanent and alloc pools. These pools can grow dynamically on demand.

The system cache is the amount of memory available for use by the operating system after the NetWare server is loaded into memory. After the operating system is loaded into memory, the supervisor can choose cache buffers of 4, 8, or 16 KB. Cache buffers are used by the operating system in a variety of ways, including: LAN drivers, disk drivers, utilities and NLMs, caching each volume's file allocation table (FAT), caching some of each volume's directory allocation table (DAT), and building a hash table of directory names for its own various uses.

The system cache buffer pool stores the most frequently used files. This is the pool from which all other pools obtain additional memory. Two sub-pools, cache movable and cache non-movable, obtain and return memory from the system cache buffer pool.

The cache movable pool is used for system tables that change size, for example, FAT's and hash tables. The cache non-movable pool is used for NLMs. Both the cache movable and cache non-movable memory pools return memory directly to the system cache buffer pool when the memory is no longer needed.

The permanent pool is used for long-term memory needs, such as packet receive buffers and directory cache buffers.

The alloc memory pool stores information such as drive mappings, service request buffers, files, SAP information, connection information, broadcast messages, NLM tables, and QMS tables.

Since NetWare 3.12's memory architecture is globally based, complex memory management scenarios are commonly encountered. Each memory pool provided memory to system processes according to certain needs. For example:

  • The operating system allocated memory from the permanent pool for data structures that needed to be present in memory for as long as the server was running.

  • The alloc pool was mostly used for short-term allocations.

  • Only three of the five allocation pools returned unused memory to system cache. Two of the pools had to rely on other pools for their supply of memory.

While in theory these five memory pools successfully satisfied various memory needs, real-life implementation generated unfavorable conditions. There are four main reasons why the NetWare server memory architecture was redesigned for NetWare v4.x:

  1. A new memory configuration that reduces the number of NetWare 3.12's memory pools.

  2. A new memory map that allows NetWare to write-protect OS and NLM code space.

  3. A new memory allocation scheme which is designed to be easier to program to and better synchronized to production server characteristics.

  4. The NetWare 3.12 implementation became incompatible with future plans regarding memory protection.

NetWare 4.x Memory Architecture

NetWare 4.x uses page protection hardware that is a part of the Intel 386 and greater processors. This memory scheme divides the server's memory, beginning at address 0 to the end of physical memory, into usable units called pages. A page based memory scheme not only provides a foundation for memory protection, but also provides a cleaner environment for internal memory allocation. The server's cache memory now exists as a global pool of 4KB physical pages that the operating system maps to various processes.

As you load an NLM and request memory, NetWare takes the requested memory from cache in page-sized increments. When the NLM is unloaded or releases memory, the pages are returned to the cache. Since the system cache uses all memory not used by the operating system and currently loaded NLMs, none of the server's memory goes unused. All the pages that the operating system allocates can be deallocated and returned to system cache.

NetWare 4.x's new memory organization improves on NetWare 3.12 in several ways. In NetWare 3.12, some memory pools permanently allocated memory that could never be returned or reused by other NLMs. NetWare 4.x's memory management scheme allows all of the memory allocated by NLMs to be returned to the system cache for reuse by the next requesting process. This greatly reduces fragmentation of memory. In NetWare 3.12, memory pages could be shared by two or more NLMs. This made it very difficult to reuse memory pages until all NLMs were finished with each shared page. NetWare 4.x alleviates this problem by not allowing NLMs to share memory pages.

Memory Map

NetWare 4.x uses a process provided by the system CPU called mapping. This allows physical memory to be reorganized much more efficiently. It also allows system memory to have two or more views:

  • Physical memory, which is the actual memory.

  • Logical memory, a new way of looking at system memory.

The mechanism that translates between physical memory and logical memory is called a memory map. By using memory maps, NetWare can reduce memory fragmentation by assembling non-contiguous blocks of physical memory into contiguous blocks of logical memory.

NetWare 4.x's operating-system code is loaded high in logical memory in the F5000000 and F8000000 address spaces. All NLM code is also loaded into logical memory between F0000000 and F5000000. The NLM's data regions are mapped down to the physical memory area. Since this code is loaded high, NetWare is able to take advantage of memory write-protection services provided by the CPU to protect the OS and NLM code pools from being overwritten.

Memory Allocation

NetWare 4.x's memory allocation APIs (see below) are much more simple than NetWare 3.12. The APIs are more usable and noticeably more efficient. Each process requests memory from its own pool and can return the memory to the pool.

The page-based global memory pool will adequately fill all memory needs. The operating system gives each process/NLM its own allocation pool of memory. Since each process manages its own memory, allocation is faster and less complex.

NetWare 4.x's memory allocating system manages free memory made up of blocks of memory left over during a previous allocation process and blocks that have been released by NLMs. NetWare 3.12 used one linked list to manage free memory. This linked list is made up of memory blocks in sizes ranging from 8 bytes to greater than 4KB. On a given server, after time, this free list becomes very large. Each time an NLM allocates a block of memory, the memory allocation system traverses this free list looking for the first best fit. Ideally this allocation process should be very fast, but in reality, due to the length of the list, this process slows down with time.

This same process is repeated each time memory is returned to the free list. As a result, NetWare 3.12 servers could become bogged down when allocating and deallocating memory.

To overcome the problems associated with traversing large linked lists, NetWare 4.x manages 77 separate free lists for each NLM. These free lists are based on size and include:

  • An array of 64 free lists in 16-byte increments for memory blocks up to 1KB in size.

  • An array of 12 free lists in 256-byte increments for memory blocks from 1KB to 4KB in size.

  • A single free list for memory blocks larger than 4KB in size.

There are many benefits to each process/NLM having its own private allocation scheme, some of which include the following:

  • Greatly improves overall memory performance especially in allocates and deallocates.

  • The operating system is now able to anticipate memory requests and prepare memory in advance. When an NLM allocates memory, the operating system rounds the request up to the nearest 4KB increment and then doubles the amount before taking the total amount from cache. This chunk of cache memory is then split upto satisfy the NLM's request. The remaining memory produced from doubling the request size is placed in one of the NLM's free lists in anticipation of similar requests of the same size. These extra memory blocks can have an important impact on performance because requests serviced from the NLM's free list are much less time consuming than those serviced from the server'smemory cache.

  • NLM applications tend to make a set number of allocations and then use that pool of memory over and over during the life of the process. The NetWare 4.x memory management scheme is perfect for this scenario. The memory allocation routine creates the proper number and sizes of memory structures during initial allocation requests. Subsequent malloc/free calls then proceed quickly.

  • Novell Research has found that individual NLMs favor specific sizes of memory blocks. For example, one NLM may request memory in 32-byte and 2KB blocks, while another NLM may request only 4KB blocks. This allocation scheme provides each NLM with its own collection of right-sized memory blocks.

Memory Deallocation

The NetWare memory free routine is simple and fast. After performing checks for overwrites, and returning the resource tag, the free routine links the memory structure back onto the same linked list from which it was allocated. The typical execution path through the free code is fast and efficient, typically executing less then 20 instructions.

The memory free routine does not perform garbage collection at every instance. Comparing NetWare 3.12 with NetWare 4.x shows that continual garbage collection, which is the process of reclaiming used memory, is counterproductive.

Garbage Collection

Reclaiming used memory is important in the NetWare 4.x memory scheme. In the situation where a program uses small sizes of memory to initialize and then larger amounts of memory to perform the rest of its operations, garbage collection is critical. Unless the smaller nodes used first are collected and returned to the allocation pool, system memory becomes used up over time.

The internal routine that handles garbage collection sorts the memory structures currently on the NLM's memory linked lists and combines them into larger pieces. The larger pieces are then linked to the appropriate list head. If the garbage collection routine frees an entire 4KB page, that memory is returned to system cache memory. The garbage collection routine is interruptible and can run in the background.

System garbage collection is controlled via server-settable parameters, which are beyond the scope of this article. Please refer to the NetWare 4.0 Architecture manual in the SDK documentation.

NetWare 3.12 Specific Memory Management APIs

Alloc. This function allocates memory from the short-term memory pool. This call should be used for small amounts of memory (less than 4KB) that are to be kept for a short period of time. This function allocates the exact size of the memory requested. The main difference between Alloc and malloc is that Alloc allows you to associate a Resource Tag with the allocated memory while malloc associates a catch-all resource tag created by Clib for your NLM.

AllocNonMovableCacheMemory. This function allocates a large block of memory from the cache buffer memory pool. This function should be used for large amounts of memory (greater than 4KB). Memory can be kept for any amount of time before being returned. The returned memory goes back to the cache buffer pool. Because memory does not move, fragmentation of the cache buffer memory pool can occur in extreme cases.

AllocSemiPermMemory. This function allocates a small amount of memory from the long-term memory pool. It should be used for small amounts of memory (less than 4KB) for any period of time. This function allocates the exact size of the memory requested. Memory cannot be returned to the cache buffer pool. Memory does not move, so memory fragmentation can occur.

Free. This function releases memory allocated by Alloc.

Note: If the value passed in to this function doesn't point to previously allocated memory, the server will Abend.

FreeNonMovableCacheMemory. This function releases memory allocated by AllocNonMovableCacheMemory. If the address of the allocated memory does not point to previously allocated memory, the server will Abend.

FreeSemiPermMemory. This function releases memory allocated by AllocSemiPermMemory. If the address of the allocated memory does not point to previously allocated memory, the server will Abend.

Since all of the NetWare 4.x memory management APIs are NetWare 3.12 backwards compatible, the following list comprises memory management APIs for both platforms.

NetWare 3.12 and 4.x Memory Management APIs

Alloca. This function allocates and clears memory space for a block of memory on the stack. This function allocates memory space for an object of size bytes from the stack. The allocated space is automatically discarded when the current function exits.

Do not call free, this could have disastrous consequences.

Allocating large blocks of memory with this function has a negative impact on available stack space.

Calloc. This function allocates and clears memory space for an array of objects. The function initializes all the memory to binary zeros. Memory allocation is not limited to 64KB. The calloc function calls the malloc function.

Free. This function frees a previously allocated memory block. This function frees memory pointed to by the address passed into it.

Malloc. This function allocates a block of memory. Memory allocation is not limited to 64KB.

_msize. This function returns the size of a memory block. It returns the size of the memory block that was allocated by a call to calloc, malloc, or realloc.

NWSAlloc. This function allocates memory for NWSNUT purposes.

NWSFree. This function frees memory allocated by NWSAlloc.

__qcalloc. This function allocates memory space for an array of objects. This function initializes all the memory to binary zeros. This function is non-blocking along with __qmalloc and __qrealloc.

__qmalloc. This function allocates memory space for an object.

__qrealloc. This function reallocates memory space for an object. This function calls __qmalloc to enlarge a block of memory.

realloc. This function reallocates a block of memory. It calls malloc to enlarge a block of memory.

stackavail. This function returns the number of bytes currently available in the stack. This value is usually used to determine an appropriate amount to allocate using alloca.

t_alloc. This function dynamically allocates memory for the various TLI function argument structures. It also allocates memory for buffers referenced by the structure. The memory allocated by t_alloc is freed by calling t_close.

t_close. This function informs the transport provider that the user is finished with the specified transport endpoint and frees any local library resources associated with the endpoint, including any memory allocated by t_alloc.

Green River

Memory management in Green River, NetWare 4.x's next release, is essentially the same as NetWare 4.10 with a few minor exceptions:

  1. Domain functionality has been removed. DOMAIN.NLM will not ship with Green River. In addition, DOMAIN.NLM will not load on a Green River server.

  2. The garbage collection routines have been enhanced to run significantly faster than those in NetWare 4.10.

Green River will support the existing API interface provided for NetWare 4.10 with no changes.

Summary

NetWare 4.x offers a superior memory management system as compared to NetWare 3.12. NetWare 4.x offers a simpler and more efficient way for allocating and feeing memory compared to NetWare 3.12's multiple memory pools and its tendency to fragment memory. NetWare 4.x also offers a way to reclaim memory by performing garbage collection. Without garbage collection, memory can become unusable to either the operating system or NLMs. In addition, NetWare 4.x's API set is backwards compatible with NetWare 3.12, whereas the reverse is not true.

References

  1. NetWare 4.0 Architecture manual, from the Novell Software Developer's Kit documentation.

  2. Concepts manual, from the Novell NetWare 3.12 documentation set.

  3. "ServerMemory, The NetWare 4 Memory Architecture," from Novell Application Notes, April 1995.

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates