Novell is now a part of Micro Focus

Introduction to NetWare 5 Memory Enhancements

Articles and Tips: article

KEVIN BURNETT
Software Engineer
Stategic Partner Engineering Group

01 Dec 1999


Covers NetWare 5's new memory protection architecture, new virtual memory manager, and automatic application restart using virtual memory. Includes sample program and code.

Introduction

With the introduction of NetWare 5 in the fall of 1998, a whole new world was opened to Novell Developers. Several new features were added, prominent features were enhanced, and a generally improved product was shipped. Discussing all of NetWare 5's new features is not the point of this article; that would take up all the space in this magazine and then some.

What I would like to cover in this article is the following:

  • NetWare 5's new memory protection architecture.

  • NetWare 5's new virtual memory manager.

  • NetWare 5's automatic application restart using virtual memory.

Lastly, I would like to walk through a demonstration of NetWare 5's memory protection and NetWare 5's virtual memory manager.

Before you understand what memory protection is, there are several terms you should be familiar with.

Address Space

The set of memory locations that are accessible at one time. Basically, this is all the Physical RAM in your computer, plus any virtual RAM the OS utilizes.

Protection Fault

Generated by an Intel processor when you try to access memory outside your address space.

Supervisor Mode/User Mode

The Intel Processor supports multiple execution modes. More about this later in the article.

Functional Units

A group of NLMs that perform an independent function.

Features

NetWare 5's Memory Protection provides many new features within NetWare. The highlights are as follows:

  • There are separate user address spaces.

  • The OS is protected from NLMs.

  • NLMs are protected from other NLMs.

  • Faults are isolated to the offending functional unit.

  • The faulting functional unit is removed from memory.

  • The optional restarting of functional units.

The remainder of this part of the article looks at each of these in greater depth.

Address Spaces

Little has changed with NetWare 5 if you look at the NetWare Operating System (OS) address space. You can still load NLMs like you have in previous versions of NetWare, and the NLMs will act the same. Once loaded, the NLMs become part of the NetWare OS executing in the OS's memory. There is no memory protection within the OS's address space.

Figure 1: Classic NetWare environment.

User address spaces are new with NetWare 5. With user address spaces, NLMs are loaded into their own protected address space. They form their own independent functional unit. They do not have direct access to the OS address space. From the perspective of the Intel processor, this would be Ring 3, or protected memory. In order for an NLM loaded in a protected address space to access the OS, it must direct the requests through a controlled interface called a Call Gate. I will talk about the Call Gate in a later section.

Figure 2: User address spaces added to NetWare.

With NetWare 5's Memory protection you can have as many user address spaces as file server memory will permit. Virtual memory, to be discussed later, will practically allow you to have unlimited user address spaces.

An important thing to remember is that an NLM loaded into one user address space is protected from any other NLMs loaded on the NetWare 5 server. Likewise, multiple NLMs loaded in multiple unique user address spaces are protected from each other, and the OS address space is protected from them.

Figure 3: Multiple user address spaces.

Address Map

In NetWare 5 the server memory is divided into two areas, OS and User. This can be illustrated as follows:

Figure 4: NetWare 5 memory division.

When in supervisor mode, the OS and user address spaces are accessible. However, when in user mode, only the user address space is accessible. NLM applications do not need to interact with these hardware options. Only the NetWare OS needs to be concerned with them. These memory divisions correspond to the OS address space and the user address space in NetWare 5.

System Call Interface

The System Call Interface or Call Gate, implemented in NetWare 5, is a combination of software and hardware: software as a part of the NetWare 5 OS, and hardware functionality that is built into the Intel processor. The Call Gate traps requests that may reference invalid memory or attempt to directly interface with the OS in an unauthorized manner. With the NetWare Call Gate most problems can be caught by the software, but it is enforced by the Intel processor. This allows the NetWare OS to function more efficiently and protects the OS.

Figure 5: System call interface.

User Address Spaces

As stated earlier, the user address spaces in NetWare 5 are the protected places in memory where an NLM can run and be protected from other misbehaving NLMs. More importantly, the OS is protected from misbehaving NLMs. This prevents server Abends, since the misbehaving NLM's user address space will be automatically cleaned up, from affecting the OS address space.

Additionally, to provide access to libraries for NLMs running in user address spaces, CLIB has been modified to run in this environment. Normally, you would think you need to load a separate copy of CLIB in each user address space, and that would make sense, but CLIB has been engineered to allow only one instance, per NetWare 5 server, of the CLIB code segments to load. However, a separate CLIB data segment is loaded for each user address space. To sum this up, CLIB code can be shared by all the user address spaces while each user address space will have unique CLIB data.

Figure 6: Libraries in user address spaces.

Memory Protection Features

There are several features in the NetWare 5 Memory Management System. One of these is the Protection Fault Handler which detects user address space faults. It also prevents any code located in the faulting address space from being executed, scheduling the faulted address space for cleanup. OS address space protection faults are passed to the abend recovery mechanism in NetWare.

When a User Address Space faults, the user address space is cleaned up. This means the memory that was used for the user address space is returned to the NetWare OS. All the resources in the user address space are cleaned up, and a notification is displayed on the NetWare 5 Console Screen. There is an optional automatic re-creating of the user address space with automatic reloading of the NLM program. This will occur either immediately after the fault or at a specified time interval after the fault. This will be discussed in more depth later.

How To Use Memory Protection

What type of code can take advantage of a protected address space:

  • Code that doesn't enable or disable interrupts

  • Code that doesn't reference global variables located outside of the user address space

  • Code that is written to use the NLM NDK/CLIB

What type of code has to run in the OS address space:

  • NetWare OS

  • Code that manipulates interrupts, i.e., hardware device drivers (LAN and disk)

  • Code that directly references OS data

  • Code that uses "dirty hooks," i.e., OS patches

  • Code that runs in "classic" NetWare OS

New Commands

The NetWare console command interface has been modified to allow interaction with memory protection. There are several new commands and command enhancements, including:

  • Load console command enhancements

  • Protection console command (managing address spaces)

  • Protect console command (for .NCF files)

  • Unload console command enhancements

Let's look at each of these commands and command enhancements individually.

Load Console Command The syntax is as follows:

Load protection options nlm nlm options

The modifications to the traditional Loadcommand will load an NLM in a user address space when any of the following protection options are specified:

  • Protected (used to create a user address space)

  • Restart (used to create a re-startable user address space)

  • Address space=space name (used to give a desired name to a user address space)

Examples:

Load protected clib

This will create a protected user address space and place a unique data portion of CLIB in it. The user address space name will be ADDRESS_SPACE1 on a NetWare server without any previous unnamed user address spaces. NetWare 5 increments the number at the end of ADDRESS_SPACE for each unnamed user address space created. If one of these unnamed user address spaces faults, its name will not be used again until the NetWare 5 server is rebooted. The only exception to this rule is if you enable automatic restart.

Load restart clib

This will create a protected user address space and place a unique data portion of CLIB in it. The user address space name will be ADDRESS_SPACE1 on a NetWare server without any previous unnamed user address spaces. Additionally, this will make the user address space re-startable, meaning that if the user address space faults, it will be re-created and the NLMs reloaded into it. If you have an NLM that is misbehaving, i.e., abending the user address space continually (continual loop of creating the user address space, loading the NLM, abending the user address space), then you can take advantage of two new NetWare 5 console commands.

Memory Protection Restart Count: (default) 1 Limits: 0 to 1000 Can be set in the startup ncf file. Description: This set parameter works in conjunction with the Memory Protection No Restart Interval parameter. Administrators can use this value to specify the number of restarts allowed during the Memory Protection No Restart Interval. The default value for this set parameter is 1, such that if more than one protection fault occurs in an address space, the address space will only be restarted once. If you wish to configure NetWare to allow two restarts during a three-minute period, then you should set the Memory Protection Restart Count to 2 and the Memory Protection No Restart Interval to 3.

Memory Protection No Restart Interval: (default) 1 Limits: 0 to 525600 Can be set in the startup ncf file. Description: Don't restart a user address space if the address space is faulting more than once during the specified number of minutes. A value of 0 disables this set parameter. If a memory protection violation is attempted in a restartable address space, the offending address space, and its loaded NLMs, are removed, and their resources are returned to the system. A new user address space with the same name is created and the NLMs are reloaded. The restart feature is disabled if the user address space was restarted more recently than the interval specified by this parameter.

Load address space=GroupWise restart clib

This will create a user address space called GroupWise and make it restartable. Then, it will load a unique data portion of CLIB into it.

Protection Console Command The syntax is as follows:

Protection [[no]restart]address space

This new console command will display information about the protected memory address spaces in the system. Additionally, it provides the ability to toggle the restart feature for a protected address space.

Examples:

Protection GroupWise

Displays the user address space called GroupWise and all the files that are contained therein.

Protection restart GroupWise

In addition to display information about the user address space GroupWise, makes the user address space GroupWise restartable.

Protection no restart GroupWise

In addition to display information about the user address space GroupWise, makes the user address space GroupWise non-restartable.

Protect Console Command The syntax is as follows:

Protect .NCF file name

This new console command loads NLMs from an NCF file into a user address space. The user address space can be specified in the NCF file.

Examples:

Protect oraload

This will load the Oracle database NCF file into user address space. The details as to how many user address spaces to use and the names of the user address spaces are all handled in the NCF file.

Unload Console Command The traditional NetWare Unload command has been modified to support Memory Protection. The syntax is as follows:

Unload protection options NLM

The protection options for unload are address space=space name and kill. The first parameter allows the unloading of all the files in a specific user address space. The second allows the unloading of the files in the user address space, and then the `killing' of the user address space. The resources are then returned to the NetWare OS.

Examples:

Unload clib

Traditional unload. This unloads CLIB from the OS address space.

Unload address space=GroupWise

This unloads all the files that are in the user address space called GroupWise.

Unload address space=GroupWise clib

This unloads only CLIB from the user address space named GroupWise.

Unload kill address space=GroupWise

This unloads all the files in the user address space GroupWise, and then remove the user address space GroupWise. The resources will be returned to the NetWare OS.

Virtual Memory

Virtual memory has been introduced with NetWare 5. Some of the features of virtual memory are to provide multiple large contiguous logical address spaces, provide environment sharing, prevent many types of memory fragmentation, and enhance system performance.

Virtual memory is fully backwards compatible with previous versions of NetWare if the NLM is loaded into ring 0 or OS memory space. With virtual memory the implementation maps most of the previous memory calls to virtual memory calls for NLMs loading in protected address spaces.

Note: You can only access NetWare virtual memory through a user address space. Loading into the OS address space will prevent your NLM from using virtual memory, although the OS will use virtual memory for it's own needs.

Some developers may question how virtual memory will affect drivers and NLMs that need access to physical memory. Novell has provided driver (LAN and disk) developers with an API to allocate guaranteed physically contiguous memory and memory below 16MB, which is required for DMA transfer and so on. This API is a part of the Novell NDK available for download at www.developer.novell.com.

NetWare's Virtual Memory provides support for multiple swap files on multiple NetWare volumes. You are not limited to SYS volume like so many other things in previous versions of NetWare.

There are two easy ways to manage NetWare's virtual memory. The first is the SWAP console command. This will give you all kinds of statistics about your swap files and the way they are set up. Additionally, you can create, delete, and manipulate swap files.

SWAP ADD|Delete volume_name

This command ads or removes the swap file from a volume and sets MIN, MAX and MIN Free. If no parameters are given then swap file information is displayed.

All Values are in millions of bytes. MIN or MINIMUM = Minimum swap files size. (default = 2) MAX or MAXIMUM = Maximum swap file size. (default - Free volume space) MIN FREE or MINIMUM FREE = Minimum free space to be preserved on a volume outside the swap file; this controls the maximum size of the swap file on this volume. (default = 5)

Example:

Swap

Displays swap file statistics.

Swap add vol2

Adds a swap file to volume vol2 with default parameters.

Swap add vol3 min = 5 max = 100 min free = 10

Adds a swap file to volume vol3 with a minimum size of 5MB and a maximum size of 100MB, guaranteeing that 10MB will be preserved on the volume.

Swap delete vol3

Deletes the swap file on volume vol3.

Swap parameter vol2 min = 2 max = 1000 min free = 100

Modifies the parameters for the swap file on volume vol2 to include a minimum size of 2MB and a maximum size of 1gb, guaranteeing 100MB will be left on the volume.

Additionally, NetWare 5's Monitor utility's Virtual Memory option will allow you to do all that the SWAP console command will and more.

Demonstration Activities

The accompanying utility/source code called BADPROG.NLM will allow you to test both NetWare 5 Memory Protection and Virtual Memory. BADPROG.NLM will do all kinds of bad things to your NetWare server. Figure 7 shows the main screen of BADPROG.NLM:

Figure 7: BADPROG Main Menu.

BADPROG.NLM is nothing fancy; The user interface is very simple. Option `0' allows you to exit the program, and options 1, 2, and 3 are for experimenting with Virtual Memory. Options A through E are five ways to blow up you server and are very convenient for experimenting with Memory Protection.

The following includes two experiments, one with memory protection, and one with virtual memory.

Experiment Memory Protection

Follow these instructions to experiment with NetWare 5's Memory Protection:

  1. On your server console, type Protection.

    Figure 8: Protection Console Command.

    Note that the only Address Space you see is the OS's.

  2. Type Load Protected CLIB on the console. Note the loading of the pertinent CLIB modules into a user address space called ADDRESS_SPACE1. ADDRESS_SPACE1 may be replaced with ADDRESS_SPACEx if you already have a user ADDRESS_SPACE1 on your server. Remember that the NetWare 5 OS will assign the name ADDRESS_SPACEx to your newly created address space if you don't specify a name.

  3. Type Protectionon your server console. Note the new user address space and the CLIB files in it.

    Figure 9: Protected Address Space, ADDRESS_SPACE1.

Next, lets blow up the user address space. First, load BADPROG.NLM. Depending on where you have copied the program, type the following:

Load address space=ADDRESS_SPACE1 <path< badprog.nlm

Note: You cannot load BADPROG and expect CLIB to load automatically when using user address spaces. You must load CLIB first for all CLIB dependent programs, and then load the program.

Figure 10: BADPROG.NLM running in user address space.

Now switch your server screen over to the BADPROG screen. Select one of the five possible methods for abending your server.

Note: Selecting option D (strcpy 13 bytes to an 12 byte block) will not typically abend the server unless the developer options are enabled on the server. On the server console set Developer Option=ON. The reason for this is that the chance of over-writing memory in your user address space by one byte and corrupting something is very remote. The developer option will track variables for you and fault the user address space if a memory overwrite occurs.

Choosing option (A access a null pointer) abends the user address space producing the following on your server console:

Figure 11: Results of abending user address space.

As you can see, the user address space, ADDRESS_SPACE1, was removed because of the memory protection violation introduced by accessing a NULL pointer.

Now lets create a new user address space using CLIB and BADPROG, but lets make it restartable. Additionally, lets name the new address space. Type the following commands on your server console:

Load address space=my_space restart clib

Load address space=my_space badprog

Figure 12: User address space re-start enabled.

As you can see from Figure 12, user address space "MY_SPACE" has been created, and it is restart enabled.

Note: The next step assumes that the server SET parameters:

Memory Protection Restart Count: 1

and

Memory Protection No Restart Interval: 1

are set to their defaults.

Now, switch your server console screen over to BADPROG's screen and blow the user address space up by choosing another option (A-E). This time, when you blow up the user address space, it will automatically re-create the user address space using the same name and load CLIB and BADPROG for you. Typing "Protection" on the server console will verify for you that all has been restored as it was before (Figure 12).

Experiment Virtual Memory

Since NetWare 5's virtual memory can only be accessed by an application through a user address space, we need to create a user address space. Type the following on the server console:

Load address space=vm_test clib

This will create a user address space called vm_test and place a unique CLIB data segment in it. Typing protectionon the server console will verify that this has been created.

Next, load the BADPROG program:

Load address space=vm_test <path< BADPROG

Now go into the NetWare 5 Console Monitor and select System Resources from the Available Options menu.

Figure 13: NetWare 5 Console Monitor

Note the Total server work memory, in bytes, in this case 128MB. This is the physical memory in the server hardware. Also note the Cache buffer memory, in bytes. This is the amount of memory being used by NetWare OS. (Of course, there is other memory being used by the OS and other NLMs, but we will concern ourselves with these two amounts only.)

The object here will be to allocate more memory for the server than is physically available on the server. This will force the use of virtual memory.

Now, switch your server screen to the BADPROG screen. Use option '1' (Figure 7) to allocate 10MB memory chunks. Allocate a good 60MB to 80MB over the physical memory on the system. You can do this by subtracting the cache buffer memory from the server's physical memory and determining the amount needed.

For example, in my case, 128MB - 84MB = 44MB, which is roughly the free memory on the server. So, to go over physical memory, I need to allocate around 120MB to be about 80MB over server physical memory. However, since the NetWare 5 server gives me as many cache buffers as it can, I will allocate a good 200MB of memory. This is roughly 70MB over what the server hardware can provide!

Figure 14: NetWare 5 Console Monitor

Switch your server screen back to the NetWare 5 Console Monitor and look at the Total server work memory, in bytes. In my case, it is still 128MB as expected. Now look at Allocated memory pool, in bytes. It is now 232MB or about 100MB over physical memory. This is the virtual memory stored on hard disc. (You can use the SWAP console command to verify this.)

Once you allocate the memory, use option 3 to reference the memory. You will initially see several 10MB blocks go by very quickly. These are the 10MB blocks that have been stored in server cache. When things slow down, and you hear your server's hard disk grinding away, you know you are in virtual memory.

Note: The size of NetWare 5's virtual memory is only limited by the actual size of the server volume that is hosting the swap file.

Lastly, use option 2 to free the allocated 10MB blocks.

Summary

NetWare 5's memory protection features can be used to load NLM applications on the server, effectively isolating them from each other and protecting the server OS. NetWare 5's virtual memory can be used by NLM applications, in user protected memory, to provide ample memory only limited by the size of the server's volume that hosts the virtual memory swap file. The server OS can also use virtual memory. NLM applications do not even need to know about virtual memory to take advantage of it. When enabled on a NetWare 5 server, its use is automatic.

/**********************************************************

** Copyright (c) 1998, 1999 Novell, Inc.  All Rights 

** Reserved.

**

** THIS WORK IS SUBJECT TO U.S. AND INTERNATIONAL COPYRIGHT 

** LAWS AND TREATIES.  USE AND REDISTRIBUTION OF THIS WORK 

** IS SUBJECT TO THE LICENSE AGREEMENT ACCOMPANYING THE 

** SOFTWARE DEVELOPMENT KIT (SDK) THAT CONTAINS THIS WORK.

** 

** Pursuant to the SDK License Agreement, Novell hereby 

** grants to Developer a royalty-free, non-exclusive 

** license to include the sample code BADPROG.C and 

** derivative binaries in its product.

** Novell grants to Developer worldwide distribution rights 

** to market, distribute or sell the sample code BADPROG.C 

** and derivative binaries as a component of Developer's 

** product(s).  Novell shall have no obligations to 

** Developer or Developer's customers with respect to this 

** code.

** 

** DISCLAIMER:

** 

** Novell, Inc. makes no representations or warranties with 

** respect to the contents or use of this code, and 

** specifically disclaims any express or implied warranties 

** of merchantability or fitness for any particular 

** purpose.  Further, Novell, Inc. reserves the right to 

** revise this publication and to make changes to its 

** content, at any time, without obligation to notify any 

** person or entity of such revisions or changes.

**

** Further, Novell, Inc. makes no representations or 

** warranties with respect to any software, and 

** specifically disclaims any express or implied warranties 

** of merchantability or fitness for any particular

** purpose.  Further, Novell, Inc. reserves the right to 

** make changes to any and all parts of the software, at 

** any time, without obligation to notify any person or 

** entity of such changes.

**

**********************************************************/

#include <stdlib.h<<
#include <stdio.h<<
#include <ctype.h<<
#include <string.h<<
#include <nwthread.h<<
#include <nwconio.h<<


void *blocks[ 5000];

int blockCount = 0;



void blockAlloc( int count)

{  

  int i; 

  for(i = 0; i < count; i++)<
  {

    void *v;

    v = malloc( 1024 * 1024);

    if( v)

    {

      blocks[ blockCount++] = v;

printf( "malloc 1Mb successfully, %iMb allocated so far\n", blockCount);

    }

    else

    {

printf( "failed to allocate a 1Mb block with malloc()\n");

      return;

    }

  }

}



void blockFree( int count)

{

  while( blockCount && count)

  {

    free( blocks[ --blockCount]); count--;

  }

}



void blockUse()

{

  int i, j;

  char *c;

  

  for( i = 0; i < blockCount; i++)<
  {

    printf( "accessing block %i\n", i);

    c = (char *)blocks[ i];

    for(j = 0; j < 1024; j++)<
    {

      memset( &c[ j * 1024], (char)i, 1024);&
    }

  }

}





void blowTheStack()

{

  blowTheStack();

}
int main( int argc, char *argv[])

{

  char c;

  char *x;

  char *fred;

  void (*badfunc)(void);

  

printf("BadProg: A small list of ways to annoy a server...\n\n");



  while(TRUE)

  {

    clrscr();



    printf("0. exit\n\n");

    printf("1. malloc 10Mb memory\n");

    printf("2. free 10Mb memory\n");

    printf("3. reference memory\n\n");

    printf("A. access a null pointer\n");

    printf("B. execute code at null location\n");

    printf("C. delete a pointer twice\n");

    printf("D. strcpy 13 bytes to an 12 byte block\n");

    printf("E. blow the stack\n");





printf("\n%iMb memory currenly allocated\n",    blockCount);



    c = getch();



    if( c == '0')

    {

      return 0;

    }

    else if( c == 'A' || c == 'a')

    {

      c = 0;

      printf( "a null pointer %s", c);

    }

    else if( c == 'B' || c == 'b')

    {

      badfunc = (void *)atoi("0");

      (*badfunc)();

    }

    else if( c == 'C' || c == 'c')

    {

      x = (char *)malloc( 30);

      free( x);

      free( x);

    }

    else if( c == 'D' || c == 'd')

    {

      fred = (char *)malloc( 12);

      strcpy( fred, "SOMENAME.EXE");

      free( fred);

    }

    else if( c == 'E' || c == 'e')
{

      blowTheStack();

    }

    else if( c == '1')

    {

      blockAlloc( 10);

    }

    else if( c == '2')

    {

      blockFree( 10);

    }

    else if( c == '3')

    {

      blockUse();

    }

  }

}

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates