Novell is now a part of Micro Focus

How to Maintain Caching in eDirectory 8 and eDirectory 8.5

Articles and Tips: article

Novell Technical Services
Compiled by Scott Ivie and Shawn Pierson; technical review by Daniel Sanders, Tom Doman, Steve McLain, and Duane Buss
>Worldwide Support Services
Novell, Inc.
support@novell.com

01 Apr 2001


As Novell eDirectory implementations grow larger, it is important not to sacrifice performance to gain new functionality. Caching is a critical feature of eDirectory that should be tuned and maintained for optimal system performance. This AppNote discusses the caching implementation and offers maintenance tips for eDirectory 8 and eDirectory 8.5.

Understanding Caching in eDirectory 8 and eDirectory 8.5

With eDirectory 8.5, two types of caching have been implemented:

  • Block (or page) Caching

  • Entry Caching (sometimes referred to as record cache)

Note: DS build versions previous to eDirectory 8 build version 8.73 only support block caching.

Physical block or block caching is typically implemented by database systems. eDirectory 8 has employed this type of caching from its introduction. Before reading a block (or page) from the database, eDirectory will search its block cache for the needed information.

Entry caching was introduced in eDirectory 8 with DS build version 8.73 and above, and with the initial release of eDirectory 8.5. The entry cache contains logical entries in the Directory, as opposed to physical blocks. Caching of entries reduces the processing time required to instantiate entries in memory from the block cache. Before accessing block cache or the disk to retrieve a particular entry, eDirectory will first search its entry cache.

Block and entry caches have a certain amount of redundancy, but each is designed to boost performance for different operations. Block cache is typically used for update operations, while entry cache is most useful for operations that tend to browse and traverse the tree by reading through entries. Both cache types work to provide optimal performance for information retrieval and updates. For example, block cache is helpful when it comes to searching indexes, whereas the entry cache helps when it comes to retrieving entries referenced from an index.

How Much Memory Does eDirectory Need?

By default, eDirectory will split the allocated "NDS Cache" equally across the block and entry caches.

In light of the added support for entry caching, cache settings for DS build versions previous to 8.73 may not be sufficient with new eDirectory 8.5 versions. For example, if DS build version 8.73 performed correctly with only 50 MB of cache prior to an upgrade to eDirectory 8.5, you may find that performance has decreased after the upgrade. This is because only 25 MB is now allowed for block caching, as opposed to the 50 MB before upgrading. Likewise, the entry caching may be insufficient with only 25 MB allocated to it.

To ensure optimal performance, you should configure your system based on cache fault indicators, database size, and memory available on the server. Don't expect to see zero cache faults and don't assume that it has to be at zero to have optimal performance in eDirectory. Nevertheless, try to reduce cache faults when the number of faults per unit of time is high

Note: This is a very broad, general rule of thumb. It could vary significantly from eDirectory implementation to implementation.

Based on this information, you could use the following formula to calculate the amount of memory needed:

((SizeOfDIBSet + (SizeOfDIBSet * 4)) = MemoryForNDSDIB

where SizeOfDIBSet equals the bytes for all files that start with NDS in the DIB directory (for NetWare, this is the SYS:_NETWARE directory).

Once you have figured an amount of memory the database may need, you can check to see if it exceeds the 60% limit by dividing MemoryForNDSDIB by the Total Server Memory and multiplying that amount by 100.

If you do exceed the limit, you may want to adjust the multiplier of 4 down to 2 (do not go below 2 on this multiplier). If you still exceed the limit, either install more memory or prepare for potential performance degradation.

Monitoring and Configuring Cache for eDirectory 8

With eDirectory 8 (not to be confused with eDirectory 8.5), the administrator can regulate the amount of memory NDS will consume for cache by specifying a cache limit. If no limit is explicitly set, eDirectory will use a default of 8 MB. To find out what the current cache settings are, an administrator can turn on DSTRACE at the console and set DSTRACE to *P.

For example, to view the cache settings on a NetWare server, you would type following commands at the console:

SET DSTRACE=ON SET DSTRACE=P*

Then toggle over the Directory Services screen and you should see a display similar to the following:

TUNABLE PARAMETER VALUES ServerStateUpThreshold = 30 minutes External Reference Life Span = 193 hours JanitorInterval = 2 minutes FlatCleaningInterval = 720 minutes BacklinkInterval = 780 minutes Heartbeat Data = 60 minutes Heartbeat Schema = 240 minutes Schema synchronization enabled = 1 SMI Max Cache = 60000000 (Allocated = 5283880, Blks In Use = 0) SMI Entries Cached Per Thread = 50 (Cached: 3, ~= 6912 bytes) SMI Attr Recs Cached Per Thread = 20 (Cached: 0, ~= bytes) SMI Partitions Cached Per Thread = 20 (Cached: 5) SMI Force Checkpoint Interval = 180 (cannot change) SMI Maximum Read Transaction Seconds = 2400 (cannot change) SMI Maximum Read Transaction Inactive Seconds = 30 (cannot change)

The "SMI Max Cache" line (bolded above) represents the maximum amount of the cache allocated to eDirectory. In this example, it is currently set to 60,000,000 bytes or about 60 MB.

If you want to change the cache hard limit for eDirectory 8 (again, do not confuse this with eDirectory 8.5), you can type the following command to set a hard maximum cache amount on a NetWare server:

SET DSTRACE=!MB<amount_of_RAM_to_use_in_bytes<

For example, to set a hard limit of 8 MB, you would type the following command at the NetWare console:

SET DSTRACE=!MB8388608

The number shown is in decimal. The command will set a persistent setting in a file called _NDSDB.INI found in the SYS:\_NETWARE directory on a NetWare server.

The following represents the cache parameter that would be set in the _NDSDB.INI file when using the SET command above for 8 MB:

cache=8388608

Monitoring and Configuring Cache for eDirectory 8.5

eDirectory 8.5 provides tools for monitoring how well cache is being utilized. The iMonitor utility (released with eDirectory 8.5) allows an administrator to monitor cache hits and cache faults for both the block cache and entry cache. A cache hit occurs whenever eDirectory finds an item in its cache. A cache fault occurs whenever an item is not found in cache and must be fetched from disk. If there are lots of cache faults, increasing the amount of cache memory may improve performance. The ideal system would be one in which there were no cache faults; however, this is possible only if eDirectory is able to cache the entire database.

With the introduction of eDirectory 8.5, two types of mechanisms for controlling cache memory consumption are provided. Each is mutually exclusive of the other. These mechanisms are:

  • Dynamically Adjusting Limit

  • Hard Memory Limit

The administrator can use either one at any time, but the last one used always replaces any prior settings.

Dynamically Adjusting Limit

The first mechanism that is provided for regulating memory consumption is a dynamically adjusting limit. A dynamically adjusting limit causes eDirectory to periodically adjust its memory consumption in response to the needed memory for caching and flow of memory consumption by other processes. The user specifies the limit as a percentage of available physical memory. Using this percentage, eDirectory recalculates a new memory limit at fixed intervals. The new limit is simply the percentage of physical memory that is available at that point in time.

This limit is used to control cache memory consumption within eDirectory until the next calculation occurs. Thus, eDirectory's responsiveness in adjusting its memory consumption depends on the length of the recalculation interval. The shorter the interval, the more responsive to change. The longer the interval, the less responsive to change. Shorter intervals are not necessarily better, because they will likely cause more memory allocation and freeing to occur.

In connection with the percentage, the user can also set minimum and maximum thresholds.The minimum is the lowest number of bytes that eDirectory will adjust down to. If not set, it defaults to 16 MB. The maximum threshold determines the highest number of bytes that eDirectory will adjust up to. It can be set to either a number of bytes or number of bytes that eDirectory will leave available. If the maximum threshold has not been set using one of these two mechanisms, a default maximum of 4 GB is used.

Note: If a minimum threshold is set and a maximum threshold is set as a number of bytes to leave, it is possible that the calculated maximum will fall below the minimum. In this case, the minimum takes priority over the maximum. For example, suppose you have following settings: Minimum threshold: 8 MB Percentage: 75 Maximum threshold: leave 10 MB Suppose that there is 16 MB of available physical memory when eDirectory decides to adjust its cache limit. eDirectory would calculate a new limit of 12 MB. It would then check to see if this falls outside the range of the minimum and maximum thresholds. In this case, the maximum threshold is a calculated value: 16 MB minus 10 MB yields a maximum threshold of 6 MB. The calculated limit of 12 MB is greater than this, so eDirectory would set its limit to 6 MB. eDirectory would then check this limit against the minimum threshold. Since 6 MB is less than 8 MB, eDirectory would set its final limit to 8 MB. Thus, the minimum gets priority over the maximum whenever there is a conflict.

Hard Memory Limit

The second mechanism that is provided for regulating memory consumption is the hard memory limit. This is basically the same mechanism that exists in prior versions of eDirectory. Once a limit like this is set, it is not changed until the user either sets a different hard memory limit or sets a dynamically adjusting limit. Remember that the hard memory limit and dynamically adjusting limits are mutually exclusive. The last setting is what will apply.

An administrator is allowed to specify a hard memory limit in one of three ways:

  • A fixed number of bytes

  • Percentage of total physical memory

  • Percentage of available physical memory

When a hard memory limit is specified by percentages, it is always translated to a fixed number of bytes.

Default Mechanism for Regulating Memory Consumption

If the server contains a replica, eDirectory will use a dynamically adjusting limit of 51% of available memory, with a minimum of 8 MB and a minimum to leave 24 MB. Otherwise, eDirectory will use a hard memory limit setting of 16 MB.

Settings in the _NDSDB.INI File

At startup, eDirectory looks for the database options file, _NDSDB.INI, in the directory where DIB files stored. On NetWare servers it is SYS:\_NETWARE. On Windows NT/2000 servers, this is normally \NOVELL\NDS\DIBFILES. This file is a simple text file that can be created or modified with any text editor. If you are using DSTRACE to define your cache settings, they will automatically be populated in this file.

Note: Although a user may alter the _NDSDB.INI file at any time, the changes will not take effect until the eDirectory module has been restarted. If the change is done via the DSTRACE command, no eDirectory module restart is needed.

Below are the command syntaxes for controlling cache memory consumption within eDirectory. First, here is the "old-style" command for setting a hard memory limit:

cache=<cacheBytes<

The new syntax for setting either a hard memory limit or a dynamically adjusting limit is as follows:

cache=<cache options<

The cache options are listed in the table below. Multiple options may be specified, in any order, separated by commas. All are optional.


Option
Description

DYN or HARD

Dynamically adjusting limit or hard limit

%: <percentage<

Percentage of available or physical memory to use

AVAIL or TOTAL

Percentage is for available memory or total physical memory (Note: This is ignored for a dynamically adjusting limit, because dynamically adjusting limits are always calculated based on the available physical memory.)

MIN: <bytes<

Minimum number of bytes

MAX: <bytes<

Maximum number of bytes

LEAVE: <bytes<

Minimum number of bytes to leave

Here are some example settings. All settings must be in lowercase!

To set a dynamically adjusting limit of 60% of available memory, with a minimum of 16 MB:

To use the "old-style" method to set a hard memory limit of 8 MB:

cache=8000000

cache=DYN,%:60,MIN:16000000

To set a hard memory limit of 60% of total physical memory, with a minimum of 16 MB:

cache=HARD,%:75,MIN:16000000

In addition to the cache setting for regulating memory consumption, eDirectory also provides settings for controlling the dynamic adjust interval and the interval for cleaning up older version of entries and blocks. These are as follows:

cacheadjustinterval=<seconds<<cachecleanupinterval=<seconds<

The default for both of these commands is 15 seconds, if not set otherwise in the _NDSDB.INI file.

Finally, the following setting allows an administrator to control the percentage split between entry and block cache:

blockcachepercent=<percent<

The range for this setting is between 0 and 100 (inclusive). A value of 70 means that 70 percent of cache memory will used for block cache and 30 percent for entry cache. The default is 50 percent if not set otherwise in the _NDSDB.INI file.

Note: Setting the block cache percent to zero is never a good idea.

DSTRACE Options for Regulating Cache

From the console screen in NetWare, a number of DSTRACE commands can be used to regulate cache memory consumption within eDirectory.

The following is the syntax for setting a simple hard memory limit:

SET DSTRACE=!MB<bytes<

Here is an example of how to a simple hard memory limit:

SET DSTRACE=!MB8388608

The following is the syntax for setting a calculated hard memory limit:

SET DSTRACE=!MHARD[,AVAIL|TOTAL][,%:|percent|]|[,MIN:|minimum_bytes|][,MAX:|maximum_bytes|]|[,LEAVE:|min_bytes_to_leave|][,NOSAVE

Here is an example of how to set a hard memory limit of 75% of total physical memory, with a minimum of 16 MB, and specify not to save these options to the _NDSDB.INI file:

SET DSTRACE=!MHARD,%75,MIN:16777216,NOSAVE

The following is the syntax for setting a dynamically adjusting limit:

SET DSTRACE=!MDYN,%:<percent<[,MIN:<minimum_bytes<] [,MAX:<maximum_bytes<][,LEAVE:<min_bytes_ to_leave<][,NOSAVE]

Here is an example of how to set a dynamic limit of 75% of available memory, with a minimum of 8 MB:

SET DSTRACE=!MDYN,%:75,MIN:8388608

Note: Configuring cache via DSTRACE does not require a reset of DS or the server. Changes go into effect immediately.

Using iMonitor to Monitor and Configure Cache Settings

The best way to monitor and configure cache settings is to use iMonitor, which was released with eDirectory 8.5. Using iMonitor allows you to intuitively view information about the current cache settings, look at the statistics to determine possible adjustments, and make the appropriate settings. Best of all, this is done through an Internet browser for any operating system on the back end (NetWare, Windows NT/2000, or UNIX/Linux).

Your browser should open up iMonitor if the NDSIMON.NLM is loaded and running on the eDirectory 8.5 server. To start iMonitor, go to the ConsoleOne utility and right-click on the server you want to work with. From the resulting menu, select "Launch iMonitor," as shown in Figure 1.

Launching the iMonitor utility from ConsoleOne.

Click the Login button at the top left of your browser window, as shown in Figure 2. This will allow you to access the cache settings.

Logging in allows you to access the cache settings.

Figure 3 displays the Agent Configuration button. Clicking on this button will show you the Agent Configuration screen.

The Agent Configuration button.

In the Settings options in the left frame, you can now see the Database Cache selection as illustrated in Figure 4. Click on this selection to go the screen where you can monitor and make changes to the cache settings for this server.

The Database Cache selection.

The screen capture in Figure 5 shows the "Database Information" and "Database Cache" statistics that are found in the Agent Configuration option of iMonitor.

The Database Cache Configuration screen.

This page allows you to configure and monitor the cache segments being used by the NDS database subsystem. This covers another set of the old "!" (bang) commands created for this purpose. (iMonitor"s online context-sensitive help contains a list of which "!" commands correspond to which iMonitor options.)

The information provided on this page will help you determine whether you have an adequate cache size and, if not, help give you an idea how much you should have. For most applications of the Directory, the default behavior and cache settings will be adequate.

The Database Cache page is divided into three sections: Database Information, Database Cache Information, and Database Cache Configuration. We'll first describe each item in each section, followed by a discussion on how to use that information.

DIB Size (KB). The size in KB of the NDS database and associated files on disk. Stream files are part of the associated files but are never cached. These should be subtracted from the DIB size to compute the size of the NDS database.

DIB Block Size (KB). The size in KB of one database block on disk.

Maximum Size (KB). The maximum size in KB to which the specified cache is allowed to grow.

Current Size (KB). The current size in KB of the specified cache.

Items Cached. The number of items in the specified cache.

Old Versions Cached. The number of old versions in the specified cache. Without getting too technical, old versions of cache items are kept to maintain the consistency of read transactions in the database. In other words, if one thread is in a read transaction and another is in a write transaction, old versions of blocks modified by the writer are maintained on behalf of the reader. This is done so the reader s results are guaranteed to produce a consistent view during the life of its transaction even though modifications are taking place during that time.

Old Versions Size (KB). The size in KB of the old version items cached.

Hits. The number of times an item was successfully accessed from the specified cache.

Hit Looks. The number of items looked at in the cache before an item was successfully accessed from the specified cache. The hit look to hit ratio is a measure of cache lookup efficiency. Normally, the ratio should be quite close to 1:1.

Faults. The number of times an item was not found in the specified cache and had to be obtained from a lower level cache or from the disk.

Fault Looks. The number of items looked at in the cache before it was determined that the desired item was not in the specified cache. The fault look to fault ratio is a measure of cache lookup efficiency. Normally, the ratio should be quite close to 1:1.

The Database Cache Configuration Screen

The "Database Cache Configuration" screen found in iMonitor allows you to easily configure the eDirectory cache for a server. In fact, this is the preferred method to configure your cache. It allows you to adjust the cache for whatever server platform you are running, and it prevents you from making mistakes related to not being able to remember all the commands needed for adjusting the cache (for example, the DSTRACE functions under NetWare).

The Database Cache Configuration screen is shown in Figure 6.

The Database Cache Configuration screen.

As you can see, this screen has two radio buttons for the two methods of specifying how much cache to allow the NDS database to use: Dynamic Adjust and Hard Limit. They are mutually exclusive settings.

Following is an explanation of each of the settings on the Database Cache Configuration screen.

Dynamic Adjust. This setting allows the NDS database to dynamically adjust the amount of system memory to be used for the cache, based on the amount it thinks it needs and the parameters specified below.

Cache Adjust Percentage. This specifies the percentage of available memory allowed to be used for the record and block caches combined.

Cache Size Constraints. While dynamically adjusting, NDS will follow the constraints specified here: namely, use no less than the specified amount of memory for the cache, and no more than the total amount of available memory minus the specified amount.

Hard Limit. This setting allows the NDS administrator to specify the exact amount of system memory to be use for the cache.

Cache Maximum Size. This is the size in KB of the record and block caches combined.

Block Cache Percentage. This is the percentage of the system memory available for caching that should be allocated to the block cache. The remaining percentage will be allocated to the record cache.

Cache Adjust Interval. This interval only applies when Dynamic Adjust is set. It controls how often the cache size is adjusted based on the specified percentage and constraints.

Cache Cleanup Interval. This setting controls how often unused old versions are removed from the cache.

Cache Settings Permanent. When checked, any changes submitted via iMonitor will be made permanent overwriting any previously saved settings or system defaults.

If no cache settings have been previously specified as permanent, when the eDirectory 8.5 agent starts up, the cache is defaulted to a hard limit of 16 MB for the first 10 minutes. Since the eDirectory agent usually loads when a server is rebooted, this default behavior allows other applications the opportunity to load and request system resources first. After 10 minutes, the default behavior is to switch to Dynamic Adjust mode based on the amount of available memory.

The default values for Dynamic Adjust mode are as follows:

Cache Adjust Percentage = 51% CacheSizeConstraints = 16MB (Total Available Memory - 24 MB)

For both Dynamic Adjust and Hard Limit, the default Block Cache Percentage = 50% and the default Cache Adjust and Cache Cleanup Intervals are 15 seconds.

Conclusion

This AppNote has explained how caching works in eDirectory 8 and eDirectory 8.5. It has also presented some maintenance and tuning tips for optimizing the performance of eDirectory for various situations.

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates