Security With a Smile: A Sneak Peek at a New Novell Press Title
Articles and Tips: article
Jim Henderson
01 Jul 2004
Novell press (www.novellpress.com) is the official book publishing imprint of Novell Inc. We at Novell Connection are excited to give you a sneak peak of the soon-to-be-released Novell's Guide to Troubleshooting eDirectory. Below are excerpts from three chapters of the upcoming book by Peter Kuo and Jim Henderson. If you work with eDirectory, you're sure to find some of the tips that Peter and Jim give useful. We hope you find these excerpts helpful and encourage you to visit www.novellpress.com to get the full text upon its release.
eDirectory Health Checks
For the best performance, all complex machinery and software requires frequent monitoring. For eDirectory to perform optimally, you should maintain the directory by performing routine health check procedures and upgrading or replacing hardware when necessary. This chapter describes how to perform health checks on your eDirectory tree.
Why, When and How Often Should You Do Health Checks?
Do you keep tabs on the noises that your car makes and notes what is normal and what is not? Do you change the oil regularly, check the various fluid levels, and monitor the tires for wear and tear? Of course you do. You know you must take good care of your car, or problems can (and do) happen. If not detected early, these problems can be very expensive to repair in terms of both time and money. Your directory services (DS) tree is no different from your car in this regard.
A network administrator's role is to ensure a healthy and working environment so all users can access shared resources quickly when they need them. Unfortunately, users generally only retain memories of the times when the network is unavailable and not of the times the network is running flawlessly. Therefore, it is your responsibility to make sure your network runs like clockwork, and you should perform regular DS health checks to ensure this. Depending on the environment, the frequency of health checks varies between perhaps once per week to once per quarter. The frequency of health checks depends on whether your tree is static or dynamic. To determine whether you have a dynamic tree or static tree, consider the following definitions:
STATIC DS TREE-- A static tree has minimal routine changes. For example, you only make simple changes such as adding or deleting user objects, or you create a partition or add a server every couple of months. Because you make fewer changes to a static DS tree than to a dynamic one, you only need to perform DS health checks infrequently.
DYNAMIC DS TREE-- A dynamic tree sees frequent non-routine changes. For example, you create a partition or add a server weekly, or you are in the process of developing the tree.
If you have a dynamic DS tree, you should perform a DS health check once per week or whenever a major change is about to be made. However, as the pace of change decreases and the DS tree becomes more static, you can relax a little and perform DS health checks less frequently. Table 1 shows some general guidelines for the frequency of performing DS tree health check tasks.
Hint: Not only should you perform regular health checks as preventive measures, but you should perform a health check on the DS tree before you execute a major DS operation such as moving or deleting large numbers of objects, performing a partition operation, or adding or deleting servers.
Recommended NDS/eDirectory Health Check Task Frequency
|
DS HEALTH CHECK TASK
|
FREQUENCY
|
|
DS versions check |
Monthly or quarterly |
|
Time synchronization check |
Monthly or bimonthly |
|
DS (server-to-server) synchronization status check |
Monthly |
|
Replica synchronization (partition continuity) check |
Monthly |
|
Backlink/external references check |
Monthly |
|
Obituaries check |
Monthly |
|
Replica state check |
Monthly or quarterly |
|
Replica ring consistency check |
Monthly or quarterly |
|
Schema sync status check |
Quarterly |
|
Partition size check |
Quarterly |
|
Renamed and unknown object check |
Monthly or quarterly |
|
Schema consistency check |
Quarterly or after a schema extension operation |
|
Duplicate tree name check |
Monthly or quarterly |
|
DS tunable parameter check |
Quarterly |
|
eDirectory cache statistics check |
Monthly or bimonthly |
Health Check Prerequisites
Before you start a health check, you first need to have a baseline so you know what's already in place in terms of the environment. For instance, suppose your health check results show that there are three Unknown objects in the Users partition; is this an indication of a problem? Generally, Unknown objects are indicative of DS issues. However, if you know that this tree has a mix of NetWare 4.2 servers running NDS 6 and NetWare 5.1 servers running eDirectory 8.7.3 and that auxiliary classes are being used, the presence of Unknown objects could be considered normal (when viewed on a NetWare 4.2 server).
Note: Refer to the Auxiliary Class Object Handling section in Chapter 6, Understanding Common eDirectory Processes, to see why Unknown objects may be normal in an environment where there are mixed versions of NDS and eDirectory.
Recommended Health Check Procedures
The health check process involves a detailed list of tasks for ensuring and maintaining a healthy DS tree. We break down these tasks into three sections:
TREE HEALTH CHECK PROCEDURES-- This section provides information on how to determine the overall health of the tree. These steps should be performed on a fairly regular basis, perhaps monthly.
PARTITION HEALTH CHECK PROCEDURES-- This section provides information on procedures that should be performed before you do any major partition operations.
SERVER HEALTH CHECK PROCEDURES-- This section describes the steps necessary to identify the health of replicas on a server.
The concepts and procedures discussed in this chapter are applicable equally to all operating system platforms supported by eDirectory: NetWare, Windows, Sun Solaris, and other Unix/Linux platforms.
Figure 1: Novell eDirectory Cache Configuration in iMonitor
Note: Although NDS iMonitor makes health checking an easier task, we feel it is important to understand the underlying fundamental steps necessary for a health check. Therefore, the procedures given in the following sections are in the "long format," using traditional tools such as DSRepair and DSTrace, instead of NDS iMonitor. Furthermore, sometimes it is simply much more efficient or convenient (say, when you are standing right in front of the server console) to use non-HTTP--based tools.
eDirectory Management Techniques
Knowing how to properly use the eDirectory management tools is the first step toward understanding strategies for preventing problems. Understanding effective techniques for using these tools is as important as--if not more important than--understanding how the tools themselves work.
This chapter takes a look at some effective techniques for managing single and multiple objects, using the Novell tools described in Chapter 12, eDirectory Management Tools, and a few additional tools available from third-party vendors. After looking at basic techniques for single- and multiple-object modification, this chapter delves into advanced techniques of combining tools. These techniques overlap with some of the techniques presented for recovery in Chapter 10, Programming for eDirectory, because good techniques are effective in both reactive maintenance and preventive maintenance.
Note: Although NDS iMonitor makes health checking an easier task, we feel it is important to understand the underlying fundamental steps necessary for a health check. Therefore, the procedures given in the following sections are in the "long format," using traditional tools such as DSRepair and DSTrace, instead of NDS iMonitor. Furthermore, sometimes it is simply much more efficient or convenient (say, when you are standing right in front of the server console) to use non-HTTP--based tools.
Strategies for Managing eDirectory
The specific strategies used for managing eDirectory may vary from environment to environment; however, any strategy for good management is based on three principles:
Planning ahead
Saving time
Knowing your tools
Planning Ahead
Planning ahead can be a difficult task for many administrators--partly because most work reactively rather than proactively. When reacting to situations on a continual basis, you have a constant drain on your time. This drain results in not spending the time to figure out a better way of doing things. Reacting to different situations on a constant basis also frequently results in having to spend time figuring out how to do the same task each time you do it because you cannot remember how you did it the last time, which may have been six months ago.
A good way to start planning ahead is to spend a little extra time documenting solutions to problems as you go along. Finding the time for documentation is not always an easy task when you're moving from crisis to crisis. Remind yourself--and your management--that documenting your solutions ultimately saves you time and saves the company money.
Note: Many companies have policies that any network disruption that lasts more than a specified time (often 30 minutes) needs to have the "lesson learned"--the cause, resolution, and recommendation on how the same issue may be prevented in the future--documented and shared with affected company divisions.
Documenting changes as they are made is also a good way to save time during the troubleshooting process. By having a record of recent changes made, you may stand a better chance of solving the problem quickly. By documenting changes, you can also start to lay down a framework for standard ways of doing things. Having standards is a good way to meet the second strategy for managing eDirectory: saving time.
Saving Time
By spending a little extra time looking at how certain repetitive tasks are done, you might find ways to reduce the amount of time spent doing them. By shaving a little bit of time off each iteration, you can make yourself more productive--and in many environments, being productive is a key to promotion or to working on other projects.
Let's take a simple example: starting ConsoleOne. On a 2GHz Celeron-based machine running Windows 2000, when launched locally on the workstation ConsoleOne takes about 45 seconds to start, depending on the number of snap-ins to be loaded and the other applications running on the system. If you need to add a user to a group, that operation can take a minute or two--significantly more time than the startup of the utility.
Figure 2: Novell eDirectory Cache Settings in iMonitor
Note: Whenever possible, install a copy of your frequently used administration tools, such as ConsoleOne and NetWare Administrator, locally on your workstation. With the popularity of USB flash drives, you can easily put a copy of your favorite tools (including ConsoleOne, NetWare Administrator, and so on) on one and keep it handy on your key chain.
If you shut down ConsoleOne and have to restart it later to perform another administrative task, you face another repeat of the startup delay. While 45 seconds may not seem like much, it adds up quickly. If you start ConsoleOne an average of 10 times a day, that's over 35 minutes' worth of your time just waiting for the utility to start up over the course of a week. That may not seem like much at first, but if you can find a number of places where you can make small changes, the time adds up. Reducing the time you spend performing repetitive--and frequently boring--tasks gives you time to work on projects you want to be working on.
Coming up with standard ways of doing tasks also makes it possible to train others to do repetitive tasks. If you are a programmer, knowing when you can save time by writing a program--as opposed to using standard tools to complete the task at hand--is important. If you know your programming skills can make shorter work of a repetitive task, spend a little extra time writing the program. Using automated tools--even home-grown tools--can help ensure consistency in how tasks are performed and make your network easier to administer.
Figure 3: Server Information Report Options
Knowing Your Tools
There is nothing worse for a new administrator than the overwhelming task of learning how to effectively use all the tools available. To know when to use ICE instead of ConsoleOne, for example, you need to know the features of both utilities and be able to ascertain when one utility is better than the other.
You should spend time with the different utilities to learn the strengths and weaknesses of each one. What works for you may not work for someone else, but knowledge always works to your advantage, particularly when you're trying to save time.
You should look at older tools if they are available. Novell does not provide the DOS-based NETADMIN utility with NetWare 5 and higher, but a NetWare 4.x server on your network would have a copy of it. NETADMIN has its own features that can prove useful in making lots of changes when ICE or UImport cannot be easily applied (for example, when you're updating console operators on multiple servers or making a quick change to a login script). One limitation of NETADMIN to be aware of is that it does not support extended schema classes and attributes, not to mention some of the newer classes introduced in NetWare 5 and higher, and it definitely does not support auxiliary classes.
You should also spend time with third-party tools. If your company spends money on a management tool, the best return it can get on the investment is realized only if the tool is used effectively.
If possible, you should reuse parts of tools. For example, Chapter 13, eDirectory Health Checks, talks about the product named bv-Control for NDS eDirectory from BindView Corporation. bv-Control is an extremely powerful tool, but using it fully involves reusing reports that you have created or that are part of the standard reports included with the product. Not having to re-create reports that already exist--or modifying existing reports that almost contain the information you need--saves you time. The only way you can do this, though, is by knowing what comes with the product and organizing your reports so you can find them for reuse later.
Similarly, if you create a data file for a mass user modification with ICE or UImport, you should save the control and data files as well as the tools and scripts you used to create the data files. You never know when they might come in handy--particularly in a disaster-recovery situation.
Knowing your tools also involves knowing shortcuts for certain functions. For instance, when using NetWare Administrator, why would you use the mouse to open the Object menu and select Move when you could simply select the object and press the F7 key to accomplish the same task more quickly? Train yourself to use the shortcut keystrokes instead of using the mouse.
Figure 4: Server Information Report Options
Tuning eDirectory
Besides running eDirectory on a fast CPU, using lots of RAM, using fast disk drives in RAID configuration, and so on, there are also a few noteworthy tricks you can perform from a software perspective to enhance the performance of eDirectory without spending additional money. Because the Novell LDAP server uses eDirectory as the back-end database, its performance can be affected by that of eDirectory (for example, memory management [cache settings], indexes, replica placement, and search limits).
Certain portions of eDirectory are already configured to take advantage of the presence of multiple processors in the operating environments. The core directory, security, encryption, and LDAP modules are multiprocessor enabled. The following sections discuss the cache settings and indexes in depth. But first, this chapter takes a brief look at replication latency.
Reducing Replication Latency
eDirectory uses a slow-but-sure convergence algorithm to replicate changes from a replica server to its peers in a replication ring. A replica server can manage only a single Directory Information Base (DIB), but a DIB may contain replicas of multiple partitions. Replication uses a batch update mechanism. The period for which changes are accumulated in a replica server is adjustable, from only a few seconds to a few hours, but it defaults to 30 minutes for NDS 6 and 60 minutes for NDS 7 and higher; this is known as the "heartbeat" interval.
Changes to attributes (such as password) that are flagged as "Sync Immediate" will be scheduled for immediate synchronization. A background thread that would yield or postpone its operation if a request for a Create, Modify, or Delete operation were received handles synchronization operations. This causes a delay, or latency, in replication. Fortunately, in many instances, partitioning can minimize this delay.
You can partition a tree such that update operations are spread across multiple partitions. Placing these volatile partitions on different servers helps to minimize the peak update load because the partitions are now distributed. For example, if a tree has three containers that are volatile (whose subordinate objects undergo modifications frequently), you should isolate each container into a partition and place the partitions on separate servers. The larger the peak update rate, the smaller the replica ring; but bear in mind that a ring should be designed with at least two servers (three servers are recommended) for fault tolerance reasons. If the entire server farm is front-ended by a load-balancing switch, you should configure the switch to direct all requests to the primary servers (ones holding the Master replicas) and fail over to the secondary.
Note: The By Server synchronization method (discussed in the Multi-threaded Synchronization section in Chapter 6, Understanding Common eDirectory Processes) may also help reduce the replication latency by outbounding multiple partitions to multiple unique servers at one time.
Cache Memory Considerations for NDS 7
In versions of directory services (DS) prior to 7.55, any caching done was mainly based on NetWare's caching of the various database files (for example, *.NDS files). However, DS 7.55 and later versions of DS.NLM included the ability of caching DS objects in memory, which increases performance.
Versions 7.55 and greater of DS.NLM will, upon loading, calculate the cache limit based on the amount of free memory (cache buffers) that the server currently has. In these versions, DS determines three values: MaxMemory, which is the amount of free memory at the time; MemoryLimit, which is equal to 20 percent of MaxMemory; and BackoffPoint, which is 80 percent of MemoryLimit. For instance, on a server with 200MB of available memory at DS.NLM load time, MemoryLimit is set to 40MB (20 percent of 200MB) and BackoffPoint is 32MB (80 percent of 40MB).
As objects are being referenced, they are cached in memory. When the cache size reaches the BackoffPoint value, DS.NLM replaces the oldest objects in the cache with new objects that are coming in. Also, a cache cleaning service is scheduled to actually free up the memory within the cache that represents the oldest objects. If memory cannot be allocated for the cache, DS.NLM immediately schedules the cache cleaning service.
How Much Can DS Cache?
A typical DS object is about 4KB in size. Therefore, with a 32MB cache, DS.NLM can cache around 8,000 objects.
When required, you can manually configure DS.NLM for a lower or higher value for MemoryLimit; the BackoffPoint value cannot be changed and will always be 20 percent of whatever value you set MemoryLimit to be. You use the SET DSTRACE command to set the limit, as follows:
SET DSTRACE=!Mvalue_in_MB
Note: You do not need to restart the server for the MemoryLimit setting to take effect.
For example, SET DSTRACE=!M64 will permanently configure DS.NLM (the value is stored in the database and so is persistent between server restarts) to set the MemoryLimit to 64MB instead of the 20 percent of MaxMemory value. Therefore, the value for BackoffPoint will be 80 percent of 64MB, or 51MB. This will give you enough cache for approximately 13,000 objects.
Hint: Do not be concerned if there are 100,000 objects in the tree but your server can only cache 13,000 objects. Remember that the cached objects represent the number of objects being accessed within the same time period (meaning that Objects A and B are both being used within seconds of each other). A good estimate to aim for would be to cache 5 percent of the total objects in the tree. Novell recommends that no more than 40 percent of available memory be used for DS caching if the server is an application server. However, if the server is used for DS exclusively, up to 80 percent may be used.
Cache Memory Considerations for NDS Version 8 Instead of caching the whole NDS object, NDS 8 employed block caching from the beginning. The block (or page) cache is used to increase the performance of reading a block of data from the DS database. Before accessing a block from the database, NDS/eDirectory searches the block cache stored in memory for the requested block of data. If it is found, a cache hit occurs, and the data can be retrieved from memory rather than from disk. If the data is not found in the block cache, a cache fault occurs, and the record must be retrieved from disk. The record is then added to the block cache to prevent subsequent cache faults on the same block of data.
Note: The default database block size is 4KB. However, at times you might find that the block size is 5KB on Windows servers. You can check the block size on your server by selecting Agent Configuration, Database Cache in NDS iMonitor. You can see how to change the database block size in the Changing Database Block Size section, later in this chapter.
You can specify an upper block cache limit to regulate the amount of memory that eDirectory uses for the cache. The default block cache limit is 8MB of RAM. Using the SET DSTRACE command on the NetWare server console can change the hard limit of this cache size. For example, the following command permanently increases the database cache limit to 80MB:
SET DSTRACE=!MB83886080
Warning: You can also use SET DSTRACE=!MKB_in_hex to increase the database cache limit. As discussed in the previous section, in NDS 7, the value for the SET DSTRACE command is in megabytes. In NDS 8, however, the expected value is in kilobytes and in hexadecimal notation. Consequently, administrators who are familiar with NDS 7 frequently make the error of entering the wrong value in NDS 8 and higher environments. For example, when trying to specify a cache limit of 64MB, an administrator might wrongly use the command SET DSTRACE=!M64, which allocates only 100KB of memory for eDirectory caching. (The correct command is SET DSTRACE=!M10000.) Because of this easy confusion, we suggest that you always use the !MB option instead of the !M one.
Instead of using the DSTRACE command, you can manually create a text file named _NDSDB.INI in the SYS:\_NETWARE directory and put in a line like the following:
cache=83886080
Warning: Don't let it float. Be careful that you don't put any spaces around the equal (=)= sign. Spaces prevent the value from being set.
To maximize the amount of memory available for DS, Novell suggests using the following formula to calculate the maximum amount of memory needed:
MemoryForDSDIB = (SizeOfDIBSet + (SizeOfDIBSet x 4))
where SizeOfDIBSet equals the number of megabytes for all NDS.* files found in the DIB directory; this excludes any of the stream files, such as login scripts.
You should check the calculated amount of memory the database might need to see whether it exceeds the Novell-recommended 40 percent limit (for an application server; 80 percent for a dedicated DS server) by dividing MemoryForDSDIB by the total server memory and multiplying that amount by 100. If the result exceeds the limit, you might want to adjust the multiplier of 4 down to 2 (do not go below 2 on this multiplier). If you still exceed the limit, you should either get more memory or you can expect some performance degradation to occur.
Cache Memory Considerations for eDirectory
Entry caching was added starting with NDS 8.73 and for eDirectory 8.5 and later. The entry (or record) cache contains logical entries in the eDirectory tree rather than physical blocks of records from the eDirectory database. While traversing the database, eDirectory searches the entry cache stored in memory for the next requested entry. If the entry is found, a cache hit occurs, and the data can be retrieved from memory rather than from disk. If the data is not found in the entry cache, a cache fault occurs, and the entry record must be retrieved from the block cache (and from disk, if the required block is not in the block cache). The record is then added to the entry cache to prevent subsequent cache faults on the same entry record.
Although there is some redundancy between the block and entry caches, each cache is designed to boost performance for different operations. The block cache is most useful for update operations, whereas the entry cache is most useful for operations that browse the eDirectory tree by reading through entries, such as name resolution operations. Both the block and entry caches are useful in improving query performance. The block cache speeds up index searching, and the entry cache speeds up the retrieval of entries referenced from an index. (eDirectory indexes are discussed later in this chapter.)
Note: eDirectory caches information for the entire block--which may be more than that asked for in the block cache. However, it will cache only the entry information asked for in entry cache. Stream files (such as login scripts) are only cached on demand and are cached in the file system cache and not in the eDirectory cache. For instance, if eDirectory is asked for an octet attribute that is only 4 bytes in size, it will cache the entire block, usually 4KB, in the block cache. However, only the entry husk (which includes the entry's ID), not the entire entry, and attribute asked for are placed in the entry cache. (An entry husk is similar to a pointer in that it carries only enough information to link it to the real object.)
The following is a list of some of the information you should gather before analyzing the health of a DS tree:
The number of DS trees on the network.
The versions of NDS/eDirectory that are running.
The versions and types of operating systems running (such as NetWare and Linux).
The service pack level of each operating system.
Communication protocols used between DS servers.
The network addresses of the DS servers, including internal network addresses of NetWare Servers.
A drawing of the current tree design.
The current replica matrix, showing the number of replicas (including types) and servers holding them.
The time server infrastructure, including placement of time provider groups and any custom time synchronization configuration used.
WAN link information (number of links, speed, reliability, and so on).
Router hardware information, such as model and firmware level.
The number of objects in the tree. This does not necessarily need to be an accurate count, which can change, but it should be within 5 to 10 percent of the actual count.
A list of DS-aware applications, both from Novell and third parties. This provides you with information about possible schema extensions and use of auxiliary classes.
The server hardware configuration, such as the amount of total disk space, free disk space--and RAM. For NetWare servers, you should also note the size and free space on the boot partition.
The NetWare server software configuration--in particular, Transaction Tracking System (TTS) settings.
Titles Now Available From Novell Press
Novell NetWare 6.5 Administrators Handbook
ISBN: 0-7897-2984-9, 800 pages, $49.99
Novell ZENworks for Desktops 4 Administrator's Handbook
ISBN: 0-7897-2985-7, 500 pages, $39.99
Novell ZENworks for Servers 3 Administrator's Handbook
ISBN: 0-7897-2986-5, 512 pages, $39.99
Novell GroupWise 6.5 Administrator's Guide
ISBN: 0-7897-2982-2, 800 pages, $59.99
CNA Study Guide for NetWare 6
ISBN: 0-7897-2980-6, 650 pages, $74.99
Novell GroupWise 6.5 User's Handbook
ISBN: 0-7897-2983-0, 320 pages, $29.99
Novell's CNE Update to NetWare 6 Study Guide
ISBN: 0-7897-2979-2, 576 pages, $69.99
Titles Coming Soon
CNE Study Guide for NetWare 6
ISBN: 0-7897-2988-1, $99.99
Novell's Guide to Troubleshooting eDirectory
ISBN: 0-7897-3146-0, $59.99
Novell ZENworks 6.5 Administrator's Guide
ISBN: 0-7897-3204-1,$59.99
* Originally published in Novell Connection Magazine
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.