LDAP Benchmarking Tips
(Last modified: 14Aug2003)
This document (10079374) is provided subject to the disclaimer at the end of this document.
fact
Novell eDirectory 8.7 for All Platforms
Novell eDirectory 8.6 for All Platforms
LDAP
Third party benchmarking tools (such as Mindcraft's DirectoryMark and iPlanet's SearchRate) are being used to develop a baseline of LDAP performance against eDirectory.
This benchmarking process will be testing multiple types of queries - rather than modifications, adds, etc.
symptom
During this benchmarking process, the test software reports numerous LDAP timeouts.
Benchmarking results show very poor performance of LDAP queries against eDirectory.
cause
The benchmark software was not used properly to obtain vaild benchmarking numbers.
fix
While poor performance can be caused by any number of factors, this solution will describe several key flaws in the design of the benchmarking tests and software which can erroneously cause poor benchmarking numbers. In addition to explaining the cause of poor benchmarking results, this solution will provide a guideline for setting up a benchmarking test which will produce valid and useful performance numbers.
The most important aspect of any type of benchmarking test is to ensure that the benchmarking software is performing the tests which you intend. As an example, Mindcraft's DirectoryMark test software does not benchmark the time it takes for an LDAP server to respond to an LDAP query. It actually benchmarks the amount of time it takes for an LDAP server to send an entire search results list to a client. If you are always searching for the same number of objects, this is a valid test. However, if one search returns one hit, and another returns 10,000 hits, the performance numbers will be dramatically different even if the LDAP server sent the first set of results to the client in the same amount of time.
The iPlanet utility SearchRate performs benchmarking based on the response time of the LDAP server to an LDAP search request. However, this utility will also report timeouts if the search result list is not returned within the specified amount of time. (These timeouts are quite obvious when retrieving a large amount of search results via a query such as "CN=*".) Although the timeout is reported, the performance numbers seem to be representative of the time it took the LDAP server to send the first response to the client.
After determining the type of benchmark test to perform, the next factor is deciding on what kind of query to perform. Most customers are interested in developing an average performance number for their LDAP server. To perform this test, many customers develop a test script that consists of several different types of queries. This type of test script causes performance numbers to be significantly lower than a more structured test. (Response times of 10ms are quickly offset by the 60 second timeouts on very large queries.)
A more accurate test process would be to determine several types of queries which the software will be performing and individually benchmark each type of query. Upon completion of these tests, an average of these numbers can be used to provide a quick look at performance, while detailed results show the performance difference based on the type of query being performed. For example, if the client software performs very precise queries looking for one object, and very broad queries looking for all objects of a certain type, benchmark each query individually. After compiling specific test results, a broader benchmark average of performance can be determined. In the future, any change in performance can be directly related to specific types of queries.
Another factor which greatly affects the performance of LDAP searches is whether or not a search base is specified as part of the search criteria. When eDirectory performs an LDAP search, an appropriate eDirectory Index is chosen for search optimization. This index does take into consideration the full DN of the object in question. As an object is found that meets the search criteria, the full DN is analyzed to determine if the object resides within the search scope of the request. If so, the result is sent to the client. Otherwise, the result is abandoned, and the search continues. With a precise query of CN=Messman, this operation completes quickly and the result set can be quickly analyzed to determine if the object is in the search scope. However, with a very broad search of CN=*, the search process will start at the first object in the index and check its DN to determine if it ma.tches the search scope. The search continues this way until every object in the tree has been checked and the entire result list is sent to the client. This means that if you are searching an OU which only contains 100 users, but 100,000 objects exist in the tree, all 100,000 objects need to be checked in order to send the 100 search hits to the user. To avoid this costly check of every object, be sure to use a query that is as precise as possible. Even specifying the fist character greater increases the efficiency of an LDAP search, especially when using a search base.
With the benchmarking software, and test scripts ready, steps should now be taken to ensure eDirectory is setup properly. One critical area of focus should be eDirectory Indexes configured on the LDAP server. eDirectory Indexes greatly increase the performance of searches. However, overusing indexes can have a detrimental affect on performance. A properly setup eDirectory installation would consist of an index on all attributes used heavily by client processes, and no other user defined indexes. While unnecessary indexes do not have a serious impact on small databases, as the eDirectory database grows in size, the overhead of maintaining unused indexes can seriously degrade performance. Also, LDAP queries against eDirectory never use the Presence type of index. Use Value indexes to fulfill presence queries against attributes.
It is also important to ensure your LDAP performance is not being limited by the eDirectory cache configuration. A poorly devised DS cache setup can dramatically affect LDAP performance numbers. The eDirectory cache is divided into two types of cache Entry and Block cache. The Entry cache is a logical view of the database which is primarily used during searching and reading operations. The Block cache is a physically view of the database which is primarily used during writing operations. It is possible for any one object to reside in both caches at the same time one record which is being read and another record being modified. It is also possible for one record to exist twice in either or both caches. Ultimately, this means that the DS cache size can be up to four times the size of the physical database. As it is unlikely that every record is in both caches twice, it is generally a good rule of thumb to allow DS to have a cache size specified that is 2-3 times the size of the eDirectory database.
If the DS cache size is limited to be smaller than the two times the size of the DS database, eDirectory cannot cache all records in the Entry cache. If broad searches are being performed, a condition called Cache Poisoning can occur. Cache Poisoning occurs when unnecessary data is being written to the cache causing useful records to be removed from the cache. This condition can be so severe that queries never complete, as the cache is constantly being cycled as all objects in the tree are analyzed.
To better understand this condition, consider the following example. With an eDirectory DIB size of 500Meg and a DS cache size of 200Meg, eDirectory uses 100Meg for an Entry cache. With precise queries of (CN=Messman), the query results will be easily stored within the Entry cache. However, if a broad query of (CN=*) with a search base of (OU=Test,O=Novell) is specified, every object in the tree will be analyzed to determine if it exists in the Test OU. As these objects are analyzed, the records are placed in the Entry cache. With only 100Meg available for the Entry cache, not all records can be placed in the cache. As the cache fills, the previous query results are removed from the cache and replaced with other objects in the tree. This cycle continues until the (CN=*) query completes. As the query is executed, the cache no longer contains the results of the previous query. Sh.ould the first query be executed again, the objects will need to be read from disk and re-added to the cache.
There are only two ways to avoid cache poisoning. The first is to ensure the DS cache size is large enough to hold the entire database in each cache. The second resolution to this problem is to ensure the queries being executed are precise enough to not poison the cache with unnecessary data.
Using eDirectory as an LDAP server is a very effective way of delivering data to multiple clients quickly and effectively. Thoroughly investigating and properly implementing this LDAP solution should provide exceptional performance, while failure to follow these guidelines can cause serious performance problems.
.Sample of problem script entries:
A script may have invalid search criteria or search criteria that is not indexed like the examples listed here. By simply fixing the filter and creating an index for description, the benchmark was a huge success at over 300 queries per second on a 300Mhz RS/6000. We do not recommend just creating indexes to appease benchmarking. You should use what your application would really search for so that you have an apples-apples test.
Original Scripts:
# already bound
ldap_search
o=Test
LDAP_SCOPE_SUBTREE
(seeAlso=cn=Marthe) <<<<Notice the attribute name is meshed in with cn and will cause timeouts<<<<<
objectClass,cn,sn,description,facsimileTelephoneNumber,l,postalAddress,telephoneNumber,title
The other script contained the following:
# already bound
ldap_search
o=Test
LDAP_SCOPE_SUBTREE
(description=Leontyne_Sweeney)<<< Description was not indexed and caused long delays, negating the overall benchmark<<<<<<<<<<
objectClass,cn,sn,description,facsimileTelephoneNumber,l,postalAddress,telephoneNumber,title
0
Modified Scripts:
# already bound
ldap_search
o=Test
LDAP_SCOPE_SUBTREE
(cn=Marthe)
objectClass,cn,sn,description,facsimileTelephoneNumber,l,postalAddress,telephoneNumber,title
The other script contained the following:
# already bound
ldap_search
o=Test
LDAP_SCOPE_SUBTREE
(description=Leontyne_Sweeney)<<< in this case, a Description index was created and the pause was resolved. <<<<<<<<
objectClass,cn,sn,description,facsimileTelephoneNumber,l,postalAddress,telephoneNumber,title
0
document
| Document Title: | LDAP Benchmarking Tips |
| Document ID: | 10079374 |
| Solution ID: | NOVL86353 |
| Creation Date: | 24Jan2003 |
| Modified Date: | 14Aug2003 |
| Novell Product Class: | NetWare Novell eDirectory |
disclaimer
The Origin of this information may be internal or external to Novell. Novell makes all reasonable efforts to verify this information. However, the information provided in this document is for your information only. Novell makes no explicit or implied claims to the validity of this information.
Any trademarks referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.