Bulk-load Potential of eDirectory

Articles and Tips: qna

01 May 2002

We have a very large customer who is planning on purchasing more than 60 million seats of eDirectory. They recently have been doing product comparisons however, and found that they can bulk-load users exponentially faster using Oracle Internet Directory.

There are people with a lot of political power in the customer's organization who would lean away from eDirectory, and are using this bulk loading issue to their advantage. Does anyone have any documentation that compares eDirectory/NDS to OID, or any helpful information on what OID is doing (or not doing) during the bulk-load (we'll compare this to the processes that occur during NDS bulk loading)?

Cheering for eDirectory!

Dear Cheering: Having performed numerous eCommerce directories. I can honestly say the bulkloading operation is the least significant part of the deployment, but the most commonly debated. If you are planning to reinstall your directory every day, then maybe it would be something to talk about.

As far as disaster recovery is concerned, I have some better options than a reload of the data, such as:

Replication
DIB restore
Disaster recovery tree via DirXML

Typically, things like indexing, password generation, and schema checking slow down bulkload times. You can decrease the bulkload time by moving some of the indexes offline, especially the substring ones, along with adding the passwords in a second LDIF.

You could adjust the block/record cache ratios and/or cache percentage to the hard limit. You could use a mirror instead of RAID5. Every OS is different as far as file system tuning goes.

You might also try breaking up your bulkload file so you can bulkload to multiple servers at once. Replication is a beautiful thing. A friend of mine and I bulkloaded 999,900 objects in 1 hour 46 minutes during a shoot-out in Paris a few years ago on NetWare. We created a custom object class and adjusted the transaction size, which you can still do with the command line ICE.

Also, if you take a look under the covers, one of the reasons that OID may be faster is they are a DATABASE. Have them perform a real study. Get the data in there and then compare access speeds. There will be a remarkable difference.

Remember, databases are great for writes and not so great for reads. There is a great article in the November 2001 AppNotes that covers this comparison. See: http://support.novell.com/techcenter/articles/ana20011101.html

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.