Using the Novell Import Convert Export Utility

Articles and Tips: article

Nancy McLain
Senior Editor
DeveloperNet University
nmclain@novell.com

Roger Harrison
Senior Software Engineer
Network and Directory Services
roger_harrison@novell.com

01 Oct 2000

With the release of Novell Directory Services (NDS) eDirectory 8.5, Novell is including the Novell Import Convert Export Utility. This utility assists you in importing data into an LDAP directory, modifying objects that exist in an LDAP directory, exporting data from an LDAP directory, and migrating data between LDAP directories. This AppNote provides an overview of this new utility and gives examples and tips for using it to import and export directory information.

• Introduction
• Novell Import Convert Export Utility Architecture
• Example Imports and Exports
• Tips and Tricks for Using the Novell Import Convert Export Utility
• Debugging Your LDIF Files
• Conclusion

Introduction

As its name implies, the Novell Import Convert Export Utility allows you import data into and export data from directories. It can be used to import LDIF (LDAP Data Interchange Format) data into an LDAP directory, to export data from an LDAP directory to an LDIF file, and to migrate data between two directories without using an intermediate file format. Because the Novell Import Convert Export Utility is designed to efficiently import LDIF data into an LDAP directory it is especially well-suited for performing bulk loads of directory data. Its user interface is designed to simplify the process of importing, exporting, and migrating directory data. You can also use the utility to perform tasks such as schema mapping, entry creation, entry modification, schema modification, and LDIF file format checking.

The Novell Import Convert Export Utility is included with NDS eDirectory 8.5. It has both a graphical user interface (GUI) and a command-line interface. It is available on all platforms that run NDS eDirectory 8.5 or its associated ConsoleOne including:

• Win32 (Windows 2000, Windows NT, Windows 98 and Windows 95)

• NetWare

• Solaris

• Linux

The GUI version of the utility is a ConsoleOne snap-in wizard called the NDS Import/Export wizard. You can run it by clicking Wizards-->NDS Import/Export in ConsoleOne.

The command line version of the utility is named "ICE" (an acronym for Import Convert Export). The executable name follows the naming conventions for executables on the platform where it is running, so on Win32 platforms, its full name is ICE.EXE, on NetWare, its name is ICE.NLM, and on Unix platforms, its name is ice.

As of July 2000, the command line version of the Novell Import Convert Export Utility is also included in the LDAP Libraries for C SDK in the Novell Developer Kit (NDK) available at http://developer.novell.com. This provides a convenient way for developers to accomplish tasks such as updating the schema or creating directory entries during their application's installation. Since these tasks are usually automated as part of the install process, the ConsoleOne wizard snap-in is not included with the LDAP Libraries for C SDK.

Novell Import Convert Export Utility Architecture

The Novell Import Convert Export Utility uses a client/server architecture to allow administrators to perform operations from a remote client machine rather than having to perform them directly on a server running NDS. Unlike similar utilities that currently exist, it is extremely efficient when running in client/server mode, which makes it suitable for doing large bulkloads of data even from a remote client. You can also run the utility directly on an NDS server.

The heart of the Novell Import Convert Export Utility is the Import Convert Export (ICE) engine. This engine is the central hub that handles the data transfer and processing. It orchestrates the interaction between a source handler and a destination handler (see Figure 1). The source handler reads data from some source and sends it to the engine. The destination handler receives data from the engine and writes it to a destination. In Figure 1, the data flow is from the upper left to lower right.

Figure 1: The basic architecture of the Novell Import Convert Export Utility.

The architecture of the Novell Import Convert Export Utility allows handlers to be added to support new types of sources and destinations. Currently, the Novell Import Convert Export Utility has source and destination handlers for LDIF files and LDAP directories. In the future, Novell will likely ship handlers for additional sources and destinations.

The processing of the ICE engine is record oriented. Source handlers read data and send it to the ICE engine a record at a time. The ICE engine processes each record based on XML processing rules, if any are defined, and then sends each record to the specified destination handler for writing. We'll talk more about XML processing rules later in this AppNote.

The LDIF Source Handler

The LDIF source handler reads records from an LDIF file. It supports all RFC 2849 conformant LDIF files including files with either content or change type records. Because the LDIF source handler validates the syntax of LDIF files as it reads them, you can also use the LDIF source handler to check the syntax of LDIF files (as explained later in this AppNote).

A tutorial on LDIF is beyond the scope of this AppNote. LDIF is defined in RFC 2849 (http://www.ietf.org/rfc/rfc2849.txt ). Additional information on the LDIF definition and writing conformant LDIF files can be found in several of the resources listed at the conclusion of this AppNote.

The LDIF Destination Handler

The LDIF destination handler writes data to an LDIF file in changetype record format.

The LDAP Source Handler

The LDAP source handler reads entries from an LDAP server using the LDAP search operation. The LDAP source handler allows you to specify the following LDAP search parameters:

• An LDAP server (IP address or DNS name) and port

• An RFC 2252 compliant filter

• A list of attributes to be returned

• A base DN

• A search scope

• A flag to control alias dereferencing behavior

• Size and time limits

Although you can control any of the above parameters, you don't have to specify all of them because the LDAP source handler provides typical defaults for most of the parameters.

The LDAP Destination Handler

The LDAP destination handler allows you to send LDAP update operations to an LDAP server. The LDAP destination handler performs all of the RFC 2251 LDAP update operations on an LDAP server, which are:

• Add

• Delete

• Modify

• Moddn (rename or move)

LBURP and Asynchronous Updating. The LDAP destination handler supports the LDAP Bulk Update/Replication Protocol (LBURP). This protocol is extremely efficient: it blasts data from client to server very quickly, as if it were "burping" data across the wire.

Typically, when you perform several LDAP update operations, the operations are sent as individual synchronous requests. For each operation, the client sends a request to the server and waits for the server to process it and respond before sending the next request. This is done to ensure that the updates are applied by the server in the correct order since two or more changes to the same entry could have a very different end result if they are applied in the wrong order. While this synchronous approach to performing updates is safe, it is also slow because the server must wait for the client to send each update operation.

LBURP allows a highly efficient transfer of updates between client and server because updates can be sent asynchronously and still be kept in the proper order by the server. The asynchronous updating method allows the client to "blast" the data at the server. The server picks it off the wire as fast as it can, processes the data, sends the response to the client, and starts on another piece of data without waiting for the packet's round trip between the client and the server.

Forward References. The LDAP destination handler also supports the use of forward references. A forward reference is a temporary placeholder the server creates for an entry that would normally need to exist before an update operation could succeed. Also, unlike the creation of other entries, when the server receives a command to create an entry that already exists as a forward reference, it transforms the forward reference into a real entry rather than returning an error that the entry already exists.

For example, suppose you're creating the user cn=john,o=Novell, but o=Novell doesn't yet exist. Usually, this causes the add operation to fail because you can't create an entry in a non-existent container. But if your server supports forward references and the LDAP destination handler has enabled their use, the server creates the o=Novell entry as a forward reference. This means the server has a temporary placeholder that allows it to create cn=john,o=Novell. Later, if you try to create the o=Novell container, the server transforms the forward reference for o=Novell into a real entry.

NMAS Simple Password Storage. The LDAP destination handler supports the NMAS (Novell Modular Authentication Services) simple password method for storing passwords. This allows you to add entries with password information without having to expend the processing overhead to generate key pairs for them as you typically have to do in NDS. The password values you provide are stored safely in NMAS' secret store within NDS.

Because NMAS supports several hash formats, you can migrate hashed password values from other directories, such as Netscape. We'll discuss this at more length later in this AppNote.

Error Logging

The ICE engine includes an LDIF error logging facility. This facility allows you to write all records that fail to an output file in LDIF format. You can use this information to help debug import or export sessions. It can also help you track down records that have something wrong with them and fix them.

XML Processing Rules

The ICE engine supports XML processing rules to modify records as they pass through the ICE engine. It supports the same creation, placement and schema mapping rules used in DirXML. We'll discuss XML processing rules at more length later in this AppNote.

Example Imports and Exports

The Novell Import Convert Export Utility makes importing and exporting directory data straightforward and easy. This section contains command line examples of three common tasks--importing LDIF files, exporting LDIF files, and performing data migration between LDAP servers--that you can perform using the utility. The ConsoleOne NDS Import/Export wizard is task oriented and guides you through each of the tasks discussed in this section, however you may find the descriptions of how the ICE engine performs each of these tasks useful as you use the wizard.

Command Line Rules

As mentioned earlier, the name of the command line executable is ICE, which is an acronym for Import Convert Export. For Win32 and NetWare, the ICE executable is installed as part of ConsoleOne and is located in the ConsoleOne/bin directory. On Unix platforms, the ICE executable is installed in the /usr/bin directory.

The format of the command line is:

ice <general options<-S<source handler< <source options<-D<destination handler< <destination options<

Before launching into command line examples, it is important for you to know several rules about the command line for the Novell Import Convert Export Utility:

• All command line options are case sensitive.

• General options must come before any source or destination options (that is, all general options must come between "ice" and the first -S or -D option.

• The source (-S) and destination (-D) sections may come in any order, but the options for each section must be kept within that section to unambiguously identify them with the proper section. Thus, these two command lines are equivalent (assuming that <gen opts<, <src opts<, and <dst ops< are constant):

• ice <gen opts< -S <src opts< -D <dst opts<ice <gen opts< -D <dst opts< -S <src opts<

• The options within each section can be in any order as long as they remain within their corresponding section.

An exhaustive discussion of all ICE command line options is beyond the scope of this document. The user documentation for the Novell Import Convert Export Utility contains comprehensive documentation about each of the available options and their usage. You can also get abbreviated help for all ICE command line options by typing ice -h at the command line.

Importing LDIF Files to an LDAP Server

To import an LDIF file, you tell the ICE engine to use an LDIF file source handler and an LDAP directory destination handler. In other words, you tell the ICE engine to read an LDIF file and send it to an LDAP destination. Figure 2 illustrates this process.

Figure 2: Importing an LDIF file to an LDAP server.

If you were to use the utility's command line interface for this import, the command would look something like this (entered all on one line):

ice -SLDIF -fc:\temp\MyFile.ldif -DLDAP -sserver2.acme.com-p389 -dcn=admin,c=us -wsecret

The parameters are as follows:

-S	is the source handler type. In this case, the value of LDIF tells the ICE engine that the source is an LDIF file.
-f	is the name of your LDIF file. In this case the file name is c:\temp\MyFile.ldif.
-D	is the destination handler type. In this case, the value is LDAP, which tells you the ICE engine that the destination is an LDAP server.
-s	is the DNS name or IP address of an LDAP server. In this example, the server is server2.acme.com.
-p	is the port number for the LDAP service on your server. In this example, the port number is 389.
-d	is the DN of the entry used to bind to the LDAP server. The name in this example is cn=admin,c=us.
-w	is the password of the entry specified by the -d option. In this case, the password is "secret."

Exporting LDAP Data from an LDAP Server

To export data from an LDAP directory to an LDIF file, you tell the ICE engine to use an LDAP directory source handler and an LDIF file destination handler. In other words, you tell the ICE engine to read entries from an LDAP directory put them into an LDIF file. Figure 3 illustrates this process.

Figure 3: Exporting LDAP data to an LDIF file.

If you were to use the utility's command line interface for this export, the command would look something like this (entered all on one line):

ice -SLDAP -sserver1.acme.com -p389 -dcn=admin,c=us-wpassword -lobjectClass=* -csub - DLDIF-fc:\temp\Server1File.ldif

The parameters are as follows:

-S	is the source type. In this case, the value of LDAP tells the ICE engine that the source is an LDAP server.
-s	is the DNS name or IP address of an LDAP server. In this example, the server is server2.acme.com
-p	is the port number for the LDAP service on the server. In this example, the port number is 389.
-d	is the DN of the entry used to bind to the LDAP server. The name in this example is cn=admin,c=us.
-w	is the password of the entry specified by the -d option. In this case, the password is "password."
-l	is the search filter. In this case the filter says to include any object class matching the wildcard * which means that you're searching for all object classes.
-c	is the search scope. The value of "sub" means that the example will be doing a sub-tree search. Because this example doesn't specify a base DN name, the base entry of the search is the root of the tree.
-D	is the destination handler type. In this case, the destination handler type is LDIF, which tells you that the destination is an LDIF file.
-f	is the output file name. In this example, the file name is c:\temp\Server1File.ldif.

Server-to-Server Data Migration

When you migrate data between LDAP servers, the LDAP source handler reads entries from an LDAP directory to the ICE engine. The ICE engine then sends the data to an LDAP directory. Before the Novell Import Convert Export Utility a data migration would have required the following distinct steps:

• Export the data in an LDAP directory to an LDIF file.

• Move the LDIF file somewhere (probably from one machine to another).

• Import the LDIF file into the destination LDAP server.

The Novell Import Convert Export Utility combines these tasks into a single step. Figure 4 shows this data migration process as a block diagram.

Figure 4: Server-to-server data migration process.

The server-to-server data migration command line looks something like this (entered all on one line):

ice -SLDAP -sserver1.acme.com -p389 -dcn=admin,c=us-wpassword -lobjectClass=* -csub -DLDAP -sserver2.acme.com-p389 -dcn=admin,c=us -wsecret

The parameters are as follows:

-S	is the source type. In this case, the value of LDAP tells the ICE engine that the source is an LDAP server.
-s	is the DNS name or IP address of the source LDAP server. In this example, the server is server1.acme.com.
-p	is the port number for the LDAP service on your server. In this example, the port number is 389.
-d	is the DN of the entry used to bind to the source LDAP server. The name in this example is cn=admin,c=us.
-w	is the password of the entry specified by the -d option for the source LDAP server. In this case, the password is "password."
-l	is the search filter. In this case the filter says to include any object class matching the wildcard * which means that you're searching for all object classes.
-c	is the search scope. The value of "sub" means that the example will be doing a sub-tree search. Because this example doesn't specify a base DN name, the base entry of the search is the root of the tree.
-D	is the destination handler type. In this case, the destination handler type is LDAP, which tells you that the destination is an LDAP server.
-s	is the DNS name or IP address of the destination LDAP server. In this example, the server is server2.acme.com
-p	is the port number for the LDAP service on the server. In this example, the port number is 389.
-d	is the DN of the entry used to bind to the destination LDAP server. The name in this example is cn=admin,c=us.
-w	is the password of the entry specified by the -d option for the destination LDAP server. In this case, the password is "secret."

Tips and Tricks for Using the Novell Import Convert Export Utility

This section describes several ways you can use the Novell Import Convert Export Utility to make some of your directory-related tasks easier.

Use XML Rules

You can use the Novell Import Convert Export Utility in conjunction with XML processing rules to make a number of your tasks easier. For example, you can:

• Solve schema mapping problems. Suppose you have an LDIF file that came from a directory (perhaps from another vendor), and that directory's schema is different from your directory's schema. The Novell Import Convert Export Utility provides a quick and easy way to solve the mapping of the two different schema elements.

• Change the way the entries are placed. Perhaps you want to reorganize the way the entries in your directory are stored as you import them. Maybe you've changed your organizational structure or you've just acquired a new company, and you want to change where entries end up. You can use placement rules to do that. In fact, you can use placement rules to pull entries from the directory, change the way they are placed, and put them back into the same directory at a different location. Then you have two copies of the information, and you can delete the ones you don't want.

• Provide default attribute values for mandatory attributes. If you're doing an import, you might be required to add attribute values to certain entries because of schema differences. Or, perhaps required attributes exist in your destination directory that don't exist in your LDIF file or source directory.

You can use XML rules to solve all of these problems and more. The XML rules work like this: The ICE engine processes each record it receives from the source handler and applies the XML rules to the record before it passes the record on to the destination handler. This allows you to automate tasks that would otherwise require you to edit an LDIF file by hand. This is especially handy in the case of server-to-server data migrations where you want to avoid creating an LDIF file in the first place.

For example, suppose you have the following LDIF file (the italicized portions indicate the information you will be working with):

Version: 1dn: cn=Peter Michaels,ou=Artists,l=San Francisco,c=US changetype: add differentAttr: Michaelsgivenname: PeterobjectClass: iNetOrgPersontelephonenumer: +l 415 555 0001mail: Peter.Michaels@aaa.comuserpassword:Peter123

In this file, you have a user named Peter Michaels. The file indicates that you are adding his record (changetype: add) to the Artists container, and the record has a different attribute than our schema allows (differentAttr: Michaels). In this example, you want to change the container where the object will be created, and you need to fix the attribute that isn't allowed in your schema.

First, to change the container where Peter's user object will be created, you must write an XML placement rule to move the inetOrgPerson entries to the o=sales,c=CA container (the container you want to move the object to). The placement rule would look like this:

<?xml version ="1.0" encoding = "ISO-8859-1" ?<<placement-rules src-dn-format="ldap" dest-dn-format="ldap"< <placement-rule< <match-class class-name="inetOrgPerson"<</match-class< <placement< o=sales,c=CA<copy-name/<</placement< </placement-rule<<?placement-rules<=

This placement rule moves all inetOrgPerson objects to the o=sales,c =CA container. First, the rule states that LDAP is the naming convention (src-dn-format="ldap" dest-dn-format="ldap"). Then the rule states that it's a placement rule for all objects with an object class of inetOrgPerson that takes each object's rdn (stored in the copy-name variable), and puts it in the o=sales,c=CA container instead of the container where it would normally be created.

Second, to fix the attribute that isn't allowed in the schema, you can create a schema rule that maps the differentAttr attribute to the sn (surname) attribute. The schema rule would look like this:

<?xml version = "1.0" encoding="ISO-8859-1"?<<attr-name-map< <attr-name< <nds-name<differentAttr</nds-name< <app-name<sn</app-name<</attr-name<</attr-name-map<

This rule is pretty straightforward: you simply give it the name of the attribute you're mapping, and the attribute name you're mapping to. Because DirXML rules are NDS-centric, the tags <nds-name< and <app-name< may seem a bit misnamed in the context of the Novell Import Convert Export Utility. You can think of <nds-name< as the source name you're mapping from and <app-name< as the target name you're mapping to.

You can set XML processing rules using the -c, -s, and -p general command line options. In the NDS Import/Export wizard, you set the XML processing rules on the Advanced Options Dialog on the Select Task window.

Omit or Exclude Attributes

When you use the Novell Import Convert Export Utility to read data from an LDAP server, you can exclude specific attributes. Sometimes there are just one or two attributes that you don't want to ever have in an LDIF file you're exporting. You can use the "omit attribute" option to make attributes disappear from your list.

To enable the "omit attributes" option, from the command line you can use the -o option of the LDAP source handler. You can omit attributes on the Attributes tab of the Set Search Criteria page of the NDS Import/Export wizard.

Figure 5: Customizing the Set Search Criteria page.

Speeding Up LDIF Imports

You can use a number of tricks to make your LDIF imports more efficient:

• Import to a server with a writeable replica.

• Use LBURP.

• Configure your database cache.

• Use simple passwords.

• Use indexes appropriately.

Import to a Server with a Writeable Replica. If you make certain that the server you're importing to has a writeable replica of all the partitions that will be affected by your import, no chaining or referrals will occur because all of your requested operations will occur on the local server. If you have entries that are only on a different server, the first server will either have to chain on your behalf and send additional data across the network, or the send a referral back to the Novell Import Convert Export Utility. The utility will then have to connect to the referred server to finish the operation. The time lapses and network traffic involved in chaining and referrals will degrade your performance.

Use LBURP. LBURP is enabled by default because its asynchronous nature makes a big difference in the efficiency of sending large numbers of LDAP update operations to a server. Although LBURP is currently being developed within the IETF (Internet Engineering Task Force), the same group that developed LDAPv3, you should keep in mind that LBURP is new--it's only available in NDS eDirectory 8.5 or later. Other vendors haven't yet shipped products with LBURP support.

If you're trying to import data using older versions of NDS or other vendors' LDAP servers, LBURP won't work. The Novell Import Convert Export Utility automatically detects this situation and switches to a synchronous update strategy so that the updates will take place correctly. In this situation, however, the speed of the bulk load operation will be far slower.

You can also explicitly disable LBURP by using the -B option on the LDAP destination handler or on the Advanced Options dialog of the destination LDAP server's page in the NDS Import/Export wizard.

Configure Your Database Cache. Configuring the FLAIM database cache is the biggest thing you can do to improve import performance (and the performance of your eDirectory server in general). You should consider allocating as much as 80% of physical memory on your server to FLAIM cache when you do an LDIF import, especially if the import is the only activity taking place on the server. Similar allocations may also give you the best day-to-day performance on eDirectory servers that are dedicated directory servers. You will probably want to use less-aggressive FLAIM cache allocations for eDirectory servers that also host other applications or services such as NetWare file and print services.

You configure FLAIM cache settings in the _NDSDB.INI file that is located in the same directory as the NDS database files. You can configure the cache to be either dynamically or statically allocated. Use the following parameters to configure the cache:

Cache=<bytes<	<bytes< is the limit (in bytes) of the cache. This setting is static.
cache=<options<	This setting is dynamic. The options can be the following:
% <percentage<	Sets a percentage of the available or total physical memory.
AVAIL | TOTAL	Determines whether the percentage is based on the available or total physical memory. (These options are ignored if the DYN option is used.)
MIN <bytes<	Causes the cache to be at least this large.
MAX <bytes<	Causes the cache to never get larger than this.
LEAVE <bytes<	Amount of memory to leave after allocating everything else to cache.
DYN | HARD	Determines whether the cache settings are static and never change or are adjusted dynamicallyCacheadjustinterval: <seconds< Determines how often the cache size is re-evaluated. This option is ignored if the limit is HARD.Cachecleanupinterval: <seconds< Determines how often the cache is purged.

Here are some examples of how to configure the database cache.

To specify a hard limit of 8 MB (the default database cache size is 8 MB):

Cache = 8000000

To specify a dynamic limit of 75% of available memory, with a minimum of16 MB:

Cache = DYN % 75 MIN 16000000

To specify a hard limit of 75% of total physical memory, with a minimum of16 MB and a maximum of 80 megabytes:

Cache = HARD % 75 MIN 16000000, MAX 80000000

Once you've edited the _NDSDB.INI file, you must save it and restart NDS for the settings to take effect.

Use NMAS Simple Passwords. Using simple passwords eliminates the need to generate key pairs when you update entries that have password attributes. Generating key pairs is a CPU intensive process, so you can really speed up your import, if you use simple passwords.

Using simple passwords requires NMAS on the server, and perhaps on the client. For LDAP clients, no special NMAS support is required, but for traditional NetWare file and print services, you must have an NMAS-enabled NetWare client. The NMAS Starter Pack is available free of charge for this purpose. You can download the NMAS Starter Pack from the Novell downloads web page at http://www.novell.com/download/index.html.

Once NMAS is installed on your NDS server, To you enable the use of NMAS simple passwords using the -l command line option on the LDAP destination handler, or on the Advanced Options dialog of the destination LDAP server's page in the NDS Import/Export wizard.

NMAS enables your application store the following types of passwords via the NMAS simple password method:

• cleartext

• SHA-1

• MD5

• Unix crypt

Use Indexes Appropriately. Using indexes appropriately may not speed up your import, but it will keep the import from going slower. Keep in mind that using extra indexes may speed up searches, but they slow down updates. So don't have more indexes than you need. Use predicate statistics to decide which indexes you need.

Debugging Your LDIF Files

If you generate your LDIF files by hand, it's difficult to create perfect files that don't need to be debugged. This section discusses common problems in LDIF files and tells how the Novell Import Convert Export Utility can help you debug them.

Use SSL or Allow Clear Text Passwords

Although this isn't really a problem with LDIF, it's the most frequently-encountered problem we hear about. By default, eDirectory is configured to require SSL connections for security reasons. Without realizing this, many users attempt to connect to the LDAP server's clear text port (usually port 389). By default, eDirectory blocks connections where a password would travel across the network without being encrypted because Novell doesn't want people sending their credentials (such as passwords) across the wire in the clear without realizing that they're doing so.

Starting with NDS eDirectory 8.5, you can enable clear text connections as part of the installation. You can also enable the "allow clear text passwords" option on the LDAP group object after you've installed eDirectory. Although enabling this option allows unencrypted connections, you can still make SSL connections to your server after this option is enabled. Novell usually enables clear passwords only in test environments. We recommend that you only enable unencrypted connections to your LDAP servers in test environments, for debugging purposes, or in situations where using unencrypted connections poses no security risk to your data (for example, when all access to the directory is public).

You can get additional detail on setting up and using SSL connections with the Novell Import Convert Export Utility in the resources listed at the end of this AppNote.

Enable Forward References

It's not uncommon to encounter situations where a two-way link exists between objects. For instance, an inetOrgPerson entry has an group membership in a group, and the group also lists that same inetOrgPerson as one of its members. You may also encounter LDIF files where entries are in the wrong order (for example, a parent entry comes after one or more of its children). In situations such as these you can enable forward references to allow the creation of temporary placeholders to let the desired add operations succeed when they would normally fail. See the discussion on forward references earlier in this AppNote for more information.

Be aware that if you enable forward references, you may see unknown objects in ConsoleOne. This occurs when a forward reference is created to resolve a reference for an entry that doesn't exist when one entry is created, but the LDIF file doesn't contain a command to actually create the forward reference entry. In this case, the server never transforms the forward reference into a real entry, and the entry is displayed as an unknown object in CosoleOne. To fix this, you can create another LDIF file that explicitly creates the object that you want to transform from a forward reference and import the LDIF file.

To enable forward references, you can use either the -F command line option on the LDAP destination handler or you enable it on the Advanced Options dialog of the destination LDAP server's page in the NDS Import/Export wizard.

Set the Server Search Entry and Search Time Limits

If you're not getting all of the records you expect during an LDIF export, the problem might be the search size and time limits. These settings limit the number of entries a server returns and the time it will spend in a single search operation. In addition to being able to set these limits from the Novell Import Convert Export Utility, Administrators can limit these parameters at the LDAP server itself to prevent users from reading large numbers of entries out of the directory in a single search and eroding network performance.

When an eDirectory LDAP server receives a search operation request, it compares the limits set in the client request and the server's configuration and uses the smaller of the two limits. For example, if you configure the LDAP server to allow unlimited time on a search, but your client request allows only 30 seconds for a search, the server will search the database for only 30 seconds. Or, if the client request allows 1000 records to be returned and the server is configured to allow only 500 records, the server will return at most 500 records. (The Novell Import Convert Export Utility sets the client values to unlimited by default.)

You can set the limits for client requests using either the -l (time) and -z (size) command line options on the LDAP source handler, or you set them on the Advanced Options dialog of the source LDAP server's page in the NDS Import/Export wizard. You can set these limits for your LDAP server by modifying the properties of the LDAP Server object in NDS.

Check the Syntax of Your LDIF File

Before you use your LDIF file to start applying the entries to your server, run the first 8 or 10 records through the syntax checker. If you use this feature, the ICE engine won't execute the LDIF file. It will just read the file and check its syntax.

To check the LDIF file's syntax, you can use either the -n command line option on the LDIF source handler, or you can set it on the Advanced Options dialog of the source LDIF file page of the NDS Import/Export wizard.

If the file has errors, the errors will be printed out on the screen. You can also send the offending records them to an LDIF error file. See the next section for information on generating an LDIF error file.

Use the LDIF Error File

The LDIF error file contains the entire contents of each failed record that passes through the ICE engine. You can correct the errors directly in the LDIF error file and then import it again to successfully complete an import that had one or more failed records.

To enable the LDIF error file, you can use either the -e command line option in the general options section, or you can set it on the Advanced Options dialog of the Select Task page of the NDS Import/Export wizard.

Use LDAP Debugging Flags

Although this is a fairly low-level technique, there are times when you may need to see how the LDAP client SDK is functioning to understand a problem. You can set LDAP SDK debugging flags for the LDAP source handler, the LDAP destination handler, or both. This functionality is currently only available in the command line interface for the Novell Import Convert Export Utility. You use the -e option for the LDAP Source and LDAP Destination handlers to enable it. The integer value you give for the -e option is a bitmask that enables various types of debugging information in the LDAP SDK. The table below shows each of the values and their associated meaning.

Value	Description
0x0001	Trace LDAP function calls.
0x0002	Print information on packets.
0x0004	Print information on arguments.
0x0008	Print connections information.
0x0010	Print BER encoding and decoding information.
0x0020	Print search filter information.
0x0040	Print configuration information.
0x0080	Print ACL information.
0x0100	Print statistical information.
0x0200	Print additional statistical information.
0x0400	Print shell information.
0x0800	Print parsing information.
0xFFFF(-1 Decimal)	Enable all debugging options.

Conclusion

The Novell Import Convert Export Utility, to be released with NDS eDirectory 8.5, will make importing data into LDAP directories, exporting data from LDAP directories, and migrating data between LDAP directories easier to do. You can also use the utility to automate tasks such as schema mapping, entry placement and LDIF file format checking.

For more information about the Novell Import Convert Export Utility, refer to the following resources:

• NDS User Documentation

• Novell NDK: LDAP Libraries for C documentation

• Novell's LDAP Developer's Guide (Novell Press; ISBN 0-7645-4720-8), Chapter 11.

• RFCs and Internet Drafts:

  • The LDIF technical specification, RFC 2849, at http://www.ietf.org/rfc/rfc2849.txt

  • The String Representation of LDAP search filters, RFC 2254, at http://www.ietf.org/rfc/rfc2254.txt

  • The LBURP specification, draft-rharrison-lburp-02.txt, at http://www.ietf.org/internet-drafts/draft-rharrison-lburp-02.txt

* Originally published in Novell AppNotes

---

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

• Feedback
• Print
  •  
  • Full
  • Simple
•  
• Request a Call
• Follow Us
  •  
  • Facebook
  • YouTube
  • Twitter
  • LinkedIn
  • Newsletter Subscription
  • RSS