Overview of Novell Replication Services
Articles and Tips: article
Product Line Manager
Internet Access Division
01 Aug 1997
Discusses such topics as the role of NRS in the workplace, synchronization, and replication hierachies. Shows how NRS meets enterprise requirements by offering distribution of productivity documents.
- Introduction
- Why Use Novell Replication Services?
- What is Novell Replication Services?
- Novell Replication Services in the Workplace
- How Does Novell Replication Services Work?
- Conclusion
Introduction
Today's fast-moving, global business environment often requires widely distributed groups of people have immediate access to information-market studies, sales presentations, financial reports, address books, and other productivity documents-normally found only on corporate servers. Many of these remote locations have only a slow, or even intermittent WAN connection, making quick access to this information difficult. Although faster WAN technologies are being created and used by companies, end users trying to access information are still faced with the problem of quickly accessing data from the main corporate server.
Additionally, keeping remote machines in synchronization with the main servers presents a difficult challenge. System administrators have often turned to replicating file systems on servers throughout a network to reduce delays and provide remote users with the latest copies of software and documents.
Replication improves WAN performance by moving information closer to the end user. However, manually replicating and synchronizing data across an entire enterprise requires daily analysis of file system changes, and frequently involves error-prone manual file transfers or writing batch files or scripts.
To maximize any benefits of replication, a new replication management tool is needed, one that
Is easy to use and administer
Optimizes WAN usage and performance
Novell Replication Services is the tool that provides these benefits.
Why Use Novell Replication Services?
Novell Replication Services (NRS) lets both administrators and system users enjoy the advantages of accessing a replicated file system on a local server, without the disadvantages of manually coordinating the replication and synchronization of the file systems.
Easy to Use
Novell Replication Services gives system users access to information at LAN speeds, even to graphics and video files. Users can more effectively share and collaborate on their work. Sales figures, engineering documents, software and code changes can be accessed and acted upon more easily by key personnel. Users do not have to perform any additional steps to replicate their files, they simply save to a server running NRS.
Novell Replication Services also simplifies the task of data replication for system administrators. NRS detects changes made in any of the replication directories and automatically replicates only those files that have changed, creating a fully synchronized file and data image at each server. NRS is a Snap-In to the standard NWAdmin utility, providing a consistent interface with other NWAdmin tools and making it easy to learn.
Optimizes WAN Usage and Performance
Novell Replication Services also lets system administrators manage WAN bandwidth. NRS has a powerful scheduling facility that lets administrators schedule replica synchronization during off-peak hours. NRS further reduces WAN traffic by putting information closer to the end users, reducing the need to access the WAN and freeing the WAN for critical applications. Additionally, end users can avoid the latency delays associated with accessing a remote server.
What is Novell Replication Services?
Novell Replication Services is a powerful new replication management tool that lets network administrators selectively replicate files from one server to other servers in the same Novell Directory Services tree. Novell Replication Services provides the infrastructure for distributed Intranet and Internet solutions on IntranetWare. Example solutions are document and software distribution, Web replication and general purpose wide area file sharing. Novell Replication Services is designed for use with the following types of files:
Productivity documents, such as word processor and spreadsheets files System files such as applications, utilities and operating systems Internet documents such as HTML files.
Novell Replication Services co-exists with other dedicated groupware applications such as electronic mail.
Novell Replication Services is a single NetWare Loadable Module (NLM) file that runs on each server from which or to which files and directories are replicated. Administration is provided by a snap-in to Novell's NetWare Administrator utility, so replication and synchronization can be managed from any network client.
Novell Directory Services provides the administrative infrastructure required by Novell Replication Services. All servers that are part of a replication tree must have corresponding server objects in the same Novell Directory Services directory tree. This lets Network Administrators use the server objects in the directory tree and NetWare Administrator to administer Novell Replication Services.
Novell Directory Services also provides a secure infrastructure. Keeping the servers in the replication tree in the same directory tree ensures the file system trustee rights are synchronized since file system trustee rights reference user and group objects in the directory tree. If only a single user has the trustee rights to update the product brochure file, then the trustee rights for the file are synchronized preventing other users on any server from updating the file.
Novell Replication Services in the Workplace
To fully appreciate the power of Novell Replication Services, consider the simple example of a global enterprise with a product brochure. An employee saves the product brochure as a single file at the company's headquarters in New York. No special action is needed to distribute the file. The act of saving the file, from within any application, onto a server running Novell Replication Services is all that is needed. Within minutes of the file being saved, it is automatically synchronized to all regional offices in the US, over the company's high bandwidth 1.5 megabits per second leased lines. From the US regional offices the file is transmitted onwards to US branch offices using 56 kilobits per second links.
At the same time, the updated file is automatically transmitted to the international offices in Buenos Aires, London, Tokyo and Sydney using 2 megabit per second links. From these international offices, the file will automatically be onwardly synchronized to country level offices around the world, and from these onwards to international branch offices. As a result, the file is available in every office world wide. The only action needed to achieve this was to save the file.
To the company employees, replication means the product brochure is more accessible because they can access it on a local server, wherever they are in the world. Local accessibility means better performance. Instead of accessing the product brochure across slow 56 kilobits per second links or high latency international links, the product brochure is available at local area network speeds of 10 to 100 megabits per second, with almost no latency delays.
A second reason why the product brochure is more accessible is that, after initial replication, the availability of the product brochure is not dependent on intermittent wide-area links.
The above example could have been achieved using electronic mail to every sales employee, company wide. However, using a replicated file service to distribute the file has two advantages: ease-of-use and manageability. End users find it easy to use because all they have to do is save the document, with their application of choice, and they're done. The primary manageability advantage is that administrators can control the scheduling time for replication. For example, replication on busy international WAN links can be delayed until off peak times.
The above example could also be implemented with a simple batch file that copies the product brochure file to all the destination offices once per day. If a company only wanted to distribute one single file to a small number of permanently connected offices, this might suffice. The real needs of an enterprise are of course not that simple. Novell Replication Services solves an enterprise's requirements with the following services:
Distribution of productivity documents- Word processor, HTML, spreadsheet, presentation and other productivity documents can be distributed or shared worldwide. These are the documents produced by the most highly paid professionals in a company. Selectively, efficiently and securely replicating these documents brings enterprise-wide productivity advantages by improving accessibility.
Distribution of software- The applications and utilities to read and update the productivity documents can be distributed so they can be installed on servers and client machines at all sites.
Ease of use -Replication is part of the standard administration software. Anadministrator can easily specify which files are to be replicated, where the copies are to be sent, and when the replication should take place. When the replication and synchronization schedules are set, replication is automatic. If a problem occurs, the management tools notify the administrator. Diagnostics tools let the administrator determine and resolve problems from any client machine.
Improved wide area network performance- Copying every document daily across a wide area communications link is an inefficient use of a costly resource. Novell Replication Services lets an administrator control the use of the wide area links by determining which directories are replicated where, and when the synchronization occurs. The replication software synchronizes only the changed files, minimizing the replication traffic on the WAN links. Replication reduces the usage of WANs by replacing peak-time file access with off-peak synchronization.
Two way synchronization- Any enterprise needs to make communication two way. Novell Replication Services can let files be updated at any location and flow back to the rest of the enterprise.
Robustness -If a network connection fails during synchronization, Novell Replication Services ensures the consistency of the files and automatically continues synchronization, without re-sending files, when the connection becomes available.
Security - Manyadministrators will be concerned about the security implications of having secure documents at multiple locations. Novell Replication Services synchronizes the security rights along with the files, guaranteeing the same level of security independent of location.
Tighter Synchronization- Because Novell Replication Services is more efficient and cost-effective in its use of communication channels, administrators can afford to replicate more often, keeping the replications in closer synchronization.
How Does Novell Replication Services Work?
The following section of this DevNote explains:
Defining a replication hierarchy- The process of defining which file system objects from which servers should be replicated on other servers
Synchronization -The process by which changes made to the file system on one server are applied to other servers
Defining a Replication Hierarchy
Novell Replication Services lets a network administrator selectively replicate NetWare file system objects (volumes, directories, files, trustees, IRFs, and name space entries) from one server to any other server in the same Novell Directory Services tree. The administrator configures the server from which file system objects are to be replicated as a master server, and each server to which file system objects are to be replicated as a replica server. The group of servers comprising a single master server and the replica servers containing objects replicated from that master server is called a replication region. A replica server can also be configured as a master server to its own set of replica servers, acting as a link server between two replication regions.
The resulting hierarchy of servers eliminates the need for direct replication from an existing source of information to every other location requiring the information. Users of the replicated file system objects will see the same directory structure from the root of the NetWare volume regardless of location in the network or whether they map to the master server or one of its replica servers.
The following example shows how a master server in New York connects to a link server in London, and onwards to replica servers at other European sites.
Figure 1: A master server connected to a link server and onwards to a replica server.
The primary advantage of using this kind of replication hierarchy is that file system objects only need to be transferred once over each WAN link. When an employee saves the product brochure file to the master server in New York, it is only transferred once across the trans-Atlantic link to the London server. From there it is transferred onwards to the other European servers.
Defining Two-Way Synchronization
Novell Replication Services supports two-way synchronization. In the above example all documents are distributed one way from the New York server. The New York server is configured as a protected master server to prevent any updates flowing back from its immediate replicas. In this example, the corporation translates all publications replicated to Europe to different languages and the different language versions are held in separate directories present only on the European Replication Region servers.
Since the European master server in London is not configured as a protected master, French versions of the publication can be created in Paris. The French translations can be synchronized back to London and onwards to other European servers to support French speakers throughout Europe.
Since Novell Replication Services supports two-way file synchronizations, it also handles conflicting updates of the same file from other servers. Novell Replication Services supports automated clash resolution that performs a default action to handle the clash, allowing synchronization to continue.
Automated clash resolution is important because synchronization is frequently scheduled for off-peak times when no one is available to manually resolve the clash. Automated clash resolution guarantees no files, or updates are lost, and creates a recovery log providing details of the clash and the location of the recovered files.
The simplest example of a clashing update is that of a file changed by two different users at two different servers in the replication tree. When this occurs, the file updated on the master server first becomes the master version, and the second file becomes a recovered file. Recovered files are stored in a recovery directory on the same volume as the original files, and preserve the same access rights as the original files. Notifications of clashes are reported by console alerts that can be filtered by a Network Management utility such as ManageWise, and then manually resolved.
Clashes are very rare for most uses of Novell Replication Services. This is because the typical uses are one way distribution of files to servers where end users do not have update rights to the files. Even in the cases where Novell Replication Services is used to support wide area document sharing with multiple users of the documents allowed, studies have shown that the frequency of clashes is typically less than once every six months per user. This frequency can be further minimized by use of access rights to allow only users at a single location at a time.
Defining Selective Replication
Novell Replication Services supports selective replication, where different directory sub-trees within a volume can be selectively replicated to different servers. The following figure shows a simplified example of the directory and files contents of the DATA-VOLUME on the New York master server and the London link server. Replication between other servers can choose different directories to replicate.
Figure 2: Selective replication, with different sub-trees replicated to different servers.
In this example, the DOCUMENTS directory is included for replication from the New York server to the London server. The WORK IN PROGRESS directory under the DOCUMENTS directory is excluded from replication to London. The contents of excluded directories are not replicated. On the London server, the SALES, WORK IN PROGRESS and RELEASES directories are all present, but are treated as name only directories. Name only directories are used to maintain equivalent naming on all servers and cannot be renamed on the replica server.
The name only directory SALES is a stem name only directory, this means it is one of the directories in the stem path between the included directory (DOCUMENTS) and the volume root. Stem name only directories not only preserve naming, they also preserve inherited access rights and ensure the same security rights are maintained on all replicated directories and files. WORK IN PROGRESS and RELEASES are leaf name only directories and do not allow files or directories to be added.
The directory TRANSLATIONS is a masked directory,meaning the effect of replication is masked from that directory downwards. Masked directories are similar to excluded directories in that their contents are not replicated. The difference is that on the replica server different files and directories can be added and their contents are not synchronized back to the master. In this example the masked directory on the link server in London can be replicated onwards to the servers at other European cities.
For the purposes of explanation, this sample directory structure and content was kept simple. In practice, the directories chosen for replication can contain large numbers of files and directories, requiring megabytes or gigabytes of storage. The amount of data to be replicated during the first synchronization can pose a problem if slow links are involved. For this reason Novell Replication Services supports pre-loading.
To avoid tying up WAN links for an extended period of time, the contents of the DOCUMENTS directory in the sample file structure could be backed up on the New York server. The backup could then be sent to London where it is pre-loaded on the London server before the initial replication. The initial replication checks the remote file system and does not transfer files if they already exist. Every synchronization thereafter only transfers the updates that have been applied on either server.
Synchronization
After defining a replication hierarchy, an administrator can then set up a synchronization schedule defining when the synchronization occurs. Any file system update affecting the names spaces used by DOS and Windows is synchronized, including file creations, deletions, renamings, modifications, and updates to attributes or access rights. Macintosh, OS/2 extended attributes and NFS name spaces will be supported in a later release.
The following is a simplified schematic of the steps involved in the synchronization of the product brochure document between a master server in New York and a replica server in London.
Figure 3: Steps involved in synchronization.
Many details of the synchronization algorithms have been omitted from this description. The NRS transaction log on each server is held in an NRS database that records not only the changes to the file system, but also the state of the file system. The state of the file system is used in the event of a server crash. When restarted, Novell Replication Services compares the file system to the database and reconciles the two so no updates are lost. In addition, the transaction log is logically compressed so multiple updates to the same file are compressed into a single MODIFY operation.
Some key benefits of the Novell Replication Services advanced synchronization algorithm are:
Low server overhead- The asynchronous intercept process allows file system updates to be tracked. Since this is a memory only operation and does not involve the performance critical read and write file operations, the effect on client response times is minimal.
File system scalability and performance- Because a log is used to track and play back file system updates, the synchronization algorithms have excellent scaling and performance properties. Unlike most first generation replication systems, the synchronization algorithms do not have to compare the states of the file systems on the master and replica serversto determine which files have changed. This means the time required to synchronize 100 file system changes when there are one million replicated files is not be significantly different from synchronizing 100 file system changes when there are only one thousand files. These characteristics are particularly important for slow WAN links when replicating large numbers of files.
Server scalability- A single master server can concurrently synchronize up to 100 link servers. Each link server can then replicate with up to 100 further replica servers. With only two levels in a replication tree it would be possible to maintain more than 10,000 servers containing the same files. The log based replication algorithms allow all servers to be synchronizing in parallel, making optimal use of the WAN links.
Robustness -Use of a transactional NRS database means Novell Replication Services recovers quickly and completely from any server or network failures and guarantees no updates are lost. In addition, the use of checksumming as part of the synchronization process means even when checksumming is turned off in the underlying transport protocol (typically the default), no corruption can occur when there is a fault on the network hardware.
File level transaction granularity- Log based synchronization records and plays back file system updates in the same order, subject to logical compression, on the target server as the source server. In addition, file content updates are applied using a temporary file. These two features mean Novell Replication Services supports dynamic, continuous updates of a file system, where any network or server failure guarantees never to leave a file partially updated.
Conclusion
Novell Replication Services provides a solution to the problem of latency delays, large data stores, and differences between LAN and WAN performance and cost characteristics. The advantages provided by NRS are:
NRS is easy to use and requires no additional actions from a user
Administrators can easily manage NRS software
End users have faster, better access to their data
For more information on Novell Replication, send an E-mail inquiry to replication@novell.com.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.