Identity-Driven Storage Management: Novell File System Factory
Articles and Tips: article
09 Dec 2004
Unless you have that dream job of running that t-shirt shop in the Carribean, you've probably noticed that your organization is becoming more dynamic by the day. Your users come and go in droves, relocate within the organization at the drop of a hat, and constantly participate in workgroups and project teams whose challenges and needs are as diverse as the people in them.
Information Technology has become more dynamic too. Gone are the days of the departmental server and its overlord administrator. IT is tying systems together, syncing business processes with online systems, and users need resources immediately and expect them to be available 24x7.
Storage is one of those heavily demanded resources. Lots of sexy things are happening in computing these days, but at the end of the day, people need to share and securely store files on the network. Otherwise, your company's data is at risk and people can't be effective in their jobs.
Everyone knows that NetWare is a solid storage solution; Linux is coming on strong; and that Cluster Services and SANs provide fault tolerance, scalability and performance. You may be contemplating a mix of these technologies now or in the near future as you move forward with storage. But if you are like most people these days, you're just trying to get what you have now under control. The good news is that Novell can help you do that and move forward with storage in your dynamic environment.
What's Your Problem?
It is well known that Novell Nsure Identity Manager 2.0 provides drivers that allow user accounts to be automatically automatically synchronize user accounts, as well as changes such as new locations, departments and group memberships across the enterprise. You can also use Novell ZENworks to automate the rollout of new applications and publish them to desktops based on new locations and group memberships that are listed in the directory. These are all great tools to save administrators time and reduce help desk costs. But what about storage?
As a network administrator or someone that manages network admins, you probably have experienced the difficulty of creating, managing, moving and cleaning up user and group storage. Users transfer to different locations or departments, change their name or leave the organization. Maybe at the last minute, an urgent request comes in for creating shared storage for a project. In addition, the storage requirements for user home directories and group storage keep growing. You need some way to better manage storage and get things under control.
These storage-management challenges are not new to administrators. The realworld costs associated with the day-to-day tasks involved in storage management are both real and considerable. Administrators still have to manually move, rename and clean up users' home directories. For moves and renames, you have to ensure that the rights are set correctly and update the home directory attribute. The demands for group storage are the same. And what about cleanup or management of the data after the owner is gone or the project is completed? You are cleaning up storage appropriately, right? Stop laughing!
If you can manage applications and desktops based on identities and policies stored in eDirectory, why can't you manage storage in the same way?
Driving Storage Management Based on Identity
If you've asked that last question, what you really want is to provision, access, manage, move and clean up storage based on identity as shown in Figure 1. And you want all this to be automatically governed by policies. The Novell Identity-Driven Computing initiative that was launched in September at Novell BrainShare in Barcelona, along with Identity-Driven Storage Management (IDSM), solves that problem.
Figure 1: The Novell Identity-Driven Computing initiative that was launched in September at Novell BrainShare in Barcelona, along with Identity-Driven Storage Management (IDSM), automatically provisions, accesses, manages, moves and cleans up storage based on identity and governed by policies.
IDSM ties storage to identity in the Directory itself so that storage is automatically provisioned, managed and cleaned up based on the user. This means automatically resolving issues of both personal and group storage for collaboration at any level. It keeps data with users as they move around in the organization and controls storage quotas based on users' roles, or identities, within the organization.
IDSM In A Box
Novell File System Factory is the industry's only IDSM solution today. With File System Factory, you manage the complete lifecycle of a user's storage based on the person's identity and role in the organization. File System Factory is an event-driven file system management solution for eDirectory and NetWare and is available today. Deliverables are in the wings for Open Enterprise Server (OES) and Linux-based storage governed by eDirectory, as well as Windows-based storage governed by Active Directory.
Ultimately, File System Factory creates, manages and deletes disk storage based on events in the directory. It is designed to provide these services for users' home directories (personal storage), as well as shared disk space (group storage). In fact, File System Factory adds a HomeDirectory attribute to the group object so workgroups can now have home directories.
File System Factory helps you get your storage-management polices off the bulletin board or out of the SOP binder on the bookshelf and add them as objects in eDirectory where they can make a difference when you apply and adhere to them. Policy definitions allow separate management methods to be used for storage for users and groups in the same part of the tree. A single policy can be assigned to multiple objects in the tree for unified management. A policy can be assigned to a container, a group or directly to an individual user object. Standard inheritance determines which policy, if any, is applied for a given object. Actually, if you know how to manage your applications using ZENworks policies, you already know a lot about how to manage your storage with File System Factory policies.
Figure 2 shows event flow and policy application. When an event occurs in the tree, the event-monitoring component of File System Factory immediately intercepts the event. Because File System Factory contains a Global Event Subsystem, you can run multiple instances of the event monitor simultaneously on multiple servers in the same replica ring without any concern for replica location collisions and duplicate event processing.
Figure 2: When an event occurs in the tree, File System Factory immediately intercepts the event and automatically applies storage management policies to create, manage and clean up storage.
The intercepted directory events are sent to a server running the File System Factory action engine component. If the engine determines that a policy has been associated, either directly or indirectly, with the object being created, modified, moved or deleted, the engine then applies the policy. Policies contain provisions for having File System Factory not only create and load-balance the file systems, but also apply quotas, set trustee assignments, and copy a given template directory containing files and subdirectories with trustees. You can manage directory cleanup so that file system deletions occur immediately after object deletion or are deferred for a specified amount of time.
You can apply templates to users or groups. File System Factory can copy in a specified template directory from anywhere on the network for each user or group home directory created. You can copy files, directories, trustee assignments and attributes. You can also use the copy subsystem to automatically move storage around on the network. When you move a user in the tree, File System Factory can automatically move the user's storage to another location on the network, just based on the move event. Administrators can also use File System Factory to migrate many users to another location, possibly a Cluster Services implementation or a SAN.
The engine has an internal transactional state machine architecture which allows the engine to resolve transient waits inherent to eDirectory synchronization, as well as unexpected outages. The state machine architecture also allows 100 percent recovery from any failure involving network communications, a target server or a server running a component of File System Factory.
Making a Difference Today
Often, a technology puts so many requirements on the customer's network that it becomes infeasible as an immediate solution. By design, File System Factory makes network administrators' lives easier today, no matter what they have or how they do things. You don't have to install Novell Identity Manager to implement File System Factory, but the two function very well together. They work well in concert to effectively make any existing driver deal with storage without modification. Because File System Factory is event- and policybased, you can create, modify, rename, move or delete objects any way you wish, today and tomorrow.
For example, a large corporation had been delaying the move to an identity management solution because its UIMPORT batch jobs were dynamically created to place home directories in a load-balanced configuration across a set of servers. Using File System Factory, the IT team was able to move to a driver-based IDM solution quickly and painlessly by pointing a File System Factory policy to the same servers and enabling the load-balancing algorithms in File System Factory. Now, the driver creates the accounts and File System Factory sees the events and applies the correct policy to provision and assign the storage.
Another good example of Identity-Driven Storage Management is a university that wanted to use multiple provisioning technologies simultaneously. An IDM-based driver creates accounts based on the student database while a Web-based system using an LDAP backend creates accounts for local community members that want to audit or take classes. Now, in both cases, as the accounts are created in eDirectory, File System Factory sees the events and provisions storage accordingly using either the student or visitor storage policy.
But what about storage and users you already have? How can File System Factory help you get things under control? File System Factory contains a "Backfill" feature that allows you to retroactively begin managing existing user objects in the tree. File System Factory dynamically provisions storage for those that don't have storage and catalogs and picks up preexisting storage for users and manages it as though File System Factory created it.
Then there's the large and diverse health-care organization that manages multiple sites in eDirectory. Using an organizational tree design, each location is responsible for management of its part of the tree. Users are introduced into the tree using a combination of a locallydeveloped Web application and manual entry into ConsoleOne or NWAdmin. In one location, the children's hospital, no users have personal storage on the network. In another location, the main hospital, some users have storage, but the users may or may not have a quota, depending on when they were added to the system.
Because of a recent outbreak of viruses (yes, they even have viruses in hospitals), management decreed that all employees were to have 200MB of storage on the network and that local hard drives were not to be used to store vital information of any kind. Creating a storage policy with a 200MB quota and pointing it toward the servers at that location was the first step. After the policy was assigned to the container, the administrator did a "backfill" on the container using the File System Factory Web-based management interface. This immediately created managed storage for everyone already at the children's hospital. The administrator at the main hospital created a similar policy, pointing at the servers in her location. The resulting backfill operation provisioned storage for everyone that didn't have a home directory, cataloged all existing home directories and began managing them all according to the policy.
All new users in both locations now automatically receive storage according to the policy; renames are now handled correctly and storage is automatically vaulted according to the policy when people leave. Operational changes were not required at either location.
Migrating Is For The Birds Unless You Automate
As I said at the beginning, organizations are dynamic with lots of users coming, going and moving en masse.
Put yourself in someone else's shoes: Suppose your employer is a school district and has implemented a Novell Nsure Identity Manager 2.0 driver which creates accounts from the district's centralized student database, as well as groups representing each class in the district. You have 46,000 students in 80 schools, and each school has its own set of local servers. Your mandate is to:
Create an ePortfolio for students so their storage stays with them from kindergarten through graduation as they move from school to school.
Create group storage for each class such that students can submit homework electronically, as well as retrieve assignments and other class data.
You easily resolve these issues with File System Factory. You create a policy for each school so that it points to the server at each respective school and the respective container. If someone transfers between schools in the middle of the year, it is as simple as moving the object in eDirectory. File System Factory will see the move event in eDirectory and determine that the policy at the new school now applies. In applying the policy, File System Factory sees that the user's storage is not in a location pointed to by the policy so it moves the user's storage to the servers at the new school. At the end of the year, just about everyone is promoted, which results in lots of users moving between schools and some graduating. By having the driver move the user objects or change the location designations, this triggers lots of storage migrations for File System Factory. Bandwidth throttle and scheduling attributes in the policy allow you to control the flow of data. That takes care of the ePortfolio following the student.
Now let's tackle the group storage for each class. The Novell Nsure Identity Manager 2.0 driver creates and populates an eDirectory group for each class. You simply apply a File System Factory group storage policy to these events, and the storage is automatically provisioned for each group. Group storage templates defined to the policy direct File System Factory to lay down a subdirectory structure within the storage, which automatically creates a personal folder for each student and sets appropriate rights for student and teacher access. Without File System Factory, this entire process would be impossible from a resource perspective alone.
In a commercial setting, the ROI involved with moves can be significant. Suppose you're in charge of the network for a company that has 10,000 users with locations spread across the United States. In one month you may get as many as 200 or more user location and departmental move requests that require you to take action regarding network accounts and storage. You want the user data to remain with the users. In other words, users' data should move from servers at their old location to the servers at their new location. Also, each location has its own storage management policies with regard to quota. As you know, these moves are resource intensive with respect to your personnel and require coordination between your IT staff members at both the source and destination locations. A simple move of the eDirectory user object triggers an Identity-Driven Data Migration that saves a significant amount of time, effort and cost.
Many customers are moving to Cluster Services and storage subsystems based on SAN infrastructures, and in the near future, OES Linux storage subsystems. In most cases, this involves a consolidation from a farm of servers to a few physical boxes. Today's environments require around-the-clock service with little opportunity for maintenance. Suppose you're in charge of this type of migration for your company, a bank with 20,000 employees. The "Backfill" operation includes an option called "Enforce Policy Paths." When selected, File System Factory also analyzes the location of the home directory and migrates it as needed to bring the home directory into compliance with the policy. You do a Backfill and go home. Any administrator faced with doing this manually or the person paying overtime will testify to the ROI there. The state machine architecture on which File System Factory is built allows it to take steps to ensure seamless operation of the migration, even if the given user is logged in at the time or if outages occur.
The Last Word
As you read this, File System Factory is provisioning, managing, migrating and cleaning up storage for millions of users in a wide array of organizations all over the world. The key to having this all work correctly from both a technical as well as a philosophical perspective is basing the solution on identity.
You can download trial software and more information about File System Factory from the Novell web site. There's a lot more to File System Factory than I've been able to cover in these few short pages, but I've given you a taste of what's possible. You can also check out the product support forum. If you have a storage management problem you need to solve, post it and we'll be glad to help solve it.
As with most decent ideas, the most common initial reaction to File System Factory and IDSM is, "Surely someone has already done this?" or perhaps more appropriately, "What took you so long?" IDSM is a solution that is powerful in both concept and implementation but simple in its execution. You are limited only by your imagination.
Driving storage allocation, management and cleanup using identity is a key component of achieving true utility-based computing. That's where you finally get out of the business of the repetitive, the mundane--and let's face it--the boring tasks. Get to the real business you happen to be in, whether that's educating people, making widgets, curing cancer or running that t-shirt shop.
Storage Dashboards
Managing storage means getting a good handle on how your storage is being used and at what rate the usage of it is growing. Both network administrators and upper management have a need for this type of information, although the detail level and presentation characteristics of the two are different.
File System Factory includes a separate interface to allow specified users to view statistics and generate reports based on the storage being managed based on policy. There are two interfaces, one for each set of constituents. These are called the Executive Dashboard and the Administrative Dashboard.
The Administrative Dashboard is built into the Web-based File System Factory management console. (See Figure 5.) This dashboard provides detailed reporting on managed storage allocation and growth. Reports and graphs can be generated based on volume, policy or tree subsection.
Figure 5: The Administrative Dashboard provides detailed reporting on managed storage allocation and growth. Reports and graphs can be generated based on volume, policy or tree subsection.
The Executive Dashboard includes a separate easy-to-use interface to allow specified users to view statistics and generate reports based on the storage managed by File System Factory. The interface may be secured using a named security principle object such as a single user or security equivalence object such as a group or organizational role.
Take the Storage Management Quiz
|
Question
|
Yes
|
No
|
|
Do you give all the users in your organization personal storage on the network? |
||
|
Are all user home directories currently on the server/volume you want and organized the way you would like? |
||
|
If you are contemplating a storage migration to Cluster Services or a SAN, can you do that in the middle of the day with no impact on users at any level of granularity? |
||
|
Can you easily load-balance storage use and access across multiple servers, volumes or storage subsystems? |
||
|
Can you easily adopt new or multiple account provisioning technologies such as Novell Nsure Identity Manager or LDAP with no changes to your storage management and provisioning schemes? |
||
|
As your users move or transfer to new locations in the organization, can their storage seamlessly move with them automatically with no action on your part? |
||
|
Do you have policies governing storage quotas and increases? Are those policies stored in eDirectory and automatically applied no matter what? Can you set and enforce quota ceilings? |
||
|
Can you allow your help desk and support staff to manage storage quotas without giving them access to the contents of the file system? |
||
|
Do you provide all working groups with collaborative storage and an easy way of finding and using that storage? |
||
|
Do you automatically clean up or vault shared storage according to policy when groups are disbanded? |
||
|
Do you have workflow capabilities in place that allow storage to be queued to managers for evaluation when someone leaves? |
||
|
Are storage dashboards available to upper management, allowing them to view growth and distribution of storage over time? |
||
|
If you answered "No" to any of these questions, treat yourself to this entire article! |
The Quota Manager
File System Factory storage policies determine what quota is automatically assigned when storage is provisioned for users and groups. Administrators can retroactively apply and change quota for users and groups simply by changing the policy.
This feature alone quickly becomes an invaluable benefit to organizations looking to control disk assets. But there are other issues surrounding quota management in organizations today and File System Factory also solves these problems. Let's look at one such feature; the Quota Manager subsystem.
Most organizations assign an initial quota to user storage and have organizational policies around requesting and granting quota increases. But where are these policies located? Many times they are on the bulletin board at the help desk, in a mail message, or worse, in a few peoples' minds. In your organization today, how easy is it to institute changes in policy regarding quota increase requests and ensure that they are adhered to?
Who can perform quota increases in your organization? Classically, you have to effectively give Supervisor rights to the file system to allow someone to manage quota. Maybe you would like to give your Help Desk rights to grant quota increases, but you can't afford the security exposure in doing so and may, in fact, be prohibited from doing so in order to meet regulatory requirements. The result is that highly paid administrators are evaluating and granting quota increase requests. Or maybe you just grant the rights and hope for the best; but that's no way to live.
As you grant rights to manage quota, you probably have no way of limiting the amount of storage that may be doled out or in what increments. Wouldn't it be nice to set a ceiling on quota increases and possibly even simplify the process to the point of providing a single button to administrative staff to grant quota increases in measured amounts instead of giving them carte blanche?
Some organizations have given up on quotas altogether because of these challenges. Quota Manager helps address these challenges and takes control of your storage. It includes a Web-based interface to allow one or more specified users (for example a help desk administrator or support personnel) to increase quotas upon request, all without having to have rights to the file system. And you can set ceilings and increments in the policy that will be used by the Quota Manager.
File System Factory adds an attribute called cccFSFactoryHomeDirectoryQuota to the Organizational Person class. A help desk administrator or support person is given the Write right to this attribute on the users he or she manages, and this attribute enables them to be a quota manager on those users who have policies with quota management enabled. Take the following steps to use Quota Manager:
Enable and configure Quota Manager on each policy covering users where you want quota management enforced. Quota Management is enabled at the policy level. To activate it, simply go to the Quota Manager Control section found on the User Properties page of the File System Factory Policy. (See Figure 3.)
Figure 3: To activate Quota Management at the policy level, simply go to the Quota Manager Control section found on the User Properties page of the File System Factory Policy.
o Select the checkbox to enable the Quota Manager control for users covered by this policy.
o Optionally set a quota ceiling.
o Optionally set a defined quota increment.
Set up the rights for the user(s) to be the administrator of the disk space. To make an object the quota manager:
Select the portion of the tree where the users exist that the quota manager user will be allowed to manage. Remember these users must be managed by File System Factory policies with quota management enabled.
Set the attribute rights of the desired object for the attribute cccFSFactoryHomeDirectoryQuota to Write. This can be set for the whole tree, for an organization or organizational unit, or all the way down to an individual user, if desired. Because this feature uses eDirectory rights, the traditional inheritance applies, including the use of inherited rights filters.
Now your administrators may manage user disk space quota increases using the external Web-based Quota Manager Interface and have the policies govern their actions. Use the Quota Manager interface in the following way:
The quota administrator logs in to the interface using their common name.
After logging in as the quota administrator, enter the common name of the user who has requested a quota increase. Wildcards may be used.
The target user will then be selected from the resulting list. Each user will have a stoplight indicator showing what percentage of their directory quota is in use. The stoplights are based on directory space available and go from green to yellow to red based on percentages.
The resulting page is shown in Figure 4. Here the quota administrator will be able to manage the quota based on the policy settings. If an increment interval has been set, the manager will be able to add the default amount by clicking the button. If no interval has been set, the manager will be able to manually type in the new quota. In addition to the quota information, the manager can view file statistics based on file extension. Statistics include the size of the largest file, the number of files and subdirectories, and a list indicating number of files by extension, and sorted by size.
Figure 4: The Quota Manager allows role-based quota management without requiring Supervisor access to the contents of the file system. Policies allow for quota ceilings and defined increments.
* Originally published in Novell Connection Magazine
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.