Novell is now a part of Micro Focus

Features of the Novell Kernel Services Programming Environment for NLMs: Part One

Articles and Tips: article

Senior Software Engineer
Server-Library Development

01 Sep 1999

First in a four-part series of articles which will review the programming features and functionality of the Novell Kernel Services (NKS) programming environment for NLMs. This article explores the purpose of this recently defined programming environment and examines the various interfaces surfaced by NKS.NLM.


Novell's Server-Library Development group ensures that developers have tools and technologies for writing applications that run as NetWare Loadable Modules (NLMs) on NetWare. This series of articles reviews the programming features and functionality of the Novell Kernel Services (NKS) programming environment for NLMs. This article explores the purpose of this recently defined programming environment and examines in some detail the various interfaces surfaced by NKS.NLM.

Part two focuses more closely on the programming concepts in the NKS environment, including discussions of multithreaded programming correctness, latency, and thread cancellation issues.

Part three completes the concepts started in part two by covering the remaining programming interfaces.

Part four discusses how LIBC sits atop NKS and comments on programming at both levels. In addition, it discusses differences in programming between LIBC and CLIB.

Novell Kernel Services for NLM Programming

In "The Future of Application Development on NetWare with NLMs," in this issue of Novell Developer Notes, I explain how the ten-year old CLIB programming environment had reached an unfortunate milestone in its attempt to continue to support existing NLMs coded to it and at the same time move forward to embrace support for future multithreaded programming technologies. I also explain the reasons why we plan to freeze CLIB in its current state (plus bug fixes) and move ahead with a new environment based on NKS plus a standard C library environment (LIBC) loosely referred to as NKS/LIBC. Rather than reproducing those explanations here, I would like to revisit the history and motivation behind this new environment.

For the purposes of our discussion, LIBC on NetWare (LIBC.NLM) remains essentially identical to age-old CLIB.NLM. It will contain the ANSI interfaces and include as many POSIX interfaces as appear relevant, which should include most interfaces already long-surfaced by CLIB, name for name.

However, NKS surfaces new interfaces that are more complete, more correct, and more numerous, and replace the entire threading model of CLIB. What's more, all the interfaces in NKS lead users to a more correct implementation of multithreaded code not that the programmer cannot make mistakes with these interfaces, but that the interfaces are complete and semantically correct. NXThreadJoin, an interface providing fuctionality impossible in CLIB, offers an example of completeness; NXThreadSuspend offers an example of correct semantics, not allowing itself to be suspended as did CLIB's SuspendThread.

Originally, NKS was conceived as a portability layer (NPL) posited to support a broad set of architectural underpinning for all of Novell's services and applications written in C or C++. Whatever the internal use of this environment, the API set became more commonly associated with the next-generation platform code-named Modesto, where it is the interface to the operating system kernel. The API set was redubbed with its present moniker.

As noted in the other article, NKS promotes a set of threading interfaces that are more sophisticated and correct than those in CLIB. Indeed, it was decided to adopt the erstwhile NPL interfaces in replacement for CLIB's after examining other alternatives such as POSIX pthreadsand UNIX International. These were the most important interfaces attracting the Server-library Development team in addition to the synchronization primitives which could replace and augment the humble semaphore, the only such interface ever made available in NetWare or CLIB before NetWare 5. Nevertheless, for completeness and to provide an upward migration path to Modesto, however different from NetWare this platform will ultimately be, it was decided to implement not only the entire NKS primitives but also the NKS virtual machine as the basic process model.

The NKS Virtual Machine

The NKS virtual machine defines a containment within which an application or service may be hosted. The virtual machine supports a multithreaded execution environment with the threads in the virtual machine sharing the following system resources: memory, the native platform security context, open file descriptors, file and record locks, and a pool of anonymous threads. The virtual machine can be transformed into a deterministic (controllable) state machine by disabling preemption and limiting the concurrency. In this mode, replay-based fault-tolerance strategies can be supported such as SFT III. The virtual machine as defined here is expected to map naturally onto the typical task or process abstractions found in modern operating systems.

While a virtual machine is a key component of the NKS architecture, to enhance portability, only a small set of operations are defined for managing the virtual machine. These are as follows:


Disable preemption in the virtual machine


Terminate the virtual machine


Get the number of on-line CPUs


Get the maximum number of threads that may be created in the anonymous thread pool


Set the limit on the number of threads that may be created in the anonymous thread pool


Transform the virtual machine to be a single threaded machine

NKS does not surface APIs for creating virtual machines or APIs for supporting intervirtual machine communication. This decision has been motivated by the need to keep the abstractions portable without sacrificing important platform-specific optimizations. All intervirtual machine communication will be supported via libraries atop NKS like LIBC, which is not discussed in this article.

NKS Thread Programming

The thread is the unit of execution in NKS. A thread is a sequential flow of execution control and is mapped onto the underlying NetWare unit of execution-also a thread, but differing depending on the version of NetWare supporting NKS. (There happen to be two very different types of threads used in NKS depending on the underlying version of NetWare. In NetWare 5, the thread is created by the new, multiprocessing kernel [MPK] whereas old versions host NKS using legacy threads.) A thread virtualizes the processing element of the underlying platform, and, on a multiprocessing platform, these threads can execute concurrently. The runtime state of a thread is defined by the context (discussed below).

The two classes of threads defined are application threads and anonymous threads. Application threads are threads explicitly created by the application, while anonymous threads are automatically created and managed within the virtual machine to support the NetWare notion of work.

One of the important transport optimizations possible when the communicating virtual machines (or a given virtual machine and some other external component in the system) are hosted on the same physical node is the use of a client thread to execute the code in the server. Anonymous threads support this optimization. A thread from outside the virtual machine can migrate into the virtual machine and become an anonymous thread in the virtual machine for the duration of the transaction.

NKS supports priority-based preemptive scheduling. The priority of a thread is the priority of the context it is currently bound to high, low, or medium. Each thread is assigned a unique identifier whose scope is local to the containing virtual machine.

A context abstracts all the runtime state of a thread that this context is bound to. Encapsulated by the context abstraction are execution stack, execution priority, hardware-specific register state, context-specific private data area, and a flexible infrastructure for managing context specific data including key-value pairs. This clear separation of the execution vehicle (the thread) from the work being executed (the context) provides an efficient and scalable infra structure for supporting services built as finite state machines.

A context gets bound to a thread as part of thread creation, as part of scheduling work to be executed by an anonymous thread, or by explicit binding of the context to a thread. A bound context gets unbound by explicitly binding itself to a different context, by the context in question being a work context and the work being completed (as signified by the return from the start function of the context), or by the hosting thread exiting. NKS guarantees that when it changes the binding status of a context as a result of the completion of work (or delayed work), or when the hosting thread exits, the state of the context is reset to its initial state.

NKS contexts can be created for use either as work contexts or normal contexts. Work contexts can only be bound to anonymous threads, while normal contexts can be only bound to application- managed threads. NKS treats work elements as first-class abstractions with semantics comparable to a regular thread; the scheduling priority, stack size and private data area can all be specified for a work context.

NKS also supports a lightweight work abstraction with semantics comparable to the traditional NetWare work-to-do abstraction. In addition to the simple private data area associated with a context, per-context data may be associated with one or more keys. While these keys and their associated data can are conceptually thread specific, they are in fact bound to the context and follow it rather than the thread.

This distinction between the running thread and context data is an important difference with other thread models but especially with the model in force on NetWare since its inception. The interfaces defined to manage the threads abstraction are as follows:


Cancel a previously scheduled lightweight work


Cancel a previously scheduled work


Allocate a context


Destroy a context


Get the identity of the currently executing context


Get information on the specified context


Initialize a previously allocated context


Allocate a key for the calling thread/context


Delete the specified key


Retrieve the value associated with the key


Process the specified set of interthread interrupts


Schedule work to be executed at the specified time. The API for deleting a scheduled work can also be used to cancel delayed work


Schedule lightweight work to be executed by an anonymous thread


Schedule work to be executed by an anonymous thread


Associate the specified value with the specified key


Bind the thread to a specific CPU


Continue the execution of suspended thread


Create a thread


Delay the invoking thread


Terminate the thread


Get the CPU binding information


Get the thread ID


Get the thread/context priority


Get the thread/context private data pointer


Interrupt the specified thread


Check to see if the thread has been interrupted


Wait for the specified thread


Set the execution priority for the specified thread/context


Suspend the execution of the specified thread


Get the context currently bound to a thread and bind a newly specified context.


Yield the CPU

NKS Memory Management

NKS surfaces both typed as well as untyped memory, though there are peculiar semantics across the NetWare kernel and user-address spaces. Untyped memory is potentially pageable unless it is explicitly wired down using NXMemCtl. Hence, to be portable, services are expected to manage their memory needs carefully. Virtual memory allocated and not locked down by an NLM running in ring 0 (the kernel) opens execution to preemption. All execution is open to preemption when the NLM is running in a user-address space.


Get the cache line size


Get the platform-specific page size


Allocate typed memory


Perform control operations on the specified memory


Free memory


Change the size of the memory allocation


Allocate untyped memory


Free memory

NKS Synchronization Services

NKS supports a comprehensive set of synchronization primitives. The composition of this set was based on an evaluation of the synchronization needs of some of our services that might be definitively ported to NKS. Mutual-exclusion and read-write locks, condition variables, and semaphores are supported. Spin locks were rejected because NKS is essentially a user-level package. Barriers were rejected because other locks suffice to implement the concept.

Mutual-exclusion locks are to be used to serialize access to a shared state. To enhance concurrency when the shared state is mostly read-only, we also support read-write locks. Because NKS is designed to be hosted both within the kernel as well as in user space, how a thread sleeps awaiting a contested lock is not specified. To deal with locking-related deadlock issues, NKS supports a hierarchy with respect to locks. An application is free to specify the order in which it expects to acquire the application-defined locks, and this order is asserted by NKS in debug mode.

To support performance tuning and debugging, the system also collects statistics with respect to locks (again, if appropriate, compilation switches are defined). Given the frequent use expected of some of these locks, the function names have been appropriately abbreviated instead of prefixing them with mutex or RwLock.

Condition variables can be used to synchronize threads based on arbitrary external state. Since the state used to synchronize is external to the condition variable, they can be used to solve a variety of synchronization problems. Classical counting semaphores can be used to control access to a set of identical resources. Here are the synchronization primitives offered by NKS:



Statically allocate and initialize a mutex or read-write lock information structure


Acquire the mutex


Allocate a mutex


Deallocate a read-write lock


Deinitialize the mutex


Initialize the mutex


Check to see if the mutex is owned


Try to acquire the mutex


Release the mutex

Read-write Locks


Statically allocate and initialize a mutex or read-write lock information structure


Acquire the lock in the read mode


Allocate a read-write lock


Deallocate a read-write lock


Deinitialize the read-write lock


Initialize the read-write lock


Check to see if the read-write lock is held in the specified mode


Unlock the read-write lock


Try to acquire the lock in the read mode


Try to acquire the lock in the write mode


Acquire the lock in the write mode

Condition Variables


Allocate a condition variable


Wake up all the threads blocked on the condition variable


Deallocate the condition variable


Deinitialize the condition variable


Initialize the condition variable


Wake up one thread blocked on the condition variable


Wait on a condition variable no more than the specified time


Wait on a condition variable



Allocate a counting semaphore


Deallocate a counting semaphore


Deinitialize a counting semaphore


Initialize a counting semaphore


Release a semaphore


Try to acquire a semaphore


Acquire a semaphore

NKS Time and Time-out Services

Both one-shot as well as cyclic time-outs are supported. In addition to the function listed below, the thread interface, NXScheduleDelayedWork, works on a time-out basis.


Cancel a scheduled time-out


Get time in seconds or microseconds since1january 1980


Get the interval between two consecutive clock interrupts


Schedule a function to be executed at the specified time

NKS File and Directory I/O

NKS file and directory I/O interfaces collectively provide a common minimal set of capabilities for using the file and directory functionality available on any platform (not just NetWare). This may tend to limit the depth of the interfaces in comparison to those usually subscribed to by NLMs. These capabilities consist of creating or deleting file and directory objects, navigating a hierarchical name space of directories to find files or directories of interest, reading from and writing to the files, and obtaining status information on files and directories.

The complete set of features of the legacy Novell File System (NFS) as well as the new Novell Storage Services NSS is not available through NKS. Appropriate interfaces (including NSS interfaces) are indeed used to access the directory or file opened. Instead, the NKS approach with respect to the delivery of the file-system services is to split out certain common operations that can be expected to be found on all platforms and file systems including NetWare and NSS, and surface them as NKS file and directory functions. Any Novell service that currently relies on deeper-provided capabilities can continue to use those implementations on the host platform. This mode of use does not require the use of the NKS interfaces described in this section. The remaining common operations available on the local file system (i.e., the platform's native file system) are accessed through the interfaces described here.

NKS is not equipped with such namespace composition operations as MOUNT, UNMOUNT, or REMOUNT, or with support for selecting from a multiplicity of naming syntaxes. It is designed largely so that the available file system can be used for performing simple read, write, create, delete, search, and file-synchronization operations.

Operations that enforce specific caching modes, such as read-ahead or controlling the length of time data is buffered for reading or writing by the local file systems, are excluded from this interface. This is based on the assumption that the services using this API will most likely use the local file system for very limited purposes and not be affected by, nor affect the performance of the local file system.


Rollback to the last committed state


Cancel a previously scheduled asynchronous I/O operation


Clear the transactional attribute for the file


Close a directory open for searching


Close an open file


Commit the transactions (writes) against the file


Create a new directory


Create a new file or rewrite an existing file


Enable caching on file writes


Disable caching on file writes


Perform file/record locking operations


Get the attributes of a file/directory


Get the length of a file


Get the component separator character


Check to see if the file is transactional


Open the specified directory for searching


Open the specified file


Atomically position the file pointer and read


Issue an asynchronous read operation


Remove the specified directory


Remove the specified file


Read the contents of an opened directory


Set the path separator character


Set the transactional attribute for the file


Atomically position the file pointer and write


Issue an asynchronous write operation

Transport (Communications) Interfaces

Specific communications interfaces were designed in NKS, but the feeling for now is that WinSock will become the common transport for communication on NetWare, as on other platforms where NKS will be found. WinSock is already available for NetWare 4.11 and, indeed, the NKS NCP requester uses it.


These are the interfaces that make up Novell Kernel Services on all platforms to which it has been ported. Most of the detail covers all platforms, and I covered very little NLM-specific detail. As noted in the introduction, part two in this series discusses writing NLMs in NKS, with particular attention paid to the thread-programming model.

Part three undertakes to cover the concepts of programming to the remainder of the library.

Part four talks about the relationship between NKS and LIBC, which sits atop the former, and the differences in programming to LIBC and NKS, making relevant comments about and comparisons to programming to CLIB.

* Originally published in Novell AppNotes


The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates