Novell is now a part of Micro Focus

KLib: A Kernel Runtime Library

Articles and Tips: article

Russell Bateman
Senior Software Engineer
Novell Server-library Development
rbateman@novell.com

01 Aug 2000


In this AppNote, we will elaborate on this new library and discuss development-related issues and techniques that leave the domain of application programming (based on CLib or NKS/LibC) and enter the domain of code that is properly thought of as kernel-level.

Introduction

This article covers a new runtime library, KLib, in use on the NetWare operating system. KLib is implemented by KLib.NLM and supplies interfaces which were once the domain of CLib to NLM developers that don't want to load atop CLib. The K in KLib stands for kernel because these functions might have been provided and consumed at that level. In any case, the furnished functionality has no obligation to library concepts of thread binding, context, and ownership. KLib interfaces act only upon input parameters and, when making use of global or other data not passed in through the interface (an example is __ctype), treat such data as read-only. Consequently, KLib interfaces are completely reentrant and thread-safe. They are well-suited to drivers, stacks, libraries, and other low-level code. KLib provides them at this lower level for consumption by low-level code as well as applications.

What KLib Is

As stated in the introduction, KLib

  • is a kernel-level library in concept

  • is thread-safe

  • requires no library context to operate

In addition, KLib

  • loads in a protected address space

  • is the basis for many libraries on NetWare in that it furnishes a great number of common interfaces

  • is being delivered in service packs on NetWare as far back as NetWare 4.11. To find out which ones and when, please contact Novell Developer Support. However, since it mostly provides symbols that were formerly provided by CLib, existing applications need not concern themselves with existence (or non existence).

Developers of drivers, protocol stacks, other kernel code, and also libraries want standard C runtime interfaces like strcpy and yet don't usually want to write their code in terms of library context. (We'll touch on what library context is in a moment.) While they have been able to consume these interfaces from CLib without needing context, the way to do it was somewhat shrouded in uncertainty. Nowdays, these developers also want intrinsically thread-safe interfaces, interfaces that operate on stack-based data (called auto-class in C) to perform operations that other threads, calling the same interfaces, cannot corrupt. KLib satisfies these two conditions because the interfaces it furnishes don't require library context nor do they rely on static data to preserve state from one call to another.

KLib loads at the bottom of the heap, speaking of libraries, and offers its services to CLib, NKS, LibC, and to any other component running on NetWare, be it driver, protocol stack, library, or application. Load-dependence is an observed phenomenon on NetWare. In interest of anyone who cares, the accompanying diagram illustrates KLib's relationship to the two main library environments on NetWare. (Lib0 through NIT and CLib are the NLMs that make up the CLib library.)

Figure 1: .KLib's relationship to the two main library environments on NetWare.

Library Context

Also called thread context, thread-library context, or CLib context, it is the glue that makes of an NLM and its threads an application. While NKS/LibC (see the series of five articles begun in the September 1999 issue of Developer Notes and running until the following December) also has issues of context--they are virtually identical to CLib's--we will only speak of CLib context here to represent the issue.

Each NLM uses static data that, from the library's point of view, must be instanced for each of its consuming NLMs. The instancing begins by the application NLM's use of the library's prelude code, linked into the NLM using prelude.obj. The start- up code called from that object sets up the application's private data and then ultimately calls main. From then on, data like the current screen, the current NDS tree, open files, and so on, are all kept in account by the library. Each thread created, from the first one that runs through main to the last, also has specific data items, the best example of which is errno. This is all done through the magic of the library. However, the magic of the library doesn't come without cost, and that cost is:

  • linking with prelude.obj

  • maintaining (via the library) a considerable amount of data

Such magic occurs on other operating systems too, like UNIX and NT. But the most common programming environments are at the user level, so these issues of context are not obvious and do not intrude.

Drivers don't want to be load-dependent on a library like CLib because it was never certain that it would be available. Even in NetWare 5.0, where it loads very early and unavoidably, it still isn't available until after the system volume has been mounted. KLib doesn't suffer from this problem because it is distributed on the DOS partition with server.exe.

More importantly, drivers don't want to be bound to CLib because of context. Often, non-CLib threads run through them, and endowing these client threads with CLib context is expensive and usually unnecessary. Still, the interfaces in CLib that don't require context remains a mystery to most outside the library team. KLib remedies this situation by not promoting any interfaces that require library context of any kind.

Thread Safety

Because KLib interfaces do not expect thread context, they perform their function largely in complete isolation from concepts of NLM and thread ownership. This means that KLib interfaces are reentrant and thread-safe. The best illustration of this concern is the contrast between ANSI (ISO) strtok and POSIX strtok_r. They are prototyped as follows:

char *strtok( char *string, const char *charlist );
char *strtok_r( char *string, const char *charlist, char **stringp );

strtok is an ANSI string search function that breaks a string into a sequence of tokens. The delimiting characters are passed in charlist. When called, strtok traverses string looking for a character not contained in the set of delimiters; then it moves forward to find the end of the token (again, by skipping characters that aren't found among the delimiters), where it drops a null and returns a pointer to the beginning of the token. Subsequent calls to strtok are made passing NULL in place of string, which tells the function to use its internal pointer.

Therefore, strtok keeps a secret pointer into string so that each time it is called, it can move on from the last place it returned a pointer to. Consequently, the only reason it is thread-safe is because the library implementing it (CLib or LibC) carefully preserves this pointer in the calling thread's per-thread data. This is context. On the other hand, POSIX mandates strtok_r as a reentrant function that does not require thread context to safeguard against this problem. Instead, the address of a string pointer to be used in this role is passed explicitly each time as the third argument and is adjusted at the end of every call to point at the right place for the next call. The caller need only keep passing it until string has been parsed (strtok and strtok_r return NULL).

How to Write Applications to KLib

In fact, applications aren't written to KLib. We don't really want developers to consider KLib specially unless they are writing non-application code (this will be discussed next). CLib's and LibC's link import files will list all the interfaces in KLib; these libraries just won't really be supplying them. It is already the case that CLib's link import file list interfaces that are really in the OS. These are used by applications and CLib's file was the only way to discover them to the linker.

An application is linked as always by including clib.imp or libc.imp in its link definition statement. Merely think of KLib's interfaces as also being CLib's and LibC's. Don't think about where they come from. KLib has no more effect upon an application than splitting CLib into six or seven NLMs in NetWare 4.11 had. All the symbols listed in the link import file are guaranteed to be present when the application loads. In summary, use the import file corresponding to the prelude code linked into the application. If you use prelude.obj or nwpre.obj, then use clib.imp. If you link libcpre.o (or libcpre.obj), then also use libc.imp.

No load dependency on KLib is necessary nor encouraged. The application should only cause CLib or LibC to be auto-loaded.

How to Write Drivers and Other Components to KLib instead of CLib

klib.imp will be issued for the writer of code that is low-level, like a driver or stack, who may want to use it in the link phase of development to ensure that no contextful interfaces have crept into code to tie it to CLib or to LibC. For example, if the low- level code contains a call to strtok, using clib.imp or libc.imp would tell the linker that that symbol will be present at load time. Indeed, it probably will be, but since it requires library context and the driver code probably has none, the worst possible result would ensue. If instead, klib.imp were used, the linker would issue an error to the effect that strtok doesn't exist. The developer would then realize his or her mistake and replace the call with strtok_r. In summary, if you aren't linking a prelude object (one of nwpre.obj, prelude.obj, or libcpre.o), then link with klib.imp. This will result in a pure, low-level binary with no user library dependencies.

While KLib will most likely be loaded by the time any driver or stack is to load, it would be a good idea to auto-load KLib from low-level code anyway.

The following table might be useful in beginning to determine what import file (and therefore what library) to use in writing applications, drivers, or other libraries.


CODE TYPE
USING CLIB
USING LIBC
USING NO LIBRARY

Application

clib.impstart-up code from prelude.obj/nwpre.obj

libc.impstart-up code from libcpre.o

(unlikely)

Driver, stack or other code

(unlikely)own start-up code

(unlikely)own start-up code

klib.imp and os.impown start-up code

Library1

(hand-crafted)own start-up code

libc.impown start-up code

--own start-up code

Library2

--own start-up code

--own start-up code

klib.impown start-up code

In this table, Library1 is a library that wants to support directly both CLib- and NKS/LibC-based applications. Maybe it needs to maintain thread-specific data on behalf of both types of threads in which case, it could import symbols from LibC and CLib, though such importing might have to be specially handcrafted. Library2 is one that, while it might support applications indiscriminately of their fundamental execution environment, doesn't need to coordinate its activities with either library. It only wants to start itself up and shut itself down. It won't need interfaces from either of the two libraries, but it might make use of KLib.

KLib and NetWare 5 Memory

Even though KLib sounds like a low-level, kernel library, it nevertheless loads in a user-created, protected address space (ring 3) just as do CLib and NKS/LibC. Its interfaces are accessible from there and, for the most part, cause no ring transitions at all due to the simplicity of the majority of the interfaces. Ring transitions are fairly expensive and may contribute an estimated 5-7 percent degradation in execution performance on NetWare 5 for the application. The fewer times execution needs to marshal down to the kernel, the smaller the impact on performance.

As far as allocations go, KLib doesn't promote malloc or free since these require library context. Driver writers already know that memory allocation must be done using AllocateResourceTag followed by calls to Alloc (or AllocSleepOK) and Free.

Threading and Synchronization

In an effort to offer low-cost and highly efficient threading and synchronization interfaces, while encouraging portability toward Novell's next-generation platform, Modesto, we are offering NKS signatures for primitives that have heretofore not been available on NetWare, especially at the kernel level. At the end of the function list are the lighter versions of NKS interfaces that can be supported without context (or, can be used to gain all the functionality of the signature minus the part that depends on NKS library context).

An example of missing functionality in the threading is the inability to swap contexts, join threads, and perform other advanced tasks that would require library context to promote. Again, an interface to NetWare 5's multiporcessing kernel (MPK) hasn't been officially surfaced except for NKS', which is at too high a level for most drivers, stacks, and other low-level code. This removes that barrier and introduces that low-level code to a portable interface that offers the ability to move that code forward to Modesto and other operating environments if it makes sense.

Contents of KLib

Though NetWare developers are migrating to other programming environments such as Metrowerks' CodeWarrior or Edinburgh Portable Compilers' (EPC) tools, the symbol namespace remains an issue. These new environments permit the prefixing of symbols. Watcom 386 C/C++ does not. In our previous example, CLib has long exported strtok and strtok_r. In order to support these functions from LibC, they must appear prefixed in a link-import file used when an NKS/LibC application is being linked. In this way, strtok is actually imported as LibC@strtok although the developer's code only says strtok.

This is only to say that KLib exports a number of symbols that used to be in CLib and would naturally be in LibC too (for completeness and to avoid the confusion of making developers code partly to LibC and partly to CLib). The symbols KLib exports must not be prefixed. Indeed, that would defeat the whole purpose. CLib draws the full complement of symbols from KLib (that is, those that don't require context). And LibC draws the full complement of symbols from KLib. We've already explained how to avoid confusion in all this symbol importing in the above table.

KLib's contents are as follows:


ctype.h:

__ctype

isalnum

isalpha

isascii

iscntrl

isdigit

isgraph

islower

isprint

ispunct

isspace

isupper

isxdigit

toascii

tolower

toupper

dlfcn.h:

dlclose

dlerror

dlopen

dlsym

math.h:

_____huge

__double

abs

acos

asin

atan

atan2

atof

atoi

atol

atoq

cabs

ceil

cos

cosh

exp

fabs

floor

fmod

frexp

htol

hypot

j0

j1

jn

labs

ldexp

log

log10

ltoa

matherr

modf

pow

remainder

sin

sinh

tan

tanh

sqrt

xldexp

y0

y1

yn

nwieeefp.h:

fpgetmask

fpgetround

fpgetsticky

fpsetmask

fpsetround

fpsetsticky

setjmp.h:

longjmp

setjmp

stdlib.h:

bsearch

itoa

itoab

max

min

qsort

qtoa

rand_r

rotl16

rotl32

rotl64

rotl8

rotr16

rotr32

rotr64

rotr8

strtod

strtoi

strtol

strtoul

swab

swaw

system

ultoa

uqtoa

utoa

string.h:

ASCIIZToLenStr

ASCIIZToMaxLenStr

LenStrCat

LenStrCmp

LenStrCpy

LenToASCIIZStr

memchr

memcmp

memcpy

memicmp

memmove

memset

strcasecmp

strcat

strchr

strcmp

strcpy

strcspn

strecpy

stricmp

strindex

strlen

strlist

strlwr

strncasecmp

strncat

strncmp

strncpy

strnicmp

strnset

strpbrk

strrchr

strrev

strrindex

strsep

strset

strspn

strstr

strsub

strtok_r

strupr

miscellaneous:

GetReleaseVersion

GetServicePackLevel

RegisterKLibHandlers

NKS-lite:

NXCondBroadcast

NXCondDeinit

NXCondInit

NXCondSignal

NXCondTimedWait

NXCondWait

NXLock

NXMutexDeinit

NXMutexInit

NXMutexIsOwned

NXRdLock

NXRwLockDeinit

NXRwLockInit

NXRwLockIsOwned

NXRwUnlock

NXSemaDeinit

NXSemaInit

NXSemaPost

NXSemaTryWait

NXSemaWait

NXThreadBind

NXThreadContinue

NXThreadCreate

NXThreadDelay

NXThreadDestroy

NXThreadExit

NXThreadGetBinding

NXThreadGetId

NXThreadGetPriority

NXThreadSetPriority

NXThreadSuspend

NXThreadYield

NXTryLock

NXTryRdLock

NXTryWrLock

NXUnlock

NXWrLock

For the most part, these functions are strictly standards-based, including ANSI (ISO), POSIX, and IEEE. For 90 percent, they were in CLib until moved to KLib. All are quite useful even in the context of a driver.

Debugging and Functions Called from KLib, CLib, and LibC

This explanation of prefixing, on the part of LibC for its symbols, seems a useful comment to make should push ever come to shove and you have to resort to the system debugger. CLib does not prefix its symbols and LibC does. KLib does not. Thus, debugging KLib symbols is no different than for CLib. The header files will not tell you this, but the import files will if you understand what you are seeing.

Whether you are writing an application using NKS/LibC or CLib, strcpy comes from KLib. In the debugger, it will appear as KLIB.NLM|strcpy. setlocale comes from CLib, and the same function, LibC@setlocale, comes from LibC if you are writing to NKS/LibC. In the debugger, it will appear as LIBC.NLM|LibC@setlocale, but you won't always see the prefix in the debugger.

As alluded to previously, disassembling or otherwise referencing a function in the debugger will be impacted by the presence of LibC exporting the same symbols as CLib. The debugger resolves the most recently loaded symbols first. In consequence, if you want to disassemble setlocale, you will most likely get LibC's version (if it is loaded) since, for the foreseeable future, it will load after its older sibling, CLib. If this annoys you, you can stop loading NKS/LibC-based applications while you are debugging a CLib application on your test server. If this isn't possible, the following syntax will ensure that CLib's version get disassembled.

# u CLIB.NLM|setlocale

Off the subject, prefixes are arbitrary; they don't have to reflect the name of the library that uses them. Developers who wish to prefix are asked to register their prefix with Novell Developer Support, so they reamin unique.

What Might KLib Offer in the Future?

Along with the SDK interfaces specifically documented for drivers, KLib's interfaces make it easier for drivers, protocol stacks, and other kernel code, including other libraries, to start-up and shut-down without using CLib and tying themselves to that library.

What KLib might offer in the future is really a question of what driver developers want. Inside Novell, driver developers are asking for low-cost threading and synchronization functions. Some have suggested we include an interface to the file system that would be easier to use than the low-level NetWare OS APIs provide, something closer to POSIX open, pread, and pwrite, which themselves require library context. In the meantime, we think KLib fulfills a need that has been long-felt at Novell and, we hope, in the greater developer community.

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates