The Future of Application Development on NetWare with NLMs
Articles and Tips: article
Senior Software Engineer
Server Library Development
01 Sep 1999
Explores the tools, technologies, programming methods, SDKs, and other issues facing NLM developers. Also shows a road map for solutions in the SDK to reflect support for technologies in the operating system.
- Introduction
- NetWare as an Application Platform
- Position Statements
- Technology Transition
- Future and Conclusions
Introduction
This article will explore the tools, technologies, programming methods, Software Developer's Kit (SDK) and other issues facing NLM developers. It will also show a road-map for solutions in the SDK to reflect support for technologies in the operating system. These issues and solutions will be discussed in an historical framework useful to the long-time NLM developer, but with enough clarity and detail to benefit new developers.
However, this article is not intended to explore the historical dispute between preemption and non-preemption, or to explain the lack of a POSIX execution environment to which applications could have been ported. I am pleased however, to see the great C/C++ tools that are now available for NetWare.
This article is written to ensure that developers have the tools and technologies they need to write applications that run as NLMs on NetWare as we move toward the future and 6 Pack. 6 Pack is the code name for a near-future release of NetWare that will include a number of enhancements. The enhancement that most affects the developer community and influences the content of this article is increased scalability through better multi-threaded and multiprocessing techniques.
NetWare as an Application Platform
NetWare wasn't originally conceived as an application platform. However, by reason of its performance, as well as its proximity to the heart of the network, the suitability of NLM programming for network applications, is undeniable. It is regrettable that NetWare hasn't evolved into a prime application environment. There are several reasons for this.
Lack of Clearly-defined Distinction Between Kernel and User Space
In fact, until NetWare 5, the only distinction was in the design and architecture of the application. NLMs were conceived originally as easily programmed and easily loaded kernel extensions. Compared to the Value-added Processes (VAPs) of NetWare 2, NLMs were far more simple to write, load, and unload.
NetWare 4's greatest technological improvement was, of course, the Directory. But NetWare 4 also contained improvements that became somewhat over-looked and appeared as little more than bullet items on a feature list.
One of these improvements was the protected address space. While this technology wasn't all that its replacement in NetWare 5 would be, it did provide a small measure of security. The protected address space permitted an application to be placed in a single, separate domain from the kernel. This only allowed you to "quarantine" the application in the case of an ABEND, leaving the rest of NetWare's activities, particularly file and print, alone. The application could not be reloaded and the server had to be shut down to clear the faulted address space.
Nevertheless, the strict isolation of an NLM in a user-domain would become a reality in NetWare 5. With the release of NetWare 5, correctly-written NLMs could be loaded into one of an infinite number of ring 3 address spaces as easily as in ring 0. In its own address space, an NLM can behave as badly as it wishes; in the worst case, the address space will fault and come down. It can be reloaded and, in fact, a method by which the NLM can be automatically reloaded is available.
Unfortunately, however, ring 3 isn't the default for loading NLMs and many NLMs were not written to load in ring 3. Many NLMs were written using direct calls into the OS, especially undocumented calls of the OS, as well as hooking data structures directly. These cannot load in ring 3. If the applications were rewritten to load in ring 3 as they do under Windows NT and UNIX, and the libraries (CLIB) were modified to expect and take advantage of this, a more application-like environment for programming could become the norm. For now, the libraries are written to permit NLMs to load indiscriminately in the kernel or in their own address space.
Lack of Support for Common Inter-procedural Communication
NetWare core engineers didn't see the absence of specific interprocess communication (IPC) support as a concern, since sharing memory and other resources isn't problematic in the same NetWare address space. However, there is a problem here in that NLMs have no reliable or formal means by which they can share memory addresses, file descriptors, or other objects.
As library developers, we have been stymied in our attempts to reconcile this for a number of reasons. IPC is a natural need arising out of the spawning of one code resource (process, etc.) from another. Since the NetWare loader is singly-threaded and since loading an NLM is a comparatively intensive process (as compared to UNIX whose loading was specifically engineered to do this quickly) the utility of doing this defies any explanation except that of being the object of a code port.
In other words, the UNIX fork model was created in the years before multi-threaded programming concepts became widespread. Multi threaded programming is a better solution to the problems than the fork model was. NetWare, from the very beginning, was designed as a platform to host multiple threads in a kernel-extension environment. It was not based on the UNIX process or user model.
We are still looking for a model that will promote the use of small, intense code resources like CGIs without requiring such architectural changes that would discourage anyone from porting them to our platform. Our search may be in vain. When it comes down to it, fork is only desirable because it is so pervasive. Providing fork on NetWare would, in theory, make porting to it much easier. But attempting to impose the concept on NetWare creates so many problems that it seems doubtful it will ever happen. However, we have a few good ideas about IPC and you start to see them in 6 Pack.
The above sections have discussed the problems that have lead to the credibility gap surrounding NetWare as an application platform. What NetWare is, is an excellent protocol engine and it has become an excellent Java execution and Web server platform. It is superb at file, print, and as a host for Novell's Directory Services. It has also excelled at a number of other things. However, while most of us have little trouble writing killer code for it, to say that writing applications for it is as easy as for other platforms like UNIX and Window NT would be lying. This is because writing applications for NetWare rarely means doing it there first, but instead, porting them from another, POSIX- or POSIX-like platform and the execution model simply isn't the same.
The purpose of this article is to tell you what we are doing to make your NLM programming more productive if not always easier. Therefore, I'm going to make a few statements about how things are today and how they are going to be in the near future.
It's our fault that the developer community hasn't received updated vision statements about low-level concerns such as those discussed in this article. Many of you have come to BrainShare expecting to get better vision, which you have got on high-level issues such as the availability of new developmental environments, but that vision has been more of a snapshot than a complete story on the nitty-gritty of evolving programming practices and other low-level issues. So, now for the answers.
Position Statements
Tools
Watcom and their tools have enjoyed a long relationship with NLM programming. Watcom C/C++ is a competent development tool that has been used to create 98% of the binaries that run on NetWare today including NetWare. Nevertheless, these tools will no longer be issued by Watcom's parent company, Sybase, Inc. Stepping up in their place will be tools produced by Metrowerks Corporation and Edinburgh Portable Compilers, Ltd. (EPC)
C/C++ Compilers
While I haven't used either of these environments much yet, I know that EPC has a long history of providing such tools to operating system vendors including UNIX OEMs and is presently creating the compiler Novell will use for generating Merced code in our Modesto project. I was a CodeWarrior toward the end of my Macintosh programming career when Metrowerks' IDE virtually eliminated the competition in development tools on the Mac platform.
The Metrowerks and EPC tools promise to rectify the long-standing problems of writing NLMs in C++ by using Novell's runtime libraries (CLIB, et al.). This should resolve some of the confusion that developers experienced with Watcom C++ in mixing in parts of their own. Sorting out header file problems was also confusing, since Novell ships its own ANSI and POSIX headers that relate to its libraries and yet, if you were writing C++ code, you needed Watcom's.
It is possible to write DLLs in Visual C++ or using Borland's development tools, and then load these DLLs on NetWare. Obviously, you cannot use any Win32 APIs if you want to load on NetWare. For more information refer to the NetWare DLL Developer Components on the Novell Developer Kit at http://developer.novell.com/ndk/dllcomp.htm.
In summary, the C/C++ compilation environments for writing NLMs are:
Metrowerks' CodeWarrior Professional and CodeWarrior for NetWare
Edinburgh Portable Compilers (EPC)
DLLs written in Visual C++ or Borland (Inprise) C++Builder
Assemblers. Some time ago, the PharLap assembler and other tools became part of a larger, more expensive package of tools that are of no particular use in writing NLMs. I am still using my last licensed copy of that tool, but this must change. Any assembler that produces COF objects can be used and I am investigating a replace ment. I must unavoidably use assembly in a few places in the libraries I produce. I don't imagine applications have much need for it, but it can be used if you can find a reason to do so.
Debuggers. Our debugger story continues to be twofold. First, the NetWare System Debugger is ubiquitous always ready at the system console. In addition, RDebug is our semi-remote, source-level debugger for C and C++. It works over either a serial cable or a dedicated NE2000 LAN. But repeated action is required at the server to engage it, so it isn't purely remote. The server- side piece of RDebug is being enhanced as an interface nub for potential use by other remote, source-level debuggers. The only requirement for using RDebug is that objects and links be produced for CodeView debugging.
Linkers. Linking hasn't always gone hand-in-hand with compiling for the NLM environment. For many years I used our own NLMLinkX (distributed for free with the SDK ), even though I compiled with Watcom. However, NLMLinkX cannot preserve the debug records needed by RDebug, so when RDebug became available, the libraries switched over to the Watcom linker (near the end of NetWare v4.11 development), and .SYM files became available. If using tools from one of the current vendors (EPC and Metrowerks) were not a big enough argument to switch away from this WLink, the fact that it doesn't support symbol prefixing is. The benefits of using symbol prefixing will be discussed in a moment.
In summary, plan to use the linker accompanying your C/C++ development environ ment including DLLs. Incidentally, when a DLL is loaded, the NetWare loader doesn't convert it to an NLM, but loads it as is. Consequently, there are no "cross- format" incompatibilities.
Documentation. I like the direction documentation has taken at Novell. I had a very difficult time using the on-line documentation that superceded the original printed manuals which I liked very much. Now, however, I have a place I can go no matter where I happen to be regardless of whether I have the SDK CD in hand or not. I can walk up to any workstation anywhere in the world and, in a matter of seconds, be looking at the peculiarities of some function or data structure. The technical documentation team that publishs the information you need to use our libraries and technologies, is continually improving both the structure and accuracy of the documentation.
The Software Developer's Kit (SDK)
The goal of Novell's SDK is to get the latest information, executables, headers, import and symbol files, and objects out to developers as quickly as possible and with as little confusion as possible. We haven't always achieved this goal. Because the content of the SDK is produced by developers in places far-flung from Provo (like Bangalore, India and San Jose, California), and because there are legacy parts to it, we occasionally ship an extra header or other file that should not ship any more.
Some mistakes have been admittedly less subtle. For example, last year we mistakenly shipped NPL.NLM with NetWare 5 making some of you think that it had become an official NLM even though it has never left the "futures" area of the SDK. We are making every effort to rectify this situation, but individual mistakes are likely to happen again given the great complexity of our products. We'll apologize for any confusion and inconvenience this may cause. One of the reasons we didn't quickly ship a successor to NPL.NLM is because we want to make sure that we get it right.
Technologies
Since our first attempt at symmetrical multiprocessing in NetWare 4, a number of important technologies have hit NetWare. These include user-address domains, multiprocessor support, virtual memory, Java, distributed computing, Console One, an HTTP portal we demonstrated at BrainShare, DLL loading, a new file system, and others I don't have room to list here. I want to discuss the ones that directly affect C or C++ developers, since those are the developers I support. These new technologies present challenges and opportunities for us and for third-party developers.
Multi-threaded and Multiprocessing Programming. When I said that from the very beginning, NetWare was designed to host multi-threaded applications, I stretched the truth a bit. This statement isn't exactly true given today's concept of multi-threaded programming. In fact, there are certain aspects of NetWare prior to version 5 that may have caused a programmer to make serious mistakes in multi-threaded programming. This was caused if not by the APIs and documentation themselves, then by observation of the way many of our own applications and code are written. I want to address these phenomena, but first you'll need some definitions for terms I will be using. These terms include concurrency, parallelism, thread-safe and unsafe programming techniques, and preemption.
Concurrency is the illusion that two things are happening at the same time. When terminals were attached to mainframes in the old days, employees worked as if they got to use the computer at the same time as their neighbor and, indeed, rarely gave any thought to their neighbor's existence (until the neighbor began a heavy compilation bringing the mini to its knees). The fact that they weren't really working concurrently was unimportant, because their environment behaved as if it was so. With threads executing, one at a time on one processor, but performing separate tasks, it is the same illusion. It is the theory, too, that work may be designed to benefit from tasks performed with apparent simultaneity.
Parallelism is both the notion of two or more things happening with genuine simultaneity. This would seek to achieve roughly the same result as on a fictional planet where two humans working on bearing the same baby can really get it done in four-and-a-half months instead of nine! It is only possible to do this in computers when they have at least two processors. Now, even with more than one processor, if the work cannot be broken down into steps that can happen in parallel, then no benefit can be had, sort of like trying to gestate a baby it's going to take nine months regardless of any attempt to speed it up. So parallel programming using multiple threads is not always the solution.
Thread-safe code is code that can perform its task independently and without error or contamination of the data on which it operates. It must be able to do this while being executed by more than one thread at a time. Conversely, thread-unsafe code is most of what is out there and for that reason Novell has employed my team the last few years to rid the libraries of it.
Preemption is the notion that a thread or process may have control of the processor in performing some work, but the operating system may interrupt this work and set it aside to give a chance to other threads or processes to perform their work. Traditionally, NetWare hasn't been preemptive, instead, it's non-preemptive. This has been of great benefit, but also a source of mischief.
Non-preemption in NetWare has led to the requirement that programmers be careful about how long they monopolize the processor. Execution only yields the processor in the case where it reaches a blocking point; such as requesting a disk I/O, or requesting a block of memory that cannot be had from the allocation pool. A programmer might also explicitly yield the processor by calling a threading primitive such as CYield or ThreadSwitch. In fact, Novell Labs created tools that examined the behavior of NLMs slated for certification and insisted on a threshold above which the NLM was deemed to hog the processor.
So programmers sprinkled explicit yields in their code to meet the threshold restric tion. As processors grew faster, more work could have been performed under the same threshold, but the yields could not magically disappear in the right ratio from the code without recompiling and relinking. In time, this "performance" requirement began to hamper performance not only of the certified NLM, but of all other work being done on the server. Conversely stated, this requirement asserts implicitly that the measure of performance of a server is the number of context switches it can perform in a second rather than the amount of real work it can get done. And it is obvious that while a processor switches contexts, it is not performing useful work. This is the first bit of mischief that non-preemption led to, but it is easily solved by backing out the explicit yields.
The second piece of mischief that non-preemption created for programmers that weren't paying attention (which was pretty much all of us until 1995 when we started thinking about multiprocessing) brings us back to our discussion of multi threaded and multiprocessing programming.
Programmers learned that as long as their code had control over the processor, no other thread could corrupt a data structure. Non-preemptive, uniprocessor execution provided a sort of implicit lock on code and data structures which proved very convenient. But it also proved to be the down-fall of much code as soon as another processor was added. With a second processor available, other threads could execute simultaneously touching the same data structures whose implicit protection had been previously ensured by a single-processor non-preemptive environment.
Multi-threaded programming on a single processor machine uses concurrency to simulate a parallel environment. This is what multiple threads were used for on Net Ware and other thread-based systems. But, when true parallelism is introduced by adding more processors to a box, all of a sudden the correctness of the multi-threaded solution (the architecture and implementation of an application) is put sorely to the test. For example, the libraries, which create and destroy threads on behalf of applications, were counting on the primitive non-preemptive environment to perform thread clean-up and on a predictable execution order according to which actions like stack-freeing could be done safely.
This brings me to a corollary I once heard. It states that "threads will run in the most evil order possible" In practice, therefore, multi-threaded programming is coding in such a way as not to depend on execution order to protect one's data structures and other resources against corruption by sibling threads that manipulate them.
NetWare Versions. Since the end of the 286 days, NetWare has evolved through a cycle of versions including 3.0, 3.11, 3.12, 4.0, 4.01, 4.02, 4.10, 3.2, 4.11, 5.0, 4.2 and, within the next year, two more. Each of these revisions has had the purpose of adding to the technological superiority of one version over another or, in the case of some, adjusting features in the previous version (bug fixing to be sure).
The major technologies introduced in some versions have made the job of providing runtime libraries for NetWare interesting, to say the least. If the libraries (CLIB) were only user libraries consisting of fopen, strcpy and printf, they would be a simple collection of code long-ago written and debugged that would never need to be touched. However, these libraries have extended the operating system to provide a programming environment and are in themselves something of a kernel built on top of the NetWare kernel.
When, for reasons to do with porting NetWare to other platforms a few years ago an effort assassinated by the spectacularly performing Pentium chip, the monolithic CLIB.NLM was split into parts, it became more apparent what went on in that mere library. The thread programming model is hosted in Threads.NLM; the server application's version of a requester by Requestr.NLM; basic NLM program ming services including all of POSIX we could fit in by NLMLib.NLM; the legacy NetWare Interface Tools similar to those originally written for DOS were separated out into NIT.NLM; and CLIB.NLM was left with only the standard ANSI C programming interfaces. The floating-point emulator needed on x86 systems in case no chip was resident, is all but useless now since hardware is a given. FPSM.NLM and Lib0.NLM were introduced to run in ring 0 of NetWare 5. These NLMs are used to bolt the rest of the libraries to NetWare whenever they are loaded in ring 3.
These libraries evolved into keeping all NLMs ever written to CLIB.NLM running no matter what the underlying NetWare kernel decided to change. I won't delve into specific problems because our success is only impressive to those who knew about and reacted to the changes. Although there were some hiccups along the way, it is still the case that some NLMs written to load on NetWare 3 will run on NetWare 5.
Where this becomes interesting to you, obviously, is where we can no longer support the direction we think you will want to take your NLMs with the present library. We call this direction MP-aware, MP-enabled or MP-scaleable. I like to think of it merely as multi-threaded. If your NLM is written to correct multi-threaded techniques (not the ones you learned from us in the early days as already noted), then you have very little to do to get your NLM to perform almost twice as much work on a two-processor machine as on a single-processor one, or eight times as much on an eight-processor machine.
This became a concern for our group, the library team, as soon as it became apparent that NetWare's new multiprocessor kernel (MPK) was going to fly so fast that these problems with multiprocessor awareness would show up in applications. Not only did we have a lot of work to do to correct the library, but you needed a new threads package to program to. It also occurred to us very quickly that we couldn't change the current underlying threads package without breaking at least half of the existing CLIB-based NLMs. So we investigated a number of solutions and contemplated threads interfaces from pthreads to UNIX International (UI) to Win32 APIs for thread programming.
But this begs the question why we needed a new threads package. The main reason was the inability of old CLIB applications to scale well across multiple processors. This is a problem created by our insurance policy against breaking legacy applications. CLIB's BeginThread calls CMakeProcess in NetWare to create and launch a new thread. This is how all code written to NetWare 4 or lower be it driver or application starts a new thread. NetWare 5's CMakeProcess is set up to run any thread, that isn't explicitly set up as safe to run on any processor, on processor 0. In fact, a thread begun with this interface instead of MPK's kCreateThread can migrate off processor 0 under certain circumstances, but then, it encounters another problem that afflicts new threads too. There are numerous non-thread-safe interfaces throughout NetWare, notably in the legacy NetWare kernel primitives.
When rearchitecting the libraries to be thread-safe, we found a number of problem interfaces. Some, like the legacy file system, were set aside since they simply funnel the executing thread back to processor 0, perform their thread-unsafe activities, then return the thread to its processor of origin when finished. Others, like the IPX interfaces, were inherently thread-unsafe and the library explicitly calls for them to be funnelled to 0 by using a tool that explicitly identifies them to the kernel. Still others must only be called from code executing on processor 0, so the library was forced to funnel them using an the explicit MPK call that performs this task.
It is this third class of funnelling that is so critical. Upon examining the semantics expected by calling applications, it became apparent that the applications themselves were counting on thread-safe behavior that only was incidental to the existence of a single processor or funnelling. Thus, merely rewriting to kCreateThread wasn't a great option, since a number of NLMs would still be broken.
Incidentally, one of the Core OS engineers wrote bench-marking tools that demonstrated the impact of several problems afflicting NLMs. Last year, I published an article on one of these problems making calls through function pointers on the Pentium processor and our solution to it. See Scheme for Optimizing Calls to Functions through Pointers on Pentium Processors in the August 1998 issue of Novell Developer Notes.
Another problem we discovered using this same tool was that an NLM that funnels to processor 0 frequently to perform its work, runs slower on a multiprocessor box than when run on a uniprocessor system.
To return to the thread package: because of CMakeProcess, because there are few CLIB threads interfaces, and because of the incorrectness of those interfaces(notably a thread's ability to use SuspendThread to suspend itself and the mere existence of Enter- and ExitCritSec), it became apparent that our NLMs needed a new package at the user level. Instead of pthreads, which we reviewed and even implemented as a prototype, or still others, we created our own package. Although this package had a number of portability goals at the time (early 1997), it was to become the interface to our next-generation operating system kernel, code-named Modesto.
Why didn't we choose an industry standard package over creating our own? This was a difficult decision, and the biggest reasons would have to be: first, we feared we couldn't adequately support pthreads semantics to make it worth adopting pthreads, and second, we needed some peculiarities that were part of the package we did create to permit the implementation of some very special applications. In particluar, the distinction between the thread and its context or state was an important one that wasn't made in standard interfaces. (This distiction is discussed in another article on NKS concepts and functionality.)
NPL.NLM. This NLM was the prototype of the new threads package hosted on Threads.NLM (CLIB). It was a prototype in that we knew that to overcome the problems that were the real reason we needed a new threads package, we had to set it atop MPK directly without which, applications would suffer from the scaling problems already noted. However, this was not done directly in the prototype as many useful library functions require context to work and NPL would not have been very attractive as a prototype even for demonstration purposes.
We placed it in the "futures" section of the Fall `97 SDK and planned to discuss it at BrainShare. Due to session length restrictions, NPL was discussed only minimally in one of my presentations on NetWare Kernel Programming at that conference. It wasn't until the 1999 BrainShare that there was room for the session dedicated entirely to NPL .
Novell Kernel Services API. In the Fall of 1998, I began the definitive version of NPL now dubbed Novell Kernel Services (NKS) API. Simultaneously, it was implemented as the interface to the Modesto kernel. NKS.NLM is written to sit directly atop whatever version of NetWare it is loaded on. Please note that NKS.NLM is not specifically engineered to load on versions before v4.10. There are reasons for its inability to do this, the most important being that its NCP requester makes use of WinSock as its transport rather than directly juggling IP, TCP and IPX. Since there is no WinSock on NetWare 3 and, if WinSock won't load on NetWare 4.10, NKS.NLM may not run on those older platforms. It will run on NetWare 4.11 and later.
Unfortunately, SDK protocols and deadlines have kept us from releasing NKS.NLM in the futures area of the SDK. I know from reading the news group that some of you are frustrated and even bewildered by the performance and reliability of NPL.NLM. However, we have a whole JVM inside Novell that was ported to and runs on NPL.NLM. Currently, NPL.NLM no longer loads on the latest NetWare 5 service pack, because it was tied to a particular version of CLIB and that version got bumped for the service pack. Some reported problems are not being fixed, since by the time we could get the fixes out officially, NKS will be almost complete and it is certainly well beyond the stability of the original prototype in its present state. It is available in the SDK.
So, what can you look forward to in NKS? NKS will include a fully functional version of the threads package and synchronization primitives, and some file I/O. Presently, most of the simple file I/O is up and running with directory I/O to go. A detailed discussion of NKS features is outside the scope of this article, but is covered in another.
NetWare Interface Tools (NIT) and Other NetWare Interfaces. These interfaces remain the mainstay of NetWare NLM programming and it is not our intention to eliminate functionality. Several years ago, we deprecated the interfaces of NIT.NLM in favor of those covered by the cross-platform APIs (so-called because they are present in virtually identical form on Windows and DOS) in CalNLM 32.NLM and others. However, NIT.NLM cannot really be deleted as long as there are NLMs using its interfaces. So NIT interfaces are merely no longer the preferred ones. I believe that documentation on them will soon disappear as well as import files and its headers have been relocated to a separate subdirectory to make access to them happen by explicit design on the part of the developer consuming them.
Old NLMs will continue to load, however, and if you can figure out how to write new code to them and link that code, then you can keep using them. Keep in mind that they will never be ported to Modesto, because the NLM format won't be supported on the 64-bit platform.
ANSI, POSIX and Other Programming Environments. Since the vast majority of programs start up using these standard functions, they won't ever be eliminated. NKS contains file and directory I/O functions that are thread-safe, but it is clearly understood that some developers may not want to rewrite to them because their code is portable to other platforms or they simply do not like them.
Technology Transition
So, the question for many developers is: "Is Novell going to disturb CLIB and make NKS the new programming paradigm?" The answer is no, Novell will never remove CLIB from the picture. In fact, since Net Ware 5, it has become impossible to unload CLIB.NLM and its companions. But, for developers wanting to write code that will scale, writing to NKS is the only way to get as much scaling as the underlying system components leaned on in the application allow for.
What this means is that if your application is going to use NIT calls, maybe you don't need to use NKS. If your application has only one thread, maybe you don't want to come over. You alone will have to make this decision, but our goal is to ensure that no reasonable lack of interfaces will be there to impede you from adopting the NKS thread model. But, if you are only starting to code your application, come over now.
Many of you may fear that programming to NKS will mean the loss of the common POSIX and ANSI interfaces. This will not be the case. These interfaces are being ported to or rewritten to NKS with the LibC library. LibC will be an NLM, but it will also be the standard C library for Modesto. The code is single- sourced between the two platforms except for a minimal amount of assembly. How is LibC overcoming NetWare's flat symbol namespace? By prefixing, a little known or used phenomenon that has been available in some linkers like NLMLinkX and NLink. Since it is part of our NLM tool specification, it is also in the new linkers produced by EPC and Metrowerks.
However, it would be dishonest to say that LibC will be completely compatible with CLIB. We are taking this opportunity to correct any bad semantics or other programming holes that exist in CLIB's rendition of these common interfaces. For example, fputs will begin returning the number of characters put rather than 0 for success and -1 for failure. You won't have access to FILE's viscera. There will be a second argument to mkdir. The O_EXCL flag will produce the expected behavior when passed to open, etc.
We suspect it won't be possible to find equivalent functionality in the NKS/LibC environment for everything you have been doing but we haven't finished our analysis of missing interfaces. Consequently, we have designed and implemented a CLIB context agent that will broker calls over from the new environment to the old for that subset of calls which do not find a functional equivalent in the new environment. Getting calls back into CLIB from the NKS/LibC environment will be simple and almost transparent from a coding stand-point. It is basically recompiling your source code with a new header to be included before the headers describing the interfaces you are using from the old environment.
Will we support the cross-platform and locale librarys', access to NDS, and other interfaces? Yes, because to offer an environment without those would be pointless. How we are going to do that isn't completely clear. We have been rewriting the bottom, software-independent layer of Cal NLM32 and others to get them to sit indiscriminately atop CLIB or NKS. For specifics, however, we'll have to get back to you. By the time this article reaches you, we'll almost certainly have those answers and with Novell Developer Support will be putting them up in the news group. We hope to have a more complete locale library as well as an improved and faster Unicode library since the NKS interfaces accept only Unicode. (Never fear, it goes both ways; we've got some spiffy interfaces to get around this for those of you staying in ASCII.)
Future and Conclusions
To avoid confusion, the NKS/LibC interfaces will be the object of a separate SDK. This probably doesn't mean a separate CD or URL on the web, but instead, separate subdirectories for headers, imports and perhaps separate manuals. In up-coming months, we will finish our stories for all the components of our final 6 Pack-oriented SDK. We will also give you information on where we have found problems in moving from CLIB to NKS/LibC.
In summary, we've discussed NetWare as an applications platform, what the assumptions are and how we are reacting to solve some problems, but not others which we see as perceptions and incompatible with the nature of NetWare (the lack of fork). We've also summarized the tools available for NLM development and outlined our library strategy. We've also discussed our new thread-programming model as well as explained why this new model is necessary. We've established our position on these issues, which was the main goal of this article. Finally, we've talked about making a few necessary technological transitions to benefit scaling across multiple processors the principal technology difference that motivates adjustments to our existing library strategy. In future articles, we will be discussing in greater detail the issues of multi threaded programming techniques, especially how to use NKS to write code that scales across multiple processors.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.