Debugging NLMS with the Dexter Debugger Extender
Articles and Tips: article
Senior Research Engineer
Novell Systems Research
01 Jan 1995
The Dexter Debugger Extender adds commands to NetWare's internal debugger to make it easier for you to test and debug NLMs (NetWare Loadable Modules). This DevNote describes the features Dexter provides and illustrates how to use them.
- Introduction
- Installing and Using Dexter
- TESTWALK.NLM
- Commands Added to NetWare 3.1x
- Command Added to NetWare 4
- Additional Commands in All Versions of Dexter
- Getting Dexter
Introduction
An article in the August 1991 issue of NetWare Application Notes (sister publication to the DevNotes, now called Novell Developer Notes), "Using the NetWare 3.x Internal Debugger," introduced how to use NetWare's internal debugger to develop and test NLMs. Dexter, a product of Alexander LAN, Inc. of Nashua, New Hampshire, extends the functionality of NetWare's internal debugger. Dexter runs on all versions of NetWare greater than 3.11. Dexter adds debugging commands to supplement those in the NetWare internal debugger.
Dexter provides access to the operating system and processor structures described below.
Call Stack for Running Process
When you are testing an NLM under development, you can usually keep track of where the NLM is in its execution and how it got there by printing messages to the screen, or by setting breakpoints. This information is harder to get after your NLM has crashed. Dexter allows you to look at the call stack by function name to analyze the sequence of function calls that resulted in the NLM crash.
Call Stack for Non-active Processes
Dexter provides symbolic access to the call stack for non-active processes (threads), just as with active processes. This is valuable, since a server crash could result from an action taken by a process that is no longer active. When developing multi-threaded NLMs it is always useful to determine the execution point of all threads comprising the NLM.
Symbol List
In order to provide symbolic access to the call stack, Dexter must first have a list of all symbols that are exported by NetWare and all NLMs. Dexter allows you to view this list, so that you can find which functions are exported by which NLMs.
Global Descriptor Table
Intel processors running in protected mode have a Global Descriptor Table (GDT) that keeps track of where memory segments are in physical memory. To resolve a memory reference, the processor uses an index into the GDT (called a selector) to locate the segment in physical memory, then use the offset relative to the beginning of the segment. Under most circumstances, you don't need to refer to the GDT when debugging an NLM. Dexter makes it available to the NET-Check memory protection utility (which you can also get from Alexander LAN), and, on the off chance that you might be interested in it, lets you look at it, too.
Interrupt Descriptor Table
An Intel processor running in protected mode has a table of 256 selector:offset addresses for each of up to 256 interrupt handlers that can be installed. When debugging an NLM, you can set a breakpoint at the handler for an exception that you know your NLM generates under specific conditions. For example, if your NLM generates a divide error, you can set a breakpoint at the divide error exception handler, then attempt to replicate the conditions that lead to the error.
When a divide error occurs, your server will break into the debugger, and Dexter will allow you to view the function call sequence that led to the exception.
Control Registers
The control registers control and provide status of protected mode and memory paging. As with the GDT, access to the control registers is primarily for memory protection software like NET-Check, and is of limited usefulness to NLM programmers.
System Memory Map
The system memory map (available only in NetWare 4) gives the addresses of operating system and process regions, such as NetWare's global code and data regions, the running process code, and the OS branch table.
Installing and Using Dexter
Dexter is a single NLM that you simply copy to SYS:SYSTEM and load from the server's AUTOEXEC.NCF. For NetWare 3.11, there is a second NLM that must be loaded before Dexter. Like the NetWare internal debugger, Dexter is inactive until
you press ALT-SHIFT-SHIFT-ESC,
an NLM executes interrupt 3,
an NLM calls the CLIB EnterDebugger function (which executes interrupt 3), or
the server abends.
You can therefore leave Dexter installed on test, development, or production servers without worrying about Dexter's impact on server performance. In fact, it is a good ideas to always load Dexter on production servers, because you can use Dexter to diagnose the reason for server abends.
TESTWALK.NLM
To help you get familiar with Dexter's capabilities, Alexander LAN provides a small example NLM called TESTWALK with Dexter. As soon as it is loaded, TESTWALK executes an INT 3 to enter the debugger. You can then single-step through the NLM and view the function call stack as TESTWALK calls its own local functions and CLIB functions. Source code for TESTWALK is given below.
/*---------------------------------------------------------------------------,
1993,94 Alexander LAN, Inc.
All rights reserved.
NOTES
This is a VERY small NLM that is useful for trying out DEXTER's stack
walking features. The actual NLM is also included on the DEXTER program
disk.
The best way to use it is the following:
0. Look over this source code carefully and understand what it does.
1. Load DEXTER
2. Load TESTWALK (which will bring you into the debugger)
3. use the '?' command to see where you are in the TESTWALK code
4. use the '/' to walk the stack frame chain.
5. see how you're now in TESTWALK's main function?
6. see how that function was called from CLIB's _SetupArgv?
7. continue to trace ('T' command) all the way through the call
to CLIB's malloc in MyFcnOne (below). Try the '/' key again.
.--------------------------------------------------------------------------)*/
#include <stdio.h<< #include <stdlib.h<< #include <conio.h<< #include <malloc.h<< #include <string.h<< LONG MyFcnOne(LONG test); // prototype for MyFcnOne void _int3(void); // debug breakpoints... #pragma aux _int3 = 0xCC; // INT 3
/*--------------------------------------------------------------------------),main
here's where we start. note
that we'll do an INT3 right away which will
put us into the debugger before anything
else happens. That way we can go through
the code step by step.
note also that after using the 'g'
key from the debugger to get back to the
console, you have to hit any key to continue
with TESTWALK..--------------------------------------------------------------------------)*/
void main(void)
{
_int3(); // do an INT3: gets us into debugger
MyFcnOne(0xface); // ...so we have something to look at
while(kbhit(-- // clear the keyboard buffer
getch();
printf("Hit any key to quit this thing.\n\r");
while(!kbhit(--
ThreadSwitch(); // spin while waiting for a key
exit(0); // back to the console
}
LONG MyFcnOne(LONG test)
{
char *MyBuffer;
MyBuffer = (char *)malloc(0xabe); // something to do
memset(MyBuffer, 0, 0xabe); // something else to do
free(MyBuffer); // better undo it
return(1); // return to main
}
When you load TESTWALK, it immediately breaks in to the debugger:
Break at F1019017 because of INT 3 breakpoint EAX = 00617E80 EBX = 00617F44 ECX = 00000000 EDX = 00617F68 ESI = 00000000 EDI = 00617F21 EBX = 006C3D7C ESP = 006C3D7C EIP = F1019017 FLAGS = 00000202 (IF) F1019017 68CEFA0000 PUSH FACE
Following the instructions included in the NLM code, if you executed Dexter's ? command, it gives you the current location in TESTWALK's code, with the symbol name for the function being executed.
# ? Dexter reports address in TESTWALK.NLM at code start +00000017h Previous: -00000017 F1019000 TESTWALK.NLM|main Current: 00000000 F1019017 Next: +0000004D F1019064 TESTWALK.NLM|MyFcnOne ESP+04 F1019084 TESTWALK.NLM|DATA LocalCode
The / command gives you the chain of function calls that got you to this point, in this case _SetupArgv, __get_stderr, then main.
# / frm addr ret addr (+ofs) location 006C3D7C F1019017 +0017 TESTWALK.NLM|main 006C3D98 F101C9BF +0062 TESTWALK.NLM|__get_stderr 006C3F86 F1163AE7 +0773 CLIB.NLM|_SetupArgv End of stack frame chain. Press any key for Dexter to continue, <ESC< to quit< ESP&__Value;&Symbolic Area;&_;OS_Memory_Region&_; ESP+04 F1019084 TESTWALK.NLM|DATA LocalCode ESP+08 00000ABE LOCAL DATA ESP+0C F1010008 UNICODE.NLM |DATA LocalCode ESP+10 00010312 UNKNOWN |UNKNOWN physical memory ESP+14 006BDD7C UNKNOWN |UNKNOWN physical memory ESP+18 0070FF21 UNKNOWN |UNKNOWN physical memory ESP+1C 00000000 LOCAL DATA ESP+20 0070FF44 UNKNOWN |UNKNOWN physical memory ESP+24 F1019021 TESTWALK.NLM|DATA LocalCode ESP+28 0000FACE LOCAL DATA ESP+2C 006BDD98 UNKNOWN |UNKNOWN physical memory ESP+30 0070FF21 UNKNOWN |UNKNOWN physical memory
The ability to know the sequence of function calls that got you to a given point in a program can be invaluable in complex, multi-threaded programs.
Commands Added to NetWare 3.1x
When used on NetWare 3.11 or 3.12, Dexter adds a few commands that are in the NetWare 4 debugger, but not in NetWare 3. These commands are described below.
.G The .G command gives you the Global Descriptor Table.
Dexter reports the Global Descriptor Table (length:28)
OFS Base Limit Lv Flags 08: 00000000 007FF 0 CODE RD/AC 10: 00000000 007FF 0 DATA WR/AC 18: 000121B0 0FFFF 0 CODE RD/AC 20: 000121B0 0FFFF 0 DATA WR/AC
.I The .I command displays the Interrupt Descriptor Table.
Dexter reports the Interrupt Descriptor Table (length:3BF)
INT DPL P Gate SEL:OFFSET Purpose 00: 0 P INTR 0008:0001464C Divide Error 01: 0 P INTR 0008:00014685 Debug Exception 02: 0 P INTR 0008:000146CA NMI Interrupt 03: 0 P INTR 0008:0001473B Breakpoint (INT 3) 04: 0 P INTR 0008:00014774 INTO Detected Overflow 05: 0 P INTR 0008:000147AD BOUND Range Exceeded 06: 0 P INTR 0008:000147E6 Invalid Opcode 07: 0 P INTR 0008:0001481F Coprocessor Not Available 08: 0 P INTR 0008:00014878 DoubleFault 09: 0 P INTR 0008:000148E4 Coprocessor Segment Overrun 0A: 0 P INTR 0008:00014952 Invalid TSS 0B: 0 P INTR 0008:000149BE Segment Not Present 0C: 0 P INTR 0008:00014A2A Stack Fault 0D: 0 P INTR 0008:00014A96 General Protection Exception 0E: 0 P INTR 0008:00014B02 Page Fault 0F: 0 P INTR 0008:00014EEB reserved 10: 0 P INTR 0008:00014B6E Coprocessor Error 11: 0 P INTR 0008:00014BA7 Alignment Check 12: 0 P INTR 0008:00014BDE
RC The RC command displays the status of the processor control registers.
Dexter reports Control Registers CR0 = 7FFFFFE5 CR1 = reserved CR2 = 00000000 CR3 = 00000000
Command Added to NetWare 4
Dexter adds the MAP command to NetWare 4 to give you the system memory map.
# MAP
Dexter's OS Memory Map
Region Name
Start Address
GlobalData / LocalDataEnd 0xF0000000
OSBranchTable 0xF03C0000
GlobalDataEnd / OSExternalTable 0xF0400000
PDataSegmentLimit 0xF0800000
LocalCode 0xF1000000
GlobalCode 0xF5000000
GlobalCodeEnd / OSCode 0xF8000000
OSCodeEnd 0xF8400000
OSData 0xF9000000
OSDataEnd 0xFA000000
OS_CACHE_CONTROL_AREA 0xFB000000
RunningProcessMAP 0xFC000000
RunningProcessPCB 0xFC3FF000
RunningProcess 0xFC3FF07C
AlternateProcessPCB 0xFC7FF000
Additional Commands in All Versions of Dexter
You already saw how to use Dexter's / and ? commands with TESTWALK. Dexter provides two other commands, > and SYM, when running with any supported version of NetWare. The > command displays the call stack for non-active processes. To use it, first execute the internal debugger's .P command to get the address of the process you are interested in. Then execute the > command, passing the address as a parameter.
.p Running process: FB001000 testwalk__P 0 Process Processes on the run queue (in run order): Low priority processes (in run order): Processes not on the run queue: FB009000 Console Command Process (Wait for an interrupt wakeup) FB008000 TimeSynchMain Process (Wait for an interrupt wakeup) FB007000 Server 02 Process (Not In Use) FB006000 Server 01 Process (Not In Use) FB005000 Remirror Process (Wait for an interrupt wakeup) FB004000 Media Manager Process (Wait for an interrupt wakeup) FB003000 Sync Clock Event Process (Wait for an interrupt wakeup) FB002000 Server 00 Process (Not In Use) FB000000 MakeThread Process (Not In Use) > FB00000000> Stack Pointer:00038BBC, Stack Limit:00035C00, Using Frame:00038B7C frm addr ret addr (+ofs) location 00038B7C Start of process/thread trace. Previous values are unknown. 00038BAC F80ADCD0 +FFFF SERVER.NLM|GetMaximumNumberOfNameSpaces 00038BEC F802280F +5187 SERVER.NLM|StuffValue ???? F80AE32D +FFFF SERVER.NLM|GetMaximumNumberOfNameSpaces
The SYM command lets you search for symbols exported by NetWare or any loaded NLM. Executing the command with no parameters gives you a list of all defined symbols, along with the name of the exporting module. If you pass the name of an NLM to the SYM command, it will give you a list of all symbols exported by that NLM. And finally, you can pass a regular expression to the SYM command, to search for symbols that match that pattern. For example, you could execute
SYM clib|str*
to get a list of all the string functions exported by CLIB.NLM. The symbol list for TESTWALK is shown below.
sym testwalk 0x005346D4 TESTWALK.NLM|_end 0x005346D4 TESTWALK.NLM|_edata 0x005344E3 TESTWALK.NLM|_cstart 0x00534509 TESTWALK.NLM|main 0x00534490 TESTWALK.NLM|_Prelude 0x005344F1 TESTWALK.NLM|_Stop 0x00534503 TESTWALK.NLM|__Null_Argv 0x00534503 TESTWALK.NLM|__Init_Argv 0x00534504 TESTWALK.NLM|__VersionEnforcement 0x005346C8 TESTWALK.NLM|_argc 0x005346D0 TESTWALK.NLM|_fltused 0x005345F6 TESTWALK.NLM|MyFcnTwo 0x005345B3 TESTWALK.NLM|MyFcnOne 0x005346C4 TESTWALK.NLM|KeepGoing 0x00534623 TESTWALK.NLM|MyThreadFcn
Getting Dexter
Dexter is available for $249 (list price) from:
Alexander LAN, Inc. 100 Perimeter Rd., 2nd Flr. Nashua, NH 03063-1301
Phone (603) 880-8800 Fax: (603) 880-8881
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.