Runtime Programming in Java: A Technology Primer
Articles and Tips: article
Software Engineer
Network Security Development
01 Jun 1999
Runtime programming is about being able to specify program logic during application execution, without going through the code-compile-execute cycle. This article describes key elements of the infrastructure required to support runtime programming in Java, and presents a fairly detailed analysis of the design.
Introduction
Runtime programming is about being able to specify program logic during application execution, without going through the code-compile-execute cycle. While it may not be possible to write a meaningful application entirely at runtime, specific application modules that need to be dynamically upgraded, enhanced, or augmented can be written and integrated without bringing down the application. Notable examples of program logic that need runtime definition are query logic, and event handlers.
Traditionally, applications use a scripting language for dynamic description of program logic. The runtime specified program logic is stored as a script written in a scripting language, and the application executes it using a built-in interpreter. Such interpreters tend to be complicated, inefficient and are not extensible. Enhancing the scripting language to include more verbs can be a daunting task, and must be done offline. Even though interpreter-based processing of runtime specified scripts is probably best suited for most programming environments, Java offers a simple and elegant alternative that is object-oriented, efficient, and inherently distributed and allows dynamic extension of the programming language verbs itself.
This article describes key elements of the infrastructure required to support runtime programming in Java, and presents a fairly detailed analysis of the design. Snippets of sample code adopted from a prototype implementation are illustrated for clarity.
Overview
The primary motivation for the development of runtime programming has been "Dynamic Definition of Event Handlers." An event-based application hinges on definition of different events and corresponding event handlers. A truly distributed, event-based application, should be capable of recognizing and handling new event types, not known at the time the application was written. An example illustrates how event-based programming can be applied to simple applications.
Consider a simple E-mail application that allows a user to define automatic rules for E-mail processing. Events relevant to the E-mail application are arrival of a new mail, opening, deleting, or filing an E-mail into a folder etc. The various attributes of an E-mail, such as sender, recipient, subject, content, date of arrival, folder to which filed, etc., are eligible attributes for each event. An event handler probably looks like an if <boolean expression> then <action> statement, where the boolean expression defines a custom filter based on the attributes of the E-mail, and action consists of a combination of operations that can be performed on an E-mail, such as auto-reply, auto-forward, auto-delete, and so on.
Suppose we wish to send appointments as E-mails, with four new attributes, namely, venue, date, time and duration. An extensible E-mail application should be capable of recognizing the four new attributes of this new event type and should seamlessly allow specification of an event handler for appointments, based on its additional attributes.
Dynamic Definition of Event Handlers
Assume a distributed, event-based Java application. "Event Objects" corresponding to significant activities in the system are generated and posted on an event queue. The event queue may allow different subsystems to subscribe to specific events. An event dispatcher selects each event in turn, evaluates the event, and executes some action based on the evaluation result. WebLogic Tengah provides a good example of an events-based application framework. A WebLogic application on the network can register interest in an event. An event might correspond to a move in stock prices or a change in the weather in a particular city. It could be linked to a performance mark on a particular machine somewhere on the network, or discovery of new nodes on the network.
The events notification and management system in the WebLogic Events Architecture requires the applications interested in specific events to hard-code the program logic for evaluate() and action() operations by implementing the logic as part of a Java class that implements Tengah-specified interfaces. This restricts an application to register for known events only. As a result, an application developer cannot meaningfully register for new event types, unknown at the time the application was written, though in a distributed events-based system, it is quite conceivable to have new types of events constantly being defined.
The technology presented in this article allows applications to register interest in new event types and create the program logic of evaluate() and action() operations at runtime, without requiring down time or suffering performance hits.
Interactive Code Execution
Another interesting application area for this technology is interactive code execution, which can be used for development of test tools. The traditional way of testing Java classes requires writing test code, compiling it, and executing it. Since writing text code requires programming knowledge in Java, the programmers themselves often write the test suites. Further, the non-interactive nature of testing (code-compile-execute vis-a-vis execute a statement, then decide what to do next) severely limits the test coverage.
Interactive testing would allow the user to write test scripts at application runtime and interactively execute a statement before even writing the next one. The complete test script would be a Java object that can be made persistent across invocations to allow regression testing. A graphical testing tool deploying this technology would allow even non-Java programmers to write a test harness for Java programs.
The program logic specified at runtime is constructed as a Java object, thus eliminating the need for an application-specific interpreter for its execution. The primitive constructs of the language used for describing the program logic are based on simple, application-independent Java classes. The program logic, once constructed as a Java object, executes much faster compared to the interpreter based execution in Java.
Serialization is used to make such Java objects persist across application invocations, thereby preserving the dynamic program logic. New language constructs can be plugged in at runtime by simply loading the additional classes for those constructs. In contrast, the interpreter- based methodology requires offline updating of the interpreter code to recognize and execute scripts written using new primitive constructs.
The following section elaborates the technical details of the proposed technology.
Design Details
Program logic essentially consists of three types of components, data, operation, and control.
Data elements might be fixed-valued, such as a constant literal or variable, such as value of data members of runtime objects, or results of method execution of runtime objects with parameters that could be fixed-valued or variable data themselves.
Operation elements refers to the primitive operations on the data elements (such as mathematical operations and boolean operations) as well as the methods provided by various classes (such as string operations).
Control refers to flow control primitives such as if-then-else, while-do, etc.
For the purpose of runtime programming, we do not consider more structured programming constructs, such as new class/method definitions.
The design goal, which is the cornerstone of this technology, is to define Java classes with appropriate interfaces for each primitive (and only the primitive) construct comprising program logic, such that upon invocation of an instance of the class the same result is achieved as that of an equivalent program logic written in Java and compiled as byte codes.
Building a complex program logic at runtime is therefore simply instantiating a set of Java classes for each primitive construct of the logic, all glued together at runtime in a hierarchy of nested references. Execution of the logic is as simple as invoking one method on the topmost object in the hierarchy, which automatically and recursively invokes the appropriate methods of objects lower down in the hierarchy.
Let's consider the design for various primitive constructs in some detail.
Data
We begin by recognizing the characteristics of data (fixed valued or variable) such as type and value, and provide an interface definition that allows getting and setting values for each characteristic. A note on type checking might be in order. Type-checking can be implemented at the stage of program logic construction by using the type query interface such as getType(). We might additionally provide type conversion interfaces to allow implicit type conversion where feasible. A simplistic definition might look like this:
public interface Data {
Object get (Object [] objs);
Class getType();
/* Type conversion interfaces ... */
boolean getBoolean(Object [] objs);
int getInteger(Object [] objs);
String getString (Object [] objs);
}
The parameter, namely Object[] objs, in the get methods is significant for the Variable data values.
Fixed valued data. One of the simplest objects to define, it only requires a persistent data member to be initialized with the fixed value via the constructor, and accessible via type-specific get() methods (e.g., getInteger(), getBoolean(), getString(), getObject() etc.).
Variable data. Variable data refers to the runtime value of a data member or return value of a method invocation for application objects. Static information such as the name and type of the field/method, number, and class of the parameters in case of a method, and the application object's class is enough to retrieve its value at runtime for a given object, using Java reflection APIs.
The Java class implementing variable data primitives initializes the required static information into persistent data members in its constructor. When its data-retrieval method (type specific get()) is invoked for a given application object, the runtime value of the desired field or method invocation is retrieved using the Java reflection APIs to access the field/method based on the static information. The application object (plus the parameters, in case of a method invocation) is passed as an argument to the get() method. For method invocations, the parameters can be fixed valued/variable data or the result of another invocation method and can be constructed recursively.
Operators
Though Java language defines a fixed number of operators that operate on primitive data types, developers have the discretion of providing as many (or as few) and as complex (or as simple) operators as desired for writing program logic at runtime. An operator takes a fixed number of operands and produces a result, all of predictable types. The operands themselves can be results of other operations, or they can be primitive data elements (fixed valued or variable).
A Java class written to provide a primitive operator takes the required number of operands (i.e., data) of appropriate type as arguments in its constructor. Each operand in turn, has an accessor function to retrieve its type as well as its value at runtime. The operator class returns as result, another object, containing an accessor function to retrieve its value. The accessor function of the result, when invoked, causes the actual operation to be performed. Such a lazy execution is the key to the design of the operator constructs.
The following simplistic classes illustrate how the lazy execution mechanism is achieved:
public abstract class BinaryOperator implements Serializable {
protected Data lhs; // left hand side of the binary operator, // initialized in constructor
protected Data rhs; // right hand side of the binary operator,
// initialized in constructor
.....
public abstract Result getResult();
.....
}
public abstract class BooleanOperator extends BinaryOperator {
....
}
/**
* This class implements the boolean operator &&
*/
public class And extends BooleanOperator {
/* The key to lazy execution */
public Result getResult() {
return new BooleanResult () {
public Object get(Object[] objs) {
return new Boolean(lhs.getBoolean(objs) &&
rhs.getBoolean(objs));
}
public String toString() {
return new String( "("+ lhs + " && "
+ rhs + ")");
}
};
}
}
The interface Result is similar to Data. n-ary operators and corresponding data types are simple extensions of the binary case and haven't been discussed here. BooleanResult is an implementation of the interface that additionally implements boolean type checking which has been omitted here for sake of clarity. Lazy evaluation is achieved by dynamic construction of an instance of an anonymous class. The toString() method illustrates how an entire hierarchy of objects can be printed recursively.
The following example illustrates how a boolean expression can be constructed at runtime. Consider the following boolean expression in Java:
(e.married && e.permanent) || ((e.age > 45) && (e.salary > 80000)),
for Employee e, assuming:
public class Employee {
public boolean permanent;
public int salary;
public String name;
public int age;
public boolean married;
....
}
The runtime created object equivalent would look like the following:
new Or( new And( new FieldData(fieldMarried),
new FieldData(fieldPermanent)),
new And( new GreaterThan(new FieldData(fieldAge),
new FixedData(45)),
new GreaterThan(new FieldData(fieldSalary),
new FixedData(80000)))),
where, fieldMarried, fieldPermanent, fieldAge and fieldSalary are instances of the Java.lang.reflect.Field class for the respective data members of class Employee. The construction of such a complex looking object should be automated via an appropriate graphical user interface to ensure there are no syntactical errors. The result of the above boolean expression object can be obtained by invoking the getResult() method on the top level object (or operator in this case). The returned Result can be then used as any other Data in construction of other operations/method invocations and can be evaluated for any given instance of an Employee by simply invoking the getBoolean() method of the Result with the employee instance as a parameter.
An observant reader would have noticed that this mechanism of evaluation of an expression does not require a separate engine, akin to the postfix evaluators historically required in languages such as C. Further, a boolean expression would be evaluated only as far as necessary to determine the result of the entire expression, with no extra effort on part of the programmer. This is achieved automatically, because of the Java language semantics.
For sake of simplicity, the above example illustrated the use of various fields of the same object in the constructed boolean expression. In a more complex scenario, fields of different objects might be used within an expression. The parameter to getBoolean() for the result of such a complex expression would consist of a dynamically constructed tree of Objects, passed as an Object [] via a reference to the root. In most applications of this technology however, such as in Event Handlers, there would be typically be a single object (such as the event object).
Flow control
Flow control constructs such as if-then-else, while-do, etc., take as arguments a boolean expression and one or more action statements. A boolean expression, as illustrated above, is the result of a boolean operation, constructed to arbitrary levels of nesting. The actions are either flow control constructs or assignment operations, which can be constructed recursively too.
The Java class written to provide an if<condition< then<action 1< else<action 2< construct takes three arguments in its constructor, a boolean result object and two action objects, each constructed using one or more primitive constructs. The evaluate() method of the if-then-else class performs the if-then-else logic upon invocation with appropriate arguments. Most flow control constructs can be provided in a similar manner.
A rather complex program logic can be constructed as a hierarchy of objects corresponding to the primitive constructs. Such a conglomerate of objects, each individually performing a simple task, and cooperating together to achieve a complex one is the key design principle in this technology and ideally suited for distributed processing. Printing the entire operation, for example, is achieved by each object printing itself and in turn, causing the objects lower in the hierarchy to print themselves.
Since the order of execution is determined by the primitive Java language constructs, considerable programming effort is saved. For example, an expression evaluates itself in the correct order, without the need to convert the expression to infix/postfix form, which is required in the interpreter-based approach. This distributed nature allows easy runtime pluggability of new primitive constructs as well. Adding a repeat-until construct or a new mathematical operator requires just the classes implementing those constructs to cooperate with the existing ones via known interfaces. The script builder can be made to dynamically read newer primitives at runtime, without changing any existing primitive or bringing down the application. The program logic thus built can be serialized and restored when required.
With the entire hierarchy glued together using object references, it is a simple matter to serialize and deserialize the entire hierarchy. Only the top node in the hierarchy needs to be assertively serialized or restored, and all other nested objects automatically follow, since the references are not transient.
The programming approach in this technology might remind the reader of the functional programming paradigm. The language primitives modeled as functions, allow complex program logic to be constructed as a functional object, which can be treated as a first class object. This might be an attractive alternative for applications naturally suited for functional programming, in particular mathematically oriented applications, which must be done in Java. Functional programming languages such as LISP and ML are strongly typed languages, something that hasn't been discussed at length here. However, as pointed out above, the designer can choose to implement as strict type checking as desired, by checking for type-compatibility during construction of each of the primitive constructs.
This technology requires construction of a visibly complex hierarchy of objects for even simple program logic, as illustrated above in the example of a boolean expression. While the construction seems largely intuitive, it should best be automated using an appropriate interactive interface (possibly a graphical user interface). The interface would allow type-safe construction of runtime objects, thereby eliminating all type mismatch errors. Separate user-interface sub-modules corresponding to the base class of each primitive language construct would be sufficient to handle even newer primitive constructs, as long as they subclass an existing base class. New base classes might be introduced with their own user-interface modules, compatible with the rest of the user- interface for dynamic pluggability of UI.
Conclusion
Runtime programming of certain specific application modules has traditionally been done via interpretation of program scripts generated at runtime. The interpretation-based approach is complex and less efficient, as well as non-extensible. This article presents an easy, extensible, distributed and efficient, object-oriented approach to create program logic at runtime. This technology makes use of some important Java language features such as serialization, reflection, and inner classes, and is best suited for the Java programming environment. The design and implementation considerations for development of this technology have been discussed at appropriate length.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.