Novell's XForms Strategy: Overcoming the Limitations of Web Forms

Articles and Tips: article

01 Sep 2003

Excerpted from the "Novell XForms Strategy" White Paper available at http://developer.novell.com

This Developer Note discusses the limitations of traditional HTML forms and explains how they are eliminated in the new XForms proposed standard. It introduces the Novell XForms Technology Preview that enables developers to begin evaluating new Web form technologies.

Topics	XForms, HTML, XML, Web forms
Products	Novell XForms Technology Preview
Audience	Web application developers
Level	intermediate
Prerequisite Skills	familiarity with HTML, XML, and Web forms
Operating System	none
Tools	none
Sample Code	yes

Introduction

The World Wide Web represents a revolution in information technology. It allows anyone, using widely available hardware and software, to browse documents, audio, and video content from anywhere in the world. And it is almost as easy to publish this content for anyone and everyone to view. This empowers everyone to be both a consumer and a producer of information.

However, the Internet is based on technologies that were first introduced in the early 1990s, and since then there have been enormous changes in information technology. The need to update these technologies led the World Wide Web Consortium (W3C)--stewards of the technology and standards used on the Internet--to develop XForms, a replacement technology for HTML forms meant to solve many of the largest drawbacks and limitations of current Web forms.

XForms reduce the complexity of developing Web pages and user interactive forms that interact with Web Services and display information to any Web-enabled device. When complexity is reduced, the costs to develop and maintain are also greatly reduced.

This Developer Note looks at the limitations of HTML forms and explains how XForms overcomes these limitations. It provides an example of how XForms improves on traditional HTML forms, and introduces the Novell XForms Technology Preview (NXTP).

The Problem: Limitations of the World Wide Web and HTML Forms

In the fall of 1990, Tim Berners-Lee and Robert Caillau invented what we now call the World Wide Web (WWW). They were working for CERN (Conseil European pour la Recherche Nucleaire), and they and their physicist colleagues were looking to stretch their limited research budgets. They wanted to find a way to avoid high courier fees when sending out their research papers.

Accordingly, Berners-Lee and Caillau began working on a project that created three things: a simplified way to mark up documents by adding tags to them, a piece of software that could retrieve and render these documents from any remote server, and the server software that would allow anyone to publish documents anywhere in the world. As we all know, the markup tags were called Hypertext Markup Language (HTML), the client-side piece of software was called a Web browser, and the server-side piece of software was called a Web server.

(For more background information about the history of the WWW Project, see http://www.w3.org/History/19921103hypertext/hypertext/WWW/TheProject.html. Information about the earliest version of HTML is available at http://www.w3.org/History/19921103hypertext/hypertext/WWW/MarkUp/MarkUp.html.)

This history lesson is essential to understanding the problem. That's because the technology underlying these three pivotal pieces of software--markup, browser, and server--has changed very little in the past twelve years. Indeed, browsers from 1994 are almost identical to browsers in use today. New technologies have emerged in each of these three areas that are capable of much more, but they are hamstrung by these earlier pieces. The biggest issue applies to the browser and the markup, specifically concerning interactive/collaborative forms.

Limitations of HTML Forms

The most commonly cited problem with HTML forms is their dependence on scripting languages like JavaScript and Visual Basic. Real-world HTML forms rely on scripts to accomplish such common tasks as marking required controls, validations, calculations, error messages, and conditional layout. This dependence results in complex documents that are expensive and time-consuming to maintain.

The next-most-commonly-cited problem is the difficulty of initializing a form with data. Each form control has its own way of defining initial data, and form controls are often littered throughout a document. To convert a blank form into a prepopulated form, either a new document needs to be constructed piece-by-piece or an existing document needs to be patched in numerous spots. Either way, the process is CPU-intensive and leads to bottlenecks on high-volume servers.

Another common complaint is that traditional HTML forms can represent only flat data, or name/value pairs. Many types of forms, including purchase orders, would benefit from a richer data representation, especially XML.

Another subtle but still serious issue is the fundamental design of the page flow of HTML forms. There is a built-in limitation of a one-step process from server to client, with processing finishing on the server. In traditional paper-forms-based applications, information often travels more complicated paths. These paths often include complex access control information where some users are allowed to view and interact with specific pieces of data while others see only a limited subset. A purchase order form, for example, might begin on an employee's desk, go to a supervisor for approval, move to the department head for final signoff, and finally reach accounting for payment processing and archiving.

Finally, because HTML forms have different input and output formats, managing them in such a scenario involves reinterpreting the data format at every step along the way. This requires an application that can handle the myriad ways the forms can be filled out.

The Solution: XForms

The World Wide Web Consortium (W3C) became aware of the limitations inherent to HTML forms and perceived an increased need to develop a new, non-backwards-compatible specification for Web Forms. In response, the W3C created a subgroup under the HTML Working Group, the group responsible for advancing the HTML standard to develop some solutions.

Eventually this subgroup became a full-fledged working group charged with defining the requirements of Web Forms and beginning their design. Web Forms are now based on XML and are called XForms.

In November 2002, the XForms 1.0 specification was submitted as a Candidate Recommendation. On August 1, 2003, it became a Proposed Recommendation and is currently undergoing a formal review process prior to being accepted as a Final Recommendation. (Check http://www.w3.org/TR/xforms/ for the latest update on the status of this specification.)

About XForms

XForms provide a new way of describing the Web forms that appear in a browser. XForms can be used with many host languages, such as XHTML (Extensible Hypertext Markup Language, a successor to HTML 4.0 that is supported by most browsers today) and WML (the markup used in Web-enabled cellular phones).

XForms present the following advantages over HTML forms:

Powerful actions, event model, and validation rules enable extensive client-side processing leading to a cleaner programming model and faster response times, without JavaScript or VBScript.
Clean separation of data, logic, and presentation makes it easy to generate data-bound controls.
Highly regular XML structure makes it possible to build WYSIWYG user interface (UI) development environments.
Abstract controls enable abstract application design that gets translated to device-specific rendering.

Where Do XForms Fit In?

XForms are not a standalone specification, but are designed to be hosted inside other markup languages. These host languages may vary in their intended target devices and end users, but all will benefit from XForms.

Figure 1 shows an example of a single host document containing two XForms.

Figure 1: Host language containing two XForms.

XHTML and WML are excellent typical examples of host markup languages, because they embody much of the diversity possible among rendering devices. XHTML is a large specification meant to be used on desktop and laptop computers running the latest Web browsers. These machines typically have large amounts of RAM and fast processors and can handle very large, complex documents, often at display resolutions up to 1600 x 1200 pixels. WML, in contrast, is a small specification designed to fit in the small computing capabilities footprint (RAM, processor speed, display size) of mobile phones and personal digital assistants (PDAs).

XForms Interaction Flow

The typical interaction flow for XForms-based applications (see Figure 2) is very similar to the orchestration for HTML forms. But XForms can perform many tasks that HTML forms cannot. This is mainly due to the XForms Processor, a program usually located in the client tier (such as a browser that supports XForms), but sometimes is in the middle tier.

Figure 2: XForms interaction flow.

The XForms Processor operates as follows:

The XForms Processor makes a request for a document of the host language--one that includes XForms markup. The method used to get the document can vary from as simple as opening a document from a local file system, to an HTTP GET on an URL, to a request issued to a document management system.
The host document is returned containing one or more XForms.
The XForms Processor is invoked, and the XForms markup is parsed and then validated along with any dependent XML Schema documents. TheXForms Processor renders the XForms ready for user interaction. The rendering is device dependent, and thus XForms Processors implemented for a wireless phone, a desktop browser, and point of sale (POS) device will all create their own representations of the XForms.
The user interacts with XForms. Usually this means entering data in some fields and editing data in others.
Client-side events fire. As the user is interacting with the XForms, various events are triggered. These events could be to enforce data type constraints or other validation rules, and could be used for conditional layout--such as to enable or disable controls depending on the value of other sections of the XForms document.
Finally, the user clicks Submit and the XForms Processor generates an XML instance document to send back to the server. The output is an XML document.

Conceptual Structure of XForms

XForms are conceptually composed of three major sections that provide separation of data, presentation, and logic. These sections are Model, UI Controls, and Event Handlers, shown in Figure 3.

Figure 3: The conceptual structure of XForms.

Model Section.The Model section can be equated to the model element in an XForms document and can contain several child elements such as instance, bind, and submit. There can be one or many instance elements, and these can be either empty or filled with data. Each instance is uniquely identified in the document by an id attribute. The instance has a dual role, first as a template for the XML that will be submitted to the server and second as initial data to prepopulate the XForms. Any values that aren't changed are submitted as is to the server. Any nodes in any of the instances that are not explicitly bound UI controls are hidden and therefore can be used like hidden fields in HTML.

XForms use bind statements to declaratively bind calculations, constraints, and validation rules to instance data elements. The bind element can also be used as one means of establishing data binding between UI controls and elements in the instance data. The submit element contains information about where and how to submit the XForms instance, similar to the HTML form attribute of action.

The following is an XForms Model and instance data example:

<xforms:model>
<xforms:instance>
<my:payment as="credit"
xmlns:my="http://www.example.com/payment">
<my:amt />
<my:cc />
<my:exp xsd:type="xsd:date" />
</my:payment>
</xforms:instance>
<xforms:submission id="submit1" method="POST"/>
<xforms:bind id="cc_number"
nodeset="my:cc" relevant="../@as='credit'" />
<xforms:bind id="cc_exp"
nodeset="my:exp" relevant="../@as='credit'" />
</xforms:model>

UI Controls Section.XForms define an abstract set of UI (user interface) controls. Each control specifies its binding to the XML instance data via either an XPath expression or the id of a bind element. The usual controls are present, such as input and textarea, which enable freeform text on a single line and multiple lines respectively. Secret can be used for entry of sensitive information, and the entry is masked as it is entered. Upload is functionally equivalent to its HTML counterpart, except that the XForms working group tried to impart more vision about what can be accomplished. Thus the XForms version of upload allows retrieving file streams from local devices including microphones, tablets, and photo scanners.

HTML has individual controls for select list (single and multiple), radio buttons, and check boxes, but XForms have just two constructs for these: select and select1. The functionality of select and select1 is inclusive of all possibilities of their HTML equivalents. The actual display rendered is dependent on the selectUI attribute, whose options include radioGroup, checkboxGroup, pulldown, listbox, and combo.

Table 1 shows the XForms constructs that are similar to HTML, along with the new controls available in XForms.

Table 1: XForms constructs and controls.

XForms Constructs That Are Similar to HTML		New Controls in XForms
Construct	In HTML like	Control	Comment
input	type=text	output	Used for arbitrary output
secret	type=password	label	Required by all controls (binds the question with the answer)
textarea	textarea	range	Integer spinner
select	type=checkbox	help	Help text
select1	type=radio	hint	Data entry hint (could be a tool tip)
filename	type=file	alert	Display user messages
submit	type=submit	--	--
trigger	type=button	--	--

XForms UI Control Binding. In XForms the UI controls are bound to the Model and instance via XPath expressions. XPath expressions are the standard method for referring to specific locations in an XML document.

The following example shows a simple XPath expression used to bind the xforms:input control to the my:cc node in the Model and instance data. The ref attribute contains the XPath expression that refers to the specific node in the Model. The XPath expression used to bind UI controls to the Model is shown inbold:

<xforms:input ref="my:cc">

<xforms:label>Credit Card Number: </xforms:label>

</xforms:input>

Figure 4 shows how the instance data in the Model is bound to UI controls using XPath statements.

Figure 4: Instance data in the Model bound to UI controls.

Table 2 shows a few examples of the new controls and their possible renderings. Note that these renderings may change radically based on the browser and what is the appropriate UI for the device, input device, and so on.

Table 2: New controls with XForms.

Control	Possible Rendering	Example
range	Integer spinner
upload	type=file
help	helptext
hint	Data entry limit
trigger	Like an HTML button

Event Handlers Section

The largest problem cited with HTML forms is their reliance on scripting to achieve even the most trivial functionality. Application developers accustomed to developing applications in the client/server style were unpreparedfor the difference in functionality offered for Web-based application development platforms. HTML alone offered no validation and no concept of strongly typed data--and was missing a few required controls like integer spinners, table and grid controls, and tree controls. All of this was created in scripting for Web-based applications. This made application development complicated and led in large part to the high failure rate of IT projects in 2000. (For an interesting article on this, see http://www.softwaremag.com/archive/2001feb/CollaborativeMgt.html.)

One of the first items on the agenda for the XForms working group was to enable the building of applications without all this script. Thus, using an XML-based declarative syntax, XForms authors can:

Set focus to a form control
Display a message to the user
Navigate to a new URI
Change the value of an instance data node
Force a recalculation, revalidation, or screen refresh
Perform other actions to deal with scrolling and manipulating repeating line item tables
Submit or reset all or a portion of the instance data

Figure 5 shows how the Event Handlers update the instance data in the Model.

Figure 5: Event Handlers updating the instance data in the Model.

Events can be categorized into four types: initialization, interaction, notification, and error. Each category has numerous events that are created when various thing happen in the XForms Processor.

XForms actions are XML elements representing functions that can be used in event handlers to change the instance data or the UI. Together with the XForms calculation and validation capabilities, XForms actions will replace most of the JavaScript used in legacy Web applications.

An Example

The following example of HTML versus XHTML and XForms illustrates the key differentiators and helps solidify the main concepts.

The Simple XHTML Form

Figure 6 shows a very simple HTML form rendered in the Internet Explorer browser. The form provides no functionality for data type validation, mandatory fields, or disabling of fields that are not required or are inappropriate based on other choices. HTML in browsers performs presentation rendering--and little else.

Figure 6: A simple HTML form rendered in Internet Explorer.

The HTML form fields are uniquely named using the name attribute, and these identifiers will be used when the form is submitted to identify the data being posted. The text of this form is shown in the following listing:

<?xml version="1.0" encoding="UTF-8"?>
<link rel="stylesheet"? type="text/css" href="example_XForm.css">
<html>
<head><title>eCommerce Form</title></head>
<body>
<form action="http://example.com/submit" method="post">
Enter Payment Amount: <input type="text" name="amt" id="amt"/><BR>
Select Payment Method: <select name="type">
<option>Cash</option>
<option>Credit</option></select><br>
Credit Card Number: <input type="text" name="cc" id="cc"/><br>
Expiration Date: <input type="text" name="exp" id="exp"/><br>
<input value="Submit" type="submit"/><br>
</form>
</body>
</html>

You can see that the label "Enter Payment Amount" is just floating in front of the input field amt. Likewise, the labels "Select Payment Method", "Credit Card Number", and "Expiration Date" are freely floating in front of input fields type, cc, and exp respectively.

Some limitations of simple HTML forms are the following:

The labels on the fields are not submitted with the data the user enters, and this limits legal enforceability.
There is an implied relationship between the form element and the Submit button that when the button is pushed the form is submitted.
Web authors often put half a dozen forms on their HTML pages to activate different functionality.

Finally, the completed form will be submitted and the data will be URL-encoded and submitted to the action URL in the form of name/value pairs looking something like this:

amt=499.95&type=credit&cc=1234%205678%2091234&exp=04/03

The server-side process will have to figure out what that data means and what to do with it--not a trivial task.

The Simple XForm

If a similar form is created in XForms, you'll see a great deal more functionality. Figure 7 shows a simple XForms rendered in the Novell XForms Technology Preview (NXTP).

Figure 7: A simple HTML form rendered in Internet Explorer.

Here is the XForms markup that creates this XForms document:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="example_XForm.css"?>
<html xmlns:xforms="http://www.w3.org/2002/xforms/cr"
xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:my="http://commerce.example.com/payment">
<head><title>eCommerce Form</title>
<xforms:model>
<xforms:instance>
<my:payment as="credit"
xmlns:my="http://commerce.example.com/payment">
<my:amt />
<my:cc />
<my:exp />
</my:payment>
</xforms:instance>
<xforms:submission id="submit1" action="example_XForm_submitted.xml"
method="POST"/>
<xforms:bind nodeset="my:cc" relevant="../@as='credit'" required="true()" />
<xforms:bind nodeset="my:exp" relevant="../@as='credit'" required="true()" />
</xforms:model>
</head>
<body>
<xforms:xform>
<xforms:group>
<xforms:input ref="my:amt">
<xforms:label>Enter Payment Amount: </xforms:label>
<xforms:setfocus ev:event="xforms-activate"/>
</xforms:input>

<xforms:select1 ref="@as" id="method">
<xforms:label>Select Payment Method:</xforms:label>
<xforms:choices>
<xforms:item>
<xforms:label>Cash</xforms:label>
<xforms:value>cash</xforms:value>
</xforms:item>

<xforms:item>
<xforms:label>Credit</xforms:label>
<xforms:value>credit</xforms:value>
</xforms:item>
</xforms:choices>
</xforms:select1>

<xforms:input ref="my:cc">
<xforms:label>Credit Card Number: </xforms:label>
</xforms:input>
<xforms:input ref="my:exp">
<xforms:label>Expiration Date: </xforms:label>
</xforms:input>

<xforms:submit submission="submit1">
<xforms:label>Submit</xforms:label>
</xforms:submit>

</xforms:group>

</xforms:xform>
</body>
</html>

This is a little longer than the HTML version, but the increased functionality is tremendous. If you examine the XForms markup closely, you will notice the model element and the template XML that will be created when the form is submitted. You will also notice the bind elements; these are the XForms directives that disable the Credit Card Number field and Expiration Date when the drop-down box is set to Cash or enabled when it is set to Credit. The relevant attribute is the piece that actually does the work, by specifying an XPath expression into the XForms Model that evaluates to true or false; when true, the fields are rendered as enabled.

Figure 8 shows the effect of selecting the Cash option: the fields of Credit Card Number and Expiration Date fields are no longer relevant and arethus disabled.

Figure 8: Effect of selecting the Cash option.

Submitted Instance

Figure 9 shows the XML instance data created by the XForms Processor and the document submitted to the server side. This can be any XML document, and standard XML parsers can parse it.

Figure 9: XML instance data created by the XForms Processor.

Here is the created XML instance data that will be submitted to the server side by the XForms Processor.

<?xml version="1.0" encoding="UTF-8"?>
<my:payment as="credit"
xmlns:my="http://commerce.example.com/payment">
<my:amt>499.95</my:amt>
<my:cc>1234 5678 91234</my:cc>
<my:exp>04/03</my:exp>
</my:payment>

Summary of Increased Functionality

XForms and XHTML are technologies that will usher in a new era in Web application development. The information presented so far should make a clear statement that XForms and XHTML offer major increases in functionality over theHTML predecessor, including multiple device support, less reliance on scripting, XML submission to servers, strong data typing, internationalization, and enhanced accessibility.

XForms reduce the need for scripting in support of common user interactions by providing powerful XML-based declarative event handlers and actions. This allows the construction of dynamic pages that can modify both the data and the UI as the user works in the form.
XForms deliver multiple device support via high-level UI controls that make it possible to retarget user interaction to different devices. Rather than impose existing HTML concepts on devices with limited display capabilities, XForms allow each device to render the UI appropriately for that device. In a desktop browser this might be a dropdown list, and on aphone it might be radio buttons.
The XForms Processor creates an XML instance document that is submitted to the server. This document can contain strongly typed data and can be directly validated on the client before it is processed by the application's back end. This reduces the number of server round trips for validation and thus speeds the filling of forms. In addition, the XML data is in one central location rather than scattered throughout the form.
Internationalization is included in the XForms, because the XForms Processor and the data it produces are all modeled in XML. Sobecausethe XML specification supports Unicode 2.1 and soon will support Unicode 3.0, XForms have support for all known written languages living and dead. Localization still has to be done, but it involves the usual tasks of using locale-appropriate date and time, currency, number formats, and so on.
Finally, the separation of content and presentation will make XForms applications more accessible to users with disabilities. Because all relevant metadata such as labels, hints, and help text are encapsulated by individual UI controls, they are easier to modify for different user modalities. The typical example is a voice browser, which can read Web pages aloud.

These new XForms and XHTML technologies will fundamentally change the way developers build Web applications and will revitalize the Web browser from a read-only, relatively inept presentation-only device into a full-fledged member of an n-tier Web application architecture.

The Novell XForms Technology Preview (NXTP)

The NXTP gives developers a hands-on introduction to the XForms standard. Novell exteNd (formerly SilverStream Software) is well known for its participation in the development of new standards and has been a leader in the rapid development space with its award-winning Page Designer. XForms are a way to bring developer productivity to standards-based applications that will run on the leading J2EE application servers.

Figure 10 shows an example of a calculator that is completely written in XForms--with no script.

Figure 10: Calculator example.

Figure 11 shows an XForms weather client that calls a public Web Service and returns weather conditions. This example uses an XForms Processor extension to call the Web Service.

Figure 11: Weather Client example.

The Novell XForms Vision

The NXTP is just the beginning of the Novell focus on XForms. Future versions of the Novell exteNd suite will leverage the XForms standard to increase developer productivity with visual tools to bind pages to Web Services, relational databases, and other data sources. Developers will be able to assemble applications easily with prebuilt custom controls or extend their applications by creating their own. Finally, developers will create a complete user experience with the Page Flow Engine to define application flow and logic. All components of exteNd will include an easy-to-use, XForms-based visual development environment. Resulting applications will run on leading application servers such as the Novell exteNd Application Server, BEA WebLogic, and IBM WebSphere.

Figure 12 illustrates Novell's vision for the future in the form of a visual XForms design with drag-and-drop data binding and application flow.

Figure 12: The future of XForms at Novell is a Visual XForms designprocess.

Conclusion

The World Wide Web hasn't had any significant changes to HTML forms since 1994. Yet there has been a tremendous amount of work on infrastructure technologies (such as XML) that can be used to simplify the process of buildingWeb forms. XForms are a significant new technology that will significantly improve the way application developers build Web-based applications--ushering in a new era where forms can be built using tools that increase productivity enormously. These simplifications will save countless dollars in the building, debugging, and maintenance of Web applications.

XForms make building a form-based application a scriptless, declarative process that can be reused across many devices and connected to Web Services directly.

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.