Three Ways to Deliver Cached Performance to Your Intranet and Internet Users
Articles and Tips: article
Senior Research Engineer
Advanced Development Group
01 Sep 1997
Looking for a powerful yet cost-effective way to squeeze more performance out of your web servers? This AppNote introduces you to the wonderful world of caching . . . proxy caching, that is, as brought to you by Novell's BorderManager.
Introduction
Network engineers and administrators are constantly trying to squeeze the highest performance out of their systems using the most cost-effective means available. Yet the widespread deployment of Internet and intranet connections has imposed new requirements that seem to be in conflict with these efforts to enhance network performance. Comprehensive security restrictions, access controls, and content filtering are crucial aspects of securing the intranet and connecting to the Internet, but they exact an additional performance penalty in an environment where users are already frustrated by busy Web servers and long response times.
Novell's BorderManager includes an Internet object cache that significantly increases the speed of web access. In the process, this technology provides a performance foundation to support your network infrastructure and offset the performance penalty you pay for the necessary security controls and filtering.
This AppNote provides an overview of BorderManager's caching technology and discusses the advantages of caching in Intranet and Internet environments. It then describes three applications of Novell's Internet object cache that provide significant benefits to intranet and Internet users:
Proxy caching
Proxy cache hierarchies
Web server acceleration
For more information on BorderManager and other AppNotes regarding these technologies, visit the Novell World Wide site at:
What is Caching?
During the 1960s, computer designers discovered that much of the program code their systems were executing was extremely repetitive-small portions of the code would be processed over and over again. Using this insight to their advantage, they began storing the repetitive portions of their programs in a small, very high-speed block of memory called a cache. This cache was tied closely to the CPU so that its cached contents were immediately available for processing, instead of having to go through the slower processes of being accessed through regular memory or from disk. As the code in these systems looped through cached portions of the program, the resulting throughput was orders of magnitude greater than in systems operating without the cache.
During the mid-eighties, Novell engineers used this same model to benefit from the naturally occurring repetitive use of shared data and services on Novell file servers. Since those early beginnings of the network computing industry, every version of NetWare has provided a shared cache to speed access to repeatedly used network information and services.
Since that time, designers have developed increasingly sophisticated caching technologies that store repeatedly used code in high-speed caches close to the CPU. Intel-architecture systems use several levels of cache, in a cache hierarchy of sorts, that attempt to keep repetitively used code and data cached so their fast CPUs are not stalled while the next instructions are fetched.
Today, all computer users benefit from this design in which frequently used data from our computer's disk subsystem is cached in main memory, frequently used code and data in main memory are cached in the Level 2 (L2) cache, and most-recently-used code and data are stored in the CPU's primary (L1) cache. This design takes advantage of the repetitive nature of applications and also saves users from having to store all of their applications and data in the expensive high-speed RAM that is used to construct high-speed caches.
The Need for Caching in the Internet and Intranet
In light of the huge benefits that caching provides in other areas of computing, it is only natural that caching should play an integral role in optimizing the performance of intranet and Internet systems and services. The Internet services we enjoy as we browse the World Wide Web and our private intranets were originally designed without caching in mind. Of course, the designers could not have anticipated the grand scale their original technologies would be taken to on the World Wide Web. But the impact of this oversight is tremendous when you consider the Internet's repetitive usage patterns and the potential benefits caching can offer in such an environment.
Consider the following characteristics of Internet/intranet access that make this the perfect application for reaping the benefits of caching:
The access patterns of the Internet's global user community closely resemble the kind of patterns seen by videocassette rental businesses, in which 10 percent of the videos account for 90 percent of the rental business. In otherwords, the overwhelming majority of traffic on the Internet comes from relatively few, but extremely popular, sites. If the contents of these sites were cached in various locations around the world, access times could be significantly increased for users across the globe.
The access patterns of individual organizations and workgroups within organizations is also repetitive in nature. Workgroups tend to share similar responsibilities and interests, and therefore need access to similar sets of information. Corporate intranets are thus another ideal environment for caching shared content.
As seen in these scenarios, Internet Service Providers (ISPs), corporate webmasters, IS staff, and end-users can all benefit from Internet caching in one form or another.
Can You Afford Not to Cache?
To give you an idea of the power of caching, we measured the performance of a web server system before and after caching was applied. We started with a Sun Ultra Enterprise 3000 system running Netscape Enterprise Server 2.5 web server software, including Sun's Solaris Internet Server Supplement. The Sun system had two UltraSPARC 167MHz CPUs, 256 MB of RAM, and connections to two load-balanced 100 Mbps Ethernet segments. This $40,000 web server was able to achieve only 550 hits per second before the CPU became saturated at 100 percent utilization.
We then added a cache, called a web server accelerator, to the web server configuration. The accelerator ran on a Compaq ProLiant 6000 system running Novell's BorderManager suite of Internet services. The Compaq system had one 200MHz Pentium Pro CPU, 256 MB of RAM, and connections to five load-balanced 100 Mbps Ethernet segments. Configured as a web server accelerator, BorderManager cached the Sun web server's site and responded to requests from its own cache. This $20,000 system peaked at 3,300 hits per second before saturating the CPU at 100 percent utilization.
These results suggest you would have to add five more Sun systems to your site to reach the high capacity provided by one standard Intel-architecture, high-volume web server accelerator. This presents an interesting business case. Suppose you own a Sun web server that isn't performing as well as you would like. To improve your Sun system's performance, you could add five Sun Ultra Enterprise 3000 web servers to your existing Sun hardware to produce a combined capability of 3,300 hits per second (see Figure 1).
Figure 1: Adding five Sun Ultra Enterprise 3000 web servers to your existing Sun web server infrastructure will produce a combined capacity of 3,300 hits per second.
Individual response times will be just as slow as responses from the single Sun configuration, because the requests are still being handled by the slower Sun Ultra Enterprise 3000s. But you will have achieved greater capacity. Unfortunately, the price tag for this solution is a hefty $200,000.
Another option is to add one Compaq ProLiant 6000 running Novell's BorderManager web server accelerator to your existing Sun system (see Figure 2).
Figure 2: Adding one Compaq ProLiant 6000 BorderManager Web Server Accelerator to your existing Sun web server infrastructure will produce a capacity of 3,300 hits per second.
With this solution you achieve the same combined capacity of 3,300 hits per second, but at $20,000, the cost is one-tenth that of the first solution.
Of course, not every site needs the capacity to handle 3,300 hits per second. Perhaps the more important aspect of these results is the difference in response time individual users see at their workstations. For similar cached responses, BorderManager's response times are approximately six times faster than those serviced by the Sun system.
Recognizing the incredible power of caching and the price-performance advantage of Intel-architecture servers, Novell has implemented this same Internet object cache as the foundation of its BorderManager suite of Internet services. The remainder of this AppNote describes three applications of this Internet object cache that provide significant benefits to both Internet and intranet users: proxy caching, proxy cache hierarchies, and web server acceleration.
Proxy Caching
Proxy caching is an integral part of the proxy services you implement at your organization's Internet "border" (the connecting point between your network and the Internet). Novell combines caching with its proxy service to provide a high-performance foundation upon which you can build your security policy, user-level access controls, and content filtering. Without adequate bandwidth and the fast response times provided by Novell's Internet object cache, the security, controls, and filtering you try to deploy will only compound your users' frustration with the Internet's long response times.
Figure 3 shows a typical way in which many organizations have provided Internet access to their user community.
Figure 3: Without a proxy cache, Internet access is typically unmanaged and unsecure.
In such a scenario, multiple users often share limited ISP connections to the Internet, making it nearly impossible to share the benefits of the Internet with everyone in the organization. It is also extremely difficult to implement the security and filtering necessary to protect against intrusion and maintain user productivity.
Figure 4 shows how BorderManager's proxy cache, implemented at your organization's Internet border, allows you to secure the Internet gateway and control user access while improving your users' average Internet response times.
Figure 4: Novell's proxy services use an Internet object cache as the performance foundation for a secure, user-level controlled, and filtered Internet border.
When configured as a proxy cache, BorderManager uses its cache to store frequently accessed web pages locally. In this configuration, web page access is significantly improved and users become more productive because they spend less time waiting for information to be delivered. The reuse of locally cached web content can also significantly reduce the workload on your Internet connection.
After installation, the proxy cache and firewall are configured with the organization's security policy, access controls, and content filtering requirements. As users access the Internet, the documents they browse are cached within the proxy's Internet object cache. When a user repeats a previous access, or when another user browses the same Internet site, BorderManager immediately responds to their request from its cache.
How Proxy Cache Works
Figure 5 illustrates how BorderManager caches HTML documents and other cacheable content.
Figure 5: A proxy cache saves repeatedly-used objects to speed access and reduce Internet traffic.
A browser issues a request for a file named DOC.HTML. This request is sent to the proxy cache over a 10 Mbps EthernetLAN segment. In this case, the request results in a "cachemiss" because the proxy cache has never serviced a request for that document before.
The proxy cache initiates a request for DOC.HTML from the origin web server on behalf of the browser. This request is sent over a T1 line to an ISP, then traverses the Internet until it arrives at the origin server.
The origin web server responds to the proxy's request by sending DOC.HTML. This transmission is much faster than a response to a browser due to the proxy's optimized receive window that can receive up to 64KB at one time and stays open to receive multiple responses. The proxy then places DOC.HTML in its cache.
The proxy cache responds to the original browser request with DOC.HTML.
Now when the same browser (or any other browser)issues a request for DOC.HTML, the request results in a "cache hit" because the proxy has kept a copy of the document in its cache.
In this case, the proxy replies immediately to the browser request because it has DOC.HTML in cache. The proxy's response is transmitted at 10 Mbps to the browser, eliminating the need to fetch the document again from the origin server on the Internet.
The proxy cache also honors Time-To-Live (TTL) tags that dictate the length of time an object can remain in cache before it must be refreshed from the origin web server. When the TTL expires on an object, the proxy cache pings the origin web server with an "if modified since" request. If the document has been modified, the proxy cache replaces the old object in its cache with a new one from the origin web server. Dynamic requests for real-time data and other "on the fly" types of requests are not cached so that consumers always get the latest information.
Organizations and workgroups that share proxy services tend to share common responsibilities and common web site interests. These usage patterns translate into a 60B70 percent cache hit rate because workgroups tend to use the same information located on the same web sites 60B70 percent of the time. The remaining 30-40 percent of traffic consists of first-time requests and non-cacheable content such as dynamic CGI requests. But even dynamic requests, such as CGI scripts, take advantage of caches when the response includes static HTML and GIFs. When these static components of the response are cached, only a small part of the response is actually non-cacheable and has to be fetched all the way from the origin server.
Once the proxy cache is operational and the cache is filled with content that is repeatedly used by your user community, the elimination of traffic on your Internet connection can reduce your bandwidth requirements and free your existing connections for fetches of new content and the servicing of dynamic, non-cacheable requests.
This performance foundation provides the performance you need to offset the unavoidable performance penalty you pay for access controls and content filtering.
Proxy Cache Hierarchies
In many organizations, the advantages of a proxy cache can be multiplied by placing additional caches throughout the organization. Multiple proxy caches can be configured in a hierarchy to move shared content closer to those who use it. With a cache hierarchy in place, first-time accesses and cache misses may be fetched from other caches within your organization, rather than returning all the way to the origin web server in your intranet or on the Internet. Using a hierarchy, you have the added advantage of caching popular intranet content on the remote end of WAN links, thus improving performance for remote users and reducing the amount of traffic going across those links.
Using a single border configuration, in which an entire organization shares a single proxy cache, large organizations and those with remote offices will still suffer from the slow response times of intranet WAN links or heavily congested LAN segments. In these cases, the performance advantage of your cached Internet gateway will be lost on these islands of computing.
An organization with distributed workgroups across large LANs or WANs is an ideal candidate for a proxy cache hierarchy. Using a cache hierarchy such as the one shown in Figure 6, remote offices and other disparate workgroups share in the benefits of cached content.
Figure 6: A proxy cache within a hierarchy saves repeatedly-used objects to speed access and reduce intranet traffic.
In addition, these proxy caches communicate with each other to resolve cache misses before going to the origin web server. Novell uses the industry standard Internet Cache Protocol (ICP) to coordinate hierarchical caching.
Using BorderManager proxy caches, you can design your own cache hierarchy. During implementation, you define the structure of the hierarchy by designating each proxy cache's peers and parents (see Figure 7).
Figure 7: You define the structure of a cache hierarchy by designating each proxy cache's peers and parents.
Both peers and parents may share their cache contents with one another, but only parents may fetch documents that don't exist in the hierarchy. When a proxy cache in the hierarchy encounters a cache miss (meaning a user requested a document that is not in its cache), the cache asks its peers and parents if they have the document. If the answer to one or more of these queries is affirmative, the cache fetches the document from the cache that was able to respond the fastest and has the highest priority. If none of its peers or parents has the requested document, the cache asks one of its parents to fetch the document from the origin web server. This way, documents enter the organization through the top of the hierarchy and flow down into the hierarchy as they are requested.
Due to the average 60-70 percent hit rate on proxy caches by their workgroups, over half of the web-related traffic on their Internet connection is eliminated. This reduction, in turn, can reduce your bandwidth requirements and free your intranet LAN and WAN links for non-cached content and the servicing of dynamic requests.
In concert with NDS, a proxy cache hierarchy also allows you to set up your security policy, access controls, and content filtering at appropriate levels within your organizational hierarchy once. All of your NDS users then inherit those controls and filters depending on their location in the directory.
Web Server Acceleration
Web servers can be a bottleneck in your intranet or Internet infrastructures. Typical web servers quickly run out of connection capacity and tend to produce slow response times. In sites where performance is important, the only options usually considered are to upgrade to a more expensive web server system or to split the content set across multiple web servers. Neither of these options make sense when caching offers such an elegant, cost-effective means to overcome the problem.
Configured as a web server accelerator, Novell's Internet object cache eliminates the web server bottleneck by placing a dedicated cache in front of the web server and handling requests for all of the web server's cacheable content directly from its own cache. Caching is the obvious solution because typical web sites are constructed with approximately 95B100 percent cacheable content. Once this material is fetched from the web server and cached in the web server accelerator, the accelerator can handle all of the requests for that content. This leaves the small percentage of dynamic requests to be "passed through" the accelerator for the origin web server to process (see Figure 8).
Figure 8: The web server accelerator offloads over 90 percent of the web server's workload and responds to requests at cached speeds.
A browser issues a request for a file named DOC.HTML.This request is received by the web server accelerator. In this case, the request results in a "cache miss" because the web server accelerator has never serviced a request for that document before.
The web server accelerator initiates a request for DOC.HTML from your web server on behalf of the browser.
The origin web server responds to the web server accelerator's request by sending DOC.HTML. This transmission is much faster than a response to a browser due to the web server accelerator's optimized receive window that can receive up to 64KB at one time and that stays open to receive multiple responses. The web server accelerator then places DOC.HTML in its cache.
The web server accelerator responds to the original browser request with DOC.HTML.
Now when the same browser (or any other browser)issues a request for DOC.HTML, the request results in a "cache hit" because the web server accelerator has kept a copy of the document in its cache.
In this case, the web server accelerator replies immediately to the browser request because it has DOC.HTML in cache. The proxy's response eliminates the need to fetch the document again from the origin web server.
In most cases, the accelerator offloads 95-100 percent of the traffic previously handled by the web server. This not only eliminates the need to upgrade your web server to improve performance, but leaves you with an underutilized web server system and ample capacity for future growth.
Figure 9 illustrates how Novell has designed its Internet infrastructure for its own corporate web server (WWW.NOVELL.COM).
Figure 9: Novell uses BorderManager to accelerate WWW.NOVELL.COM.
Using BorderManager configured as a web server accelerator, Novell was able to redraw their Internet infrastructure to their advantage. For instance, the security of the web servers was improved by moving them behind the inner firewall. The accelerators offload 95 percent of the web servers' workload and service dynamic requests through secure firewall connections. (For more information on Novell's use of web server acceleration, see "Web Server Acceleration Using BorderManager: A Case Study of WWW.NOVELL.COM" in the August 1997 issue of Novell AppNotes.)
Conclusion
In 1984, Novell made a name for itself in the fledgling LAN industry by delivering data over a network faster than it could be retrieved from a local hard disk. Now, with BorderManager, Novell is providing the technology to deliver that same cached performance to intranet and Internet users. The overwhelming cost-effectiveness of Novell's cache technologies, combined with inexpensive Intel-architecture servers, will soon make non-cached web server systems obsolete.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.