Border Manager Caches in on the Web
Articles and Tips:
01 Aug 1997
Accessing the World-Wide Web (WWW)--you know the drill. You click a fewbuttons to launch your WWW browser and connect to the Internet, you enterthe uniform resource locator (URL) for a WWW site--and then you wait. Evenif you enter the URL for your company's intranet site, you may wait a considerableamount of time. In fact, if the available bandwidth between your workstationand this intranet site is limited, you might wait as long as (or even longerthan) you would wait to access an Internet site.
If you want to speed up access to the WWW and enable users to work moreefficiently, Novell has the solution: The proxy included with Novell's newBorderManager (formerly code-named Border Services) significantly reducesthe amount of time you spend waiting for WWW files by caching the filesthat you and other users request. Because this proxy can resolve requestsfor cached files without having to access the requested Internet or intranetsite to retrieve these files, the proxy decreases traffic and increasesthroughput over your company's Internet and intranet connection. As a result,when you request a file the proxy has not cached, you wait considerablyless time than you would wait without this proxy. And when you request afile the proxy has already cached, you wait no longer for that file thanyou would wait for any other file stored on a local server.
MANY SERVICES IN ONE PRODUCT
Novell's BorderManager, currently available in a beta version and scheduledfor release this summer, is a standalone solution for the performance, security,management, and connectivity problems that often occur on an intranet andalways occur at theborder--the point at which your company's networkor intranet meets the Internet. Among other components, BorderManager includesan IPX-IP and IP-IP gateway, a virtual private network (VPN) configurationtool, and a proxy. BorderManager also includes a snap-in module for Novell'sNetWare Administrator (NWADMIN) utility that enables you to manage all ofthe BorderManager components using only one utility. (For more informationabout BorderManager, see "Novell's Border Services", NetWare Connection, May 1997, pp. 25[shy ]36.
The proxy component of BorderManager supports Gopher, FTP, and HyperTextTransport Protocol (HTTP) and serves two purposes: The proxy enhances securityand accelerates WWW access. To enhance security, the proxy checks the traffic-filteringrules you have configured to determine what types of WWW traffic are authorizedbefore relaying that traffic between clients and public or private servers.
You can use the BorderManager snap-in module for the NWADMIN utilityto configure traffic-filtering rules for individual users or groups of users.By configuring these rules, you can restrict users' access to entire sitesor to individual pages during particular times of day. For example, youcould configure a rule that would prevent members of the Everyone groupfrom accessing sports-related WWW sites during business hours.
In addition to providing security services, the proxy component of BorderManagerperforms passive caching, reverse caching, or both. When configured to performpassive caching(also callednormal caching), the proxy storesWWW files that users request from external WWW servers (assuming that youhave configured users' browsers to use the proxy for WWW access). For example,if a user requested a file that resides on a WWW server on the Internet,the user's browser would send the request to the proxy. Upon receiving thisrequest, the proxy would check its own cache. If the file were not in cache,the proxy would forward the request either to a proxy on another server(if you had configured it to do so) or to the file's source WWW server onthe Internet.
After receiving the file, the proxy would store a copy of this file incache before returning the file to the user's browser. If the file werealready in cache, the proxy would return this file directly to the user'sbrowser--without sending a request over Internet or intranet lines.
When configured to performreverse caching(also calledHTTPacceleration), the proxy component of BorderManager stores WWW filesthat users outside your company's network or intranet request from internalWWW servers. For example, if an Internet user requested a file that resideson your company's WWW server, the proxy would first check its own cache.If the file were not in cache, the proxy would forward the request to yourcompany's WWW server, which would send the file to the proxy component ofBorderManager. The proxy would store a copy of the file in cache and sendthis file to the user's browser. If the file were in cache, the proxy wouldsend this file directly to the user's browser--without forwarding the requestto your company's WWW server and consuming network or intranet bandwidth.
THE PERFECT CANDIDATE
Sounds great, doesn't it? Ralf Probst thought so. Ralf is a network administratorat Siemens, a world leader in electrical engineering and one of the largestcompanies in Germany (second only to Mercedes-Benz). Ralf works for Siemens'Technical System Services (TD) in Mannheim, Germany. TD is one of severaldepartments within the Industrial and Building Systems Group (ANL), whichis one of Siemens' 14 business units. TD provides technical services forSiemens' customers and business units and also helps Siemens' customersdesign networks, installs and assembles products, and develops software.
In December 1996, Ralf read about the proxy component of BorderManageron Novell's WWW site at http://www.novell.com and registered immediately for the Early Access Release CD-ROM. This CD-ROMincluded a pre-alpha version of BorderManager. Although Ralf was interestedin several BorderManager components, he was particularly interested in theproxy component.
As Ralf explains in an e-mail message that he sent to Novell shortlyafter receiving the Early Access Release CD-ROM, the proxy component ofBorderManager will ease TD's"daily struggle with rather slow intranet-WANlinks. This is exactly the product [TD] desperately needs."
The Network Situation
Ralf and two other members of a nine-member team run the TD network--oneof the largest independently managed networks within Siemens. The network's35 servers and approximately 500 IPX clients span three physical sites--Mannheim,Saarbruecken, and Karlsruhe. To connect these sites, TD uses versions 2.11and 3.1 of NetWare MultiProtocol Router (MPR) for ISDN by AVM and Novell.
Approximately 20 of the servers on the TD network run NetWare 4.11 or4.1; the remaining servers run NetWare 3.12. The servers also run severalNetWare Loadable Modules (NLMs) from Novell and third-party companies, suchas the following:
Servers COM and MEX (in Mannheim), KHE_COM (in Karlsruhe), and SBN_COM (in Saarbruecken) run NetWare MPR for ISDN.
Server AIM (in Mannheim) runs the Novell Web Server 3.1 and IPX-IP gateway components of BorderManager and Quarterdeck's IWare Connect (another IPX-IP gateway).
Server MOM (in Mannheim) runs Novell's GroupWise 5.
Server BUP (in Mannheim) runs Cheyenne Software's ARCserve 6 for NetWare.
Other servers run NLMs for printing, faxing, and virus protection.
The TD network serves a variety of needs: Of the 700 users at TD, approximately50 percent use the TD network to access fax capabilities, a word-processingapplication, a spreadsheet application, an e-mail application, and otherapplications. Approximately 40 percent of these users use the TD networkas a platform for both hardware and software development, and other usersuse the TD network to document their projects (by creating and storing drawingsof electrical wiring diagrams, for example). However, virtually all of theusers share one reason for using the TD network: According to Ralf, approximately90 percent of these users use the TD network to access the Internet andSiemens' intranet over Siemens Corporated Network (SCN), the infrastructurethat connects Siemens' offices worldwide.
The Internet/Intranet Connection
Before installing BorderManager, Ralf had configured the Netscape Navigatorbrowsers running on TD's workstations to use Netscape Proxy Server to accessthe Internet and Siemens' intranet. Netscape Proxy Server is located atSiemens' main office in Erlangen, about 190 miles east of Mannheim. (See Figure 1.)
Figure 1: After installing BorderManager, Ralf reconfigured users' browsers to send requests to the proxy component of BorderManager--thereby significantly reducing WAN traffic.
TD has only one server with a valid routing entry to Netscape Proxy Server:the AIM server in Mannheim. A permanent 256 kbit/s WAN link (part of SCN)connects this server and Netscape Proxy Server. TD shares this link withother Siemens' departments in its building.
The link is extremely busy, serving as the transport medium for requestsand replies to and from internal and external WWW servers. The link alsoprovides access to Siemens' BS2000 mainframe applications and handles filetransfers between hosts, online connections with strategic business partners,and e-mail traffic.
As you can imagine, connecting to WWW sites over only a portion of alow-bandwidth WAN link is a hassle."During normal office hours, thelink becomes very slow,"Ralf says."Getting data over this linkcan be a horrible experience."
Although establishing another routing entry to Netscape Proxy Serverwould improve response time, Ralf would rather use another solution. Establishinga routing entry, Ralf explains, requires a lot of paperwork and a numberof telephone calls ("you cannot imagine how many,"Ralf says)and takes as long as two weeks.
Ralf knew that redundant requests for WWW files generate a lot of traffic--moretraffic than TD's slow WAN link can handle efficiently. To accelerate users'access to the WWW, Ralf began looking for a solution that would eliminatesome of the traffic created by redundant requests.
INSTALLING BORDERMANAGER
Because Ralf has always been satisfied with Novell products, he turnedto Novell for a solution to his problem--and found this solution in BorderManager."Every other Novell product we have bought and tested has worked toour full satisfaction,"Ralf says."We presumed that BorderManagerwould be no exception."Ralf's presumption proved to be valid.
Ralf installed the pre-alpha version of BorderManager that was on theEarly Access Release CD-ROM, but he had a few problems with the proxy componentof BorderManager (a not uncommon occurrence in a pre-alpha version). Asa result, Ralf requested and received the beta version of BorderManager,which he installed on a test server running IntranetWare. The test serveris equipped with a 200 MHz Pentium processor, 128 MB of RAM, a 1 GB harddrive, and one network interface board. This board is connected to a FiberDistributed Data Interface (FDDI) backbone on the TD network.
To begin the installation process, Ralf loaded the INSTALL NLM at thetest server console and selected the appropriate options (for example, theInstall a Product Not Listed option from the Product Options menu). TheINSTALL NLM copied the NLMs needed to run and configure the BorderManagercomponents, including the Novell Proxy Server NLM (PROXY.NLM) and the NovellProxy Configuration NLM (PROXYCFG.NLM).
Next, Ralf ran the BorderManager SETUP.EXE file from a Windows 95 workstation.This file copied the BorderManager snap-in module for the NWADMIN utilityto the SYS:PUBLIC\WIN95 directory on the IntranetWare server. The file alsomodified the Windows 95 registry to make use of this snap-in module.
SETTING UP THE PROXY COMPONENT OF BORDERMANAGER
To configure the proxy component of BorderManager to perform passivecaching, Ralf enabled this proxy by clicking the Enable HTTP Proxy optionon the Web Proxy Cache screen in the NWADMIN utility. (See Figure 2.) Ralf also completed the following steps:
Figure 2: To use the passive caching capabilities of BorderManager, Ralf accessed the Web Proxy Cache screen in the NWADMIN utility and selected the Enable HTTP Proxy option.
Configured BorderManager's public and private IP addresses
Configured users' browsers to use the proxy component of BorderManager
Configured the proxy component of BorderManager to communicate with Netscape Proxy Server
Configuring IP Addresses
BorderManager uses the IP addresses bound to both the server's privatenetwork interface board (the one that attaches the proxy to your company'snetwork or intranet) and the server's public network interface board (theone that attaches the proxy to the Internet). Browsers on your company'snetwork or intranet access the proxy component of BorderManager using theprivate IP address, which does not have to be a registered IP address. Theproxy sends and receives packets to and from external WWW servers usingthe public IP address, which must be a registered IP address. The sourceaddress on all IP packets that leave the proxy--regardless of their actualpoint of origin--is the proxy's public IP address.
Because Ralf installed BorderManager on a server with only one networkinterface board, he needed to configure the same IP address for both publicand private use. On the BorderManager Setup screen in the NWADMIN utility,Ralf specified that BorderManager's IP address (the IP address bound tothe FDDI network interface board) was for both public and private use. (See Figure 3.)
Figure 3: Ralf configured the proxy component of BorderManager to use the same IP address for both public and private use.
Configuring Browsers
After specifying BorderManager's IP address, Ralf configured users' browsersto use the proxy component of BorderManager to access all internal and externalWWW servers except server AIM, a local WWW server. To configure these browsers,Ralf launched Netscape Navigator, selected the Network Preferences optionfrom the Options menu, and selected the Proxies option. Next, he selectedthe Manual Proxy Configuration option and clicked View.
Ralf then completed the FTP Proxy, Gopher Proxy, and HTTP Proxy fieldsby entering the proxy's host name and the port number (8080) that users'browsers use to access this proxy. In the No Proxy field, Ralf entered theAIM server's host name. As a result, requests for WWW files on the AIM serverare sent directly to this server rather than to the proxy component of BorderManager.
Configuring Communications
Finally, Ralf needed to configure the proxy component of BorderManagerto communicate with Netscape Proxy Server--users' single point of accessto the Internet and Siemens' intranet. Although configuring the proxy tocommunicate with Netscape Proxy Server is a relatively simple process, thetechnology that makes this communication possible is quite complex. (See the "Configuring the Neighborhood" section.)
CERN Versus Harvest-Squid
Netscape Proxy Server and the proxycomponent of BorderManager use different caching technologies. NetscapeProxy Server is based on CERN, the first-generation caching technology developedby the European Laboratory for Particle Physics. The proxy component ofBorderManager, on the other hand, is based on Harvest-Squid, a second-generationcaching technology. The original Harvest caching project was jointly developedby the University of Colorado and the University of Southern California.This project was later transferred to the National Laboratory for AdvancedNetwork Research (NLANR), and the technology was renamed Squid.
One of the differences between a CERN proxy and a Harvest-Squid proxyis the extent to which each proxy can use the cache on other proxy servers.If you configure a CERN proxy to use another proxy's cache, the CERN proxyforwards requests to the other cache without first verifying whether thiscache is functioning. However, if you configure a Harvest-Squid proxy touse another proxy's cache, the Harvest-Squid proxy sends a query to theother proxy to verify that it is functioning before forwarding the request.
Internet Caching Protocol
To locate WWW files stored in anotherproxy's cache, a Harvest-Squid proxy uses the Internet Caching Protocol(ICP). CERN proxies such as Netscape Proxy Server, on the other hand, donot use or understand ICP. To understand how ICP works, suppose that youconfigured PROXY-A to communicate with other proxies, which would then becalledneighbors. Further suppose that PROXY-A and its neighbors,PROXY-B and PROXY-C, were based on the Harvest-Squid technology.
When PROXY-A received a request for a WWW file that PROXY-A did not havein its own cache, it would send an ICP query to PROXY-B and PROXY-C. Assumingthat both proxies responded to the query (indicating that they are functioning),PROXY-A would forward the request for the file to PROXY-B and PROXY-C. Theseproxies would then return an ICP packet indicating a hit ("I have thefile") or a miss ("I don't have the file").
If both PROXY-B and PROXY-C had the WWW file, PROXY-A would retrievethis file from whichever neighbor returned the first HIT packet. If neitherPROXY-B nor PROXY-C had the file, either PROXY-A would retrieve this filefrom the file's source server, or PROXY-B or PROXY-C would retrieve thefile on behalf of PROXY-A, depending on how you had configured PROXY-A.
Parent, Peer, and Cascade Neighbors
You can configure the proxycomponent of BorderManager to communicate with three types of neighbors:
Parent neighbors
Peer neighbors
Cascade neighbors
You can configure only other BorderManager or Harvest-Squid proxies asparent neighborsorpeer neighborsto a BorderManager proxy.When a parent neighbor receives a request for a WWW file, the parent neighborchecks its own cache. If the parent neighbor has the file in cache, thisneighbor returns the file to the BorderManager proxy. If the parent neighbordoes not have the file, this neighbor retrieves the file from the file'ssource server.
When a peer neighbor receives a request for a WWW file, the peer neighboralso checks its own cache. However, if the peer neighbor does not have thefile, this neighbor responds only with a MISS packet. A peer neighbor willnot retrieve a file from the file's source server.
For example, suppose that PROXY-B were a parent neighbor to PROXY-A andPROXY-C were a peer neighbor to PROXY-A. If PROXY-A sent PROXY-B and PROXY-Ca request for a WWW file and neither neighbor had the file in its own cache,PROXY-C would return a MISS packet, and PROXY-B would retrieve the filefrom the file's source server and return this file to PROXY-A.
You can configure CERN proxies only asCascadeneighborstothe proxy component of BorderManager. Like a parent neighbor, a Cascadeneighbor can retrieve a WWW file from the file's source server if the Cascadeneighbor does not have the file in its own cache.
Configuring the Neighborhood
To communicate with Netscape ProxyServer, the proxy component of BorderManager needed rights to access NetscapeProxy Server. After these rights were granted, Ralf enabled the proxy'sICP, specified that the proxy had one neighbor, and indicated the neighbor'shost name, type, and ICP port number by completing several steps:
Ralf clicked the Advanced . . . button on the Web Proxy Cache screen in the NWADMIN utility.
From the Advanced . . . menu, Ralf selected each of the tabs that enabled him to configure Advanced options for the proxy component of BorderManager. For example, these options allow you to configure the directory in which the proxy should cache data, the maximum size of cached files, the time-to-live value for cached files, and which URLs should not be cached. Ralf accepted the default values for most of the Advanced options. He then selected the Hierarchy tab to access the Neighbors list, which allowed him to configure the proxy's neighborhood. (See Figure 4.)
Figure 4: To enable the proxy component of Border Manager to communicate with Netscape Proxy Server, Ralf had to configure the proxy's neighborhood.
In the Neighbors list, Ralf specified that the proxy had only one neighbor, which was a Cascade neighbor. (As mentioned earlier, Netscape Proxy Server is a CERN proxy, which can be configured only as a Cascade neighbor.) Ralf also specified the host name and ICP port number that the proxy component of BorderManager should use to access Netscape Proxy Server.
Ralf set Netscape Proxy Server's ICP port number to 7, a User Datagram Protocol (UDP) echo port. As a CERN proxy, Netscape Proxy Server cannot use or understand ICP. To compensate for this inherent weakness in CERN proxies, you must direct the proxy component of BorderManager to use the UDP echo port to access Cascade neighbors. Then when the proxy sends an ICP query to a Cascade neighbor, that neighbor's UDP echo port echoes the query, tricking the proxy into believing that it received a response.
For example, suppose that a user on the TD network sent the proxy component of BorderManager a request for a WWW file. If this file were not in the proxy's cache, the proxy would send an ICP query to Netscape Proxy Server's UDP echo port, which would echo the query. When the proxy received the echo, this proxy would believe that it had received a response and would determine that the Cascade neighbor was functioning. The proxy would then forward the request to Netscape Proxy Server, which would check its own cache for the file. If the file were in cache, Netscape Proxy Server would return this file. If the file were not in cache, however, Netscape Proxy Server would retrieve this file from the file's source server (just as any other parent or Cascade neighbor would do).
THE PROVERBIAL HITCH
Although installing and configuring the beta version of BorderManagerwas relatively painless, this process did have the proverbial hitch. WhileRalf was able to get the proxy component of BorderManager to resolve DomainName System (DNS) names for hosts on Siemens' intranet, he was unable toget the proxy to resolve DNS names for Internet hosts.
The TD network does not have its own DNS server. Without a DNS server,the proxy component of BorderManager relies on DNS servers on the Erlangennetwork to resolve DNS names of Internet and intranet hosts. Netscape ProxyServer on the Erlangen network resolves the DNS names of intranet hosts.Because the TD network has a routing entry to Netscape Proxy Server, theproxy component of BorderManager can resolve DNS names of intranet hosts.
However, to resolve DNS names of Internet hosts, Netscape Proxy Serverconsults another DNS server--a server to which the TD network does not havea routing entry. Therefore, the proxy component of BorderManager cannotresolve DNS names of Internet hosts.
To enable the proxy component of BorderManager to resolve DNS names ofInternet hosts, Ralf has two choices: He can install a DNS server on theTD network, or he can establish a routing entry to the DNS server on theErlangen network that resolves DNS names of Internet hosts. Ralf would ratherset up a DNS server on the TD network than establish another routing entry,but at this point, he does not know what the TD networking team will do.However, Ralf will implement one of these solutions before he installs BorderManageron his live network.
THE PLAN, THE PLAN!
Ralf plans to install BorderManager on server AIM after BorderManageris released later this summer. Ralf then plans to enable both passive andreverse caching to reduce traffic on the WAN link between the TD and Erlangennetworks.
Reverse caching will also enable Ralf to install as many Novell Web Serverson his network as he wants--without having to establish routing entriesfor each server."The goal,"Ralf explains,"is to let theoutside world know only about our proxy, no matter how many WWW serverswe have physically."Thus, if a user on Siemens' intranet outside theTD network requests information stored on one of TD's WWW servers, BorderManagerwill service that request--reducing traffic on the TD network and sparingRalf the inconvenience of establishing routing entries for each WWW serverhe installs.
Linda Boyer works for Niche Associates, which specializes in technicalwriting.
NetWare Connection, August 1997, pp.22-31
* Originally published in Novell Connection Magazine
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.