Novell SuperLab Hosts the SuperLab Challenge
Articles and Tips: article
Software/Hardware Test Associate
Novell SuperLab
01 Feb 1998
A lot can be learned when network servers are put to the ultimate stress test, and this AppNote describes just such an event that took place recently in Novell's SuperLab.
- Introduction
- About the Novell SuperLab
- SuperLab Challenge July 1997
- Network Design
- Results
- Future SuperLab Challenges
Introduction
The SuperLab Challenge is the ultimate test environment, requiring several engineers working full-time for weeks just to set it up. Novell's SuperLab is one of the best equipped labs in the world, and one of the only labs that can offer a test of this scope. In a structured event such as the SuperLab Challenge, all of these resources can be pooled together, allowing several hardware manufacturers to take advantage of this test at the same time.
The idea of the SuperLab Challenge began with a test called ServerFest. Several hardware manufacturers had requested time in the SuperLab to run large scale tests on their hardware. Instead of building a separate test environment for each hardware vendor, we created a single environment that would challenge any server. Consequently, several servers could be cycled through the same test configuration. A test of this size took months to plan, and it would be unrealistic to test each server at a different time. This was not a certification, but a stress test.
This event was held three times, in December 1996, and in January and March of 1997. In July 1997, the name of the event was changed to the SuperLab Challenge. This name change was made to broaden the scope of the test and to expand it beyond server hardware testing.
The aim of this AppNote is to show the benefits that the SuperLab Challenge brings to Novell and to Novell partners. Specifically, the focus will be on the July 1997 SuperLab Challenge test specifications and results. The test setup for the July SuperLab Challenge was more rigorous than ServerFest had been. This AppNote focuses primarily on this test configuration.
Note: Throughout this AppNote, the term "client" refers to a machine used in testing.
About the Novell SuperLab
The Novell SuperLab is a unique testing facility because of its size and automation capabilities. Several years in the making, this lab has become one of the most efficient labs in terms of test configuration and setup.
The main floor of the SuperLab holds 1,386 machines. SuperLab engineers have control of each of these machines from a single console. This control includes the ability to execute batch files on the computers and to control power to each machine. The SuperLab also makes use of disk imaging technology. This allows complicated test scenarios to be completed in a fraction of the time it would take to complete by hand. In addition to the main floor, the SuperLab has approximately 500 machines in satellite labs that are directly connected to the same backbone as the primary lab.
To segment the lab, the SuperLab uses Virtual LAN technology, which allows network configurations to be changed quickly and without re-cabling. These virtual LANs are completely isolated and have no effect on one another. In addition, the SuperLab has a telecom lab which provides WAN testing capabilities through WAN simulators, a PBX system, and satellite uplinks. Recently, the telecom lab has been set up as a test ISP environment. Also implemented were ATM switches and a Cisco 7505 router dedicated to testing. With all this technology, the SuperLab provides a stable, realistic environment for large scale testing.
SuperLab Challenge July 1997
For the July 1997 SuperLab Challenge, we decided to mirror a real-life business situation as closely as possible. To do this, we created a fictitious company, and based the test configuration on the network services this company would need. A server participating in the SuperLab Challenge was required to support the network of the fictitious "Cashmere Electronic Shipping Company" for one business day, or eight hours.
Our test network was more demanding than most companies' networks would be on a single server, but we wanted to demonstrate an extreme case. Several popular software applications were run by the clients, and other typical network service activities were required of the participating server. The test ran 24 hours a day for one week. Each server was allotted a 10 hour time period--eight hours for testing and two for set up. To pass the Novell SuperLab Challenge, a server needed to authenticate the first 1,000 clients within 1 hours, and maintain at least 1,000 connections for the remainder of the eight-hour test period.
The Fictitious Company
The network for Cashmere Electronic Shipping is a LAN throughout the building, with a WAN connection to allow customers access to the web server. There are over 1000 employees working for this company, and each employee has a computer attached to the network. Client operating systems used in this environment are Windows 95, Windows NT, and Windows 3.1 running on MS-DOS v6.22.
Cashmere Electronic Shipping is made up of four departments: Accounting, Marketing, Administration, and Research and Development. Each department has different computing needs and will place different demands on the server. Every machine used in the SuperLab Challenge was assigned to a specific department of the company before the test setup began. Figure 1 below shows the total test-time a client in a specific department spent running each application.
Figure 1: Client test time of each application by department.
Some user activities on the server are the same regardless of which department the user belongs to. These activities are login, logout, and local virus scanning. About 2% of the total run-time was spent performing each of these activities. Therefore, 4% of the time is labeled as miscellaneous in Figure 1.
Server Requirements
Each server participating in the SuperLab Challenge needed to have at minimum the following specifications:
256 MB of RAM
4 Intel Pentium processors
8 GB hard drive space
Support 4 PCI 100 MB network cards
The configuration required on a server to participate in the SuperLab challenge was very complex. Setup time took over 12 hours on average. Each of the following products was installed on the server before testing:
Novell NetWare 4.11, with Service Pack 3
Novell IntranetWare Web Server v3.1
Novell GroupWise 5.1
Corel Office 8
PerfectOffice 3.0
Oracle Database 7.3.2.2.1
Cheyenne InnocuLAN for NetWare v4.0
Test Specifications
When looking at the scope of this test, it is easy to see where the name "challenge" came from. Listed below are each of the different functions the server had to manage during its eight-hour test period in the SuperLab Challenge.
The participating server acted as a web server to the over 1000 test clients, and to at least 63 external machines that were continuously accessing www.cashmere.com across a T1 line throughout the test. These clients were not included in the count of users authenticating to the server, but were an additional load.
The server ran a GroupWise post office, and handled a large amount of e-mail messages which were sent continuously throughout the test.
A serial board from Digi International was installed in each server and attached to a modem pool via a PBX for sending faxes through the server. Throughout the test, users sent faxes across these modems using the Cheyenne FaxServe Gateway for GroupWise.
The server ran Cheyenne InnocuLAN virus detection software to scan all incoming files to the server, and periodically users would run this program against their local hard drives to check for viruses.
The server maintained an Oracle database, which users frequently accessed during the test. User activities, such as Corel 8 applications, generated traffic on the server by opening and saving files in server directories. All client activities, except for Netscape 3.0, were installed and run off of the server.
A second server was attached to the test network and installed into the Novell Directory Services (NDS) tree. This required the participating server to perform NDS replication throughout the test.
The server was required to provide routing across four separate LAN segments, using both IP and IPX packets.
Client Automation
A primary feature of the SuperLab Challenge was the automation of several user activities. All automation scripts for the SuperLab Challenge were developed in the SuperLab. The scripts were designed with the goal of the SuperLab Challenge in mind--to create a test situation that represented an actual business environment. This required user machines to run different business applications randomly throughout the day.
The fictitious company was conceptualized before any test code was written. The SuperLab performed research to determine what type of applications are being used by real people in real companies. Based on this research, we divided our company into four departments: Accounting, Research and Development, Marketing, and Administration. For each department, we chose five or six applications and estimated the amount of time a user spent running each application per day.
After determining this mock company situation, it was necessary to develop automation that would allow each machine to run applications randomly and mirror the company profile. Separate automation scripts were written for each department. Because different operating systems were used in the test, some departments required multiple scripts.
Machines running these scripts would determine at run-time which application it would run (and what activities within that application), based on a statistical application running in the background. This gave realistic and random results. Users would also wait from two to ten minutes between running each application, to make the test more realistic.
The end result of this automation was a more true-to-life test environment that, for example, allowed a client identified as part of the Accounting department to run a spreadsheet program for a specified amount of time, then a database program, and then perhaps send an e-mail message, but never in the same order.
Client Applications
Client automation scripts were written for the following applications:
Novell GroupWise 5.1. Clients generated and received e-mail messages within the local network. Also within this script, clients sent faxes through GroupWise using Cheyenne's FaxServe.
Corel WordPerfect 8. Typical word processing activities were performed by the clients running this script. Generated files were saved onto, and accessed from a client's individual directory on the server. This application was run off of the server.
Corel QuatroPro 8. Clients ran this application off of the server. Spreadsheet files were created and edited, calculations were made, and generated files were saved on the server.
Corel Presentations 8. Clients edited and ran slide show presentations. At times, clients running this script would also open and edit drawing files. All edited files were saved onto the server. This application also ran off of the server.
Oracle Database 7.3.2.2.1. The server maintained a database which was accessed and updated by clients through queries and data adding scripts.
Netscape 3.0. Each client had a list of web pages it accessed throughout the test. The server worked as a DNS server to facilitate these requests. Netscape itself was run locally.
InnocuLAN. Clients opened InnocuLAN from the server and performed a virus scan of their local hard drive.
Client Specifications
The following applications were installed on each client:
Netscape 3.0
Database front-end
Corel Office 8 (Perfect Office 3.0 for 16-bit clients), client front-end only
Either Windows 95, Windows 3.11, or Windows NT operating systems
Novell Client 32
Client automation scripts
Network Design
The network design for the SuperLab Challenge consisted of approximately 1100 individual clients on four Ethernet segments. The server was required to have four network cards, one for each segment. A simulated T1 line coming in on segment 1 allowed an additional 63 external clients to access the server's web page.
The server was also equipped with a serial board, which was attached to a modem pool. This sent faxes through the SuperLab's PBX system to 8 machines designated as fax receiving clients.
A NetWare MPR server, coming over a T1 line, was added to Directory Services for NDS Replication. A client automation master console was attached to segment 4 and used to collect data from the clients throughout the test.
Figure 2: Network topology diagram.
Results
The following servers participated in and passed the July 1997 SuperLab Challenge:
Compaq ProLiant 6000
DELL 6100
Mitsubishi Apricot FT 4200
NetFRAME 9000
Unisys SFR HS6200121
Again, the SuperLab Challenge was not meant to be a certification, but a chance for server manufacturers to show that their hardware can handle a real-life scenario. The servers which passed the SuperLab Challenge have shown to be reliable under heavily stressed conditions.
This test was designed to stress a large production server, and only high-powered production level servers were allowed to participate. The server requirements mentioned above are the minimum resources a server would need to successfully run the test. All of the servers which passed the SuperLab Challenge met or surpassed these minimum requirements.
Future SuperLab Challenges
The goal of the SuperLab Challenge is to showcase the power and scalability of Novell products and Novell partner products. Each SuperLab Challenge will be designed with this goal in mind, although the test will not be the same every time.
The SuperLab plans to continue offering the SuperLab Challenge quarterly when possible. For the time being, SuperLab resources are dedicated to the Moab (NetWare 5) development effort. The next SuperLab Challenge is scheduled to be held shortly after Moab ships later this year.
* Originally published in Novell AppNotes
Disclaimer
The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.