Novell is now a part of Micro Focus

Integrating Log4j and the Novell exteNd Application Server

Articles and Tips: article

Jeff Sackett
Field Application Engineer, exteNd
Novell, Inc.
jsackett@novell.com

01 Mar 2003


This AppNote describes how to integrate the log4j open source logging tool for Java into the Novell exteNd Application Server.


Topics

log files, Java application development

Products

Novell exteNd Application Server

Audience

developers

Level

intermediate

Prerequisite Skills

familiarity with Java programming

Operating System

n/a

Tools

log4j, LogFactor5

Sample Code

yes

Introduction

This AppNote describes how to integrate the log4j logging tool into the Novell exteNd Application Server. It covers the following topics:

  • Inserting log statements into code

  • Deploying log projects server-wide and in EARs (enterprise archives), EJBs (Enterprise Java Beans), WARs (Web archives), and CARs (client application archives)

  • Basic log4j configuration (including options for logging to the console, to files, and remote logging)

  • Dynamically changing log properties at run time

  • Analyzing log files with the LogFactor5 GUI (locally and remotely)

  • Using Nested Diagnostic Context (NDC) to measure timing in multi-threaded environments

Overview of Logging Mechanisms

Determining how and when to log messages is an important part of any application development project. The Apache Software Foundation has developed a standard logging tool called log4j. Log4j provides multiple logging options that can be determined at deployment time. It is also flexible enough to allow logging configurations to be changed dynamically at run time. The home page for log4j is http://jakarta.apache.org/log4j/docs/index.html.

The Java Community process has also created a separate logging process that will be included as part of JDK 1.4. Currently, most reviews still recommend log4j over Java Logging. As part of the multiple choices of logging APIs, the Apache Software Foundation has also introduced the Commons Logging project. This is an ultra-thin API designed to allow the logging API to be chosen at deployment time instead of development time. Currently the Commons Logging project supports both the JDK 1.4 logging and log4j. It is also easily extendible to other logging APIs. For this reason, all code samples provided with this AppNote will use the Commons Logging API for logging purposes, except for the NDC example (because the Commons Logging API does not support NDC). The home page for the Commons Logging project is http://jakarta.apache.org/commons/ logging.html.

Another useful tool is LogFactor5, a Swing-based GUI that leverages the power of the log4j logging framework and provides develoeprs with a sophisticated interface for managing log messages. You can find out more about LogFactor5 at http://jakarta.apache.org/log4j/lf5/overview.html.

It is recommended that you read a document entitled "Short Introduction to log4j" (found at http://jakarta.apache.org/log4j/docs/manual.html) prior to reading this AppNote. Although some basic concepts will be covered here, the short introduction will provide a solid foundation for understanding concepts presented in this AppNote.

Another interesting section of the log4j documentation is a description of its performance for different types of layouts and appenders. This can be found at http://jakarta.apache.org/log4j/docs/api/org/apache/log4j/performance/Logging.html.

About the Sample Code

A sample exteNd Workbench project is available for download in conjunction with this AppNote. Click on the download button below or go to http://www.novell.com/coolsolutions/tools/15536.html to download the files.

The name of the database used in the example is Applications. No user-defined schema needs to be included with this database. The WAR URL is http://localhost/Applications/SimpleWAR, which will bring up links to the different pages used.

Note: You may have to attempt a browser refresh of the pages in order to trigger the request to the server.

Batch files are included in the lf5 subdirectory to assist in starting the LogFactor5 GUI. Sample configuration files are included in the lf5 and config subdirectories.

Inserting Log Statements into Code

The code in this section shows the basics of how to insert log statements into the code. The necessary components are the two import statements to the Commons Logging interface: the initialization of the logging variable log, and the code to output the log statement. In this code, a check is done to see if the particular leveling of logging is enabled. Although the call to output the log statement is very thin, the creation of the actual log statement can be quite expensive. For this reason, it is a good habit to always check to see if the log level is enabled.

Six levels of logging can be used with the Commons Logging API: trace, debug, info, warn, error, and fatal. However, when using log4j as the underlying logging engine, trace is the same as debug. It is important to determine what the levels mean in reference to the production environment instead of focusing on development time. A good rule of thumb is the following:

  • debug - Shows detailed program flow. This level is only used in error resolution or development mode.

  • info - Shows basic program flow. These are used for timing purposes and basic debugging of information. They commonly show method entry and exit and user-defined parameters. This level is typically not enabled during production.

  • warn - Shows that an expected error condition has occurred. These are sent to the console but are not considered serious enough to keep a permanent log on.

  • error - Messages are designed to be checked periodically to indicate that a potential system or coding error exists. These are typically exceptions that are caught unexpectedly. I like to have these messages stored to a separate log file for easy analysis of potential problems.

  • fatal - Tied to a urgent messaging system such as e-mail or a JMS queue to indicate the system is in a current failure

Here is the code:

/**
* Copyright 2002 Novell, Inc. All Rights Reserved.
* This software is for demonstration purposes only.  
* No warrenty is implied or expressed.
*/

package com.novell.log4j.web;

// Include these two import statements to enable commons logging.
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

/**
* SimpleMessage
* A very basic class to simply output a log message.
*/
public class SimpleMessage{

   /**
   * Create a logging instance variable.
   */
   private static Log log = LogFactory.getLog(SimpleMessage.class);

   /**
   * This is the alternate way to initialize log4j if the log variable does not
   * need to be accessed by a static function.
   */
   // private Log log = LogFactory.getLog(this.getClass());

   /**
   * A basic function to print a message to the log
   */
   public static void printMessage(){
      if(log.isInfoEnabled()){
         log.info("Simple Message");
      }
   }
}

Deploying Log Projects

When deploying projects using log4j and Commons Logging, there is only one major consideration: where to place the log4j jar file and associated properties file.

When using the Commons Logging project, log4j is the first logger searched for by default; therefore you can let the Commons Logging default automatically. A more detailed description of the Commons Logging initialization can be found at http://jakarta.apache.org/commons/logging/api/org/apache/commons/logging/LogSource.html.

How logging is initialized is dependent on the class loaders associated with the Novell exteNd Application Server. (For more details, see the Application Server documentation at http://www.novell.com/documentation/lg/extendas.) The basic order is the following, from highest to lowest: WAR, EAR, and then server-wide.

The following sections describe different ways to deploy logging projects.

Default Server Logging

To set up logging for all applications deployed to a server from a single point, insert the log4j-1.2.7.jar and commons-logging.jar files in the AppServerInstallDir/jre/lib/ext directory. Then place the log4j.properties file in the AppServerInstallDir/jre/lib/ext/classes directory.

Note: When using this configuration and then trying to deploy an EAR that contains the log4j.jar, a deployment error will be generated. To avoid this error, undo the system logging during the deployment.

EAR-Based Logging

To set up logging by an EAR, simply include the log4j-1.2.7.jar and commons-logging.jar files in the EAR. Typically I place these in a lib directory. Also, the properties configuration file must be placed in a jar and inserted in the EAR. In the example, the properties.jar contains the configuration file.

Note: If multiple property files are deployed in the EAR either in WARs or EJB jars, the property file actually loaded will not be guaranteed.

Figure 1 shows a typical EAR setup. Although normally you would have to define the Class-Path in the manifest.mf file, the example works without having to define the Class-Path.

Figure 1: A typical EAR setup.

EJB-Based Logging

When doing EJB-based logging, the jars must be deployed either within an EAR or as an extension. The properties file can be deployed in the EJB jar; however, the properties file will affect logging for either the entire EAR or system depending on where the jar is loaded from.

WAR-Based Logging

When doing WAR-based logging, place the log4j.jar in the /WEB-INF/lib directory and the properties file in the /WEB-INF/classes directory.

Note: When the log4j.jar file is included as part of the EAR, the properties jar will not be loaded and the property file will affect the entire EAR.

CAR-Based Logging

When doing CAR-based logging, all the clients can be set up the same as in the Default Server Logging section, or the deployment plan for the EAR can be modified to reference the logging jars. The following code shows the appropriate section of the deployment plan.

<module isObject="true">
<carJar isObject="true">
      <carJarName type="String">SimpleEJBCar.jar</carJarName>
      <beanReferenceList isObject="true">
         <beanReference isObject="true">
            <name type="String">ejb/SimpleEJB</name>
            <beanLink type="String">ejb/SimpleEJBBean</beanLink>
         </beanReference>
      </beanReferenceList>
      <usesJars type="StringArray">
         <el>lib_commons-logging.jar</el>
         <el>lib_log4j-1.2.7.jar</el>
         <el>SimpleEJB-client.jar</el>
      </usesJars>
   </carJar>
</module>

Log4j Configuration

When configuring log4j, you need to understand two important elements: appenders and loggers.

An appender is an output element that defines where the output is going. Here is a sample list of appenders:

  • Console outputs the messages to the console.

  • File outputs the messages to a file.

  • Rolling File Appender outputs the messages to a file. When the file reaches a certain size, the current file is copied to a backup file and a new file is started. The number of backup files is defined at configuration time.

  • Socket Appender is used to output messages to a remote system.

  • Log Factor 5 Appender starts the LogFactor5 GUI.

  • SMTP Appender sends e-mail messages based on errors.

Appenders also define the threshold and layout of the message to be output. The most common format is defined by the pattern layout. To see how the pattern layout works, see http://jakarta.apache.org/log4j/docs/api/org/apache/log4j/PatternLayout.html.

Loggers are the categories defined by the code. They are typically fully-qualified class names or packages, such as com.novell.TestObject or simply com. Loggers have a threshold and a list of appenders that will be referenced when a message is to be output.

Below is a sample configuration file with several appenders defined. It uses appender-level thresholds to determine the level of messages to be output by the appender and removes the Struts messages below the level of Warn for the logs. (Struts is a framework from the Apache Software Foundation to simplify the retrieval of a multipart form. See http://jakarta.apache.org/struts/index.html.)

#
# Set root logger level to warning and its only appender to console.
#
log4j.rootLogger=DEBUG, CONSOLE, MAIL, ROLLING_FILE

#
# Raise level of logging for org.apache to WARN and disable from ROLLING_FILE and
# MAIL appenders
#
log4j.logger.org.apache=WARN, CONSOLE
log4j.addivity.org.apache=false

###################
# Console Appender
###################

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.Threshold=DEBUG
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

#####################
# File Appender
#####################
log4j.appender.FILE=org.apache.log4j.FileAppender
log4j.appender.FILE.File=C:/Data/Log4JPaper/logs/file.log
log4j.appender.FILE.Append=false
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

# Use this layout for LogFactor 5 analysis
#log4j.appender.CONSOLE.layout.ConversionPattern=[slf5s.start]%d{DATE}
[slf5s.DATE]%n%p[slf5s.PRIORITY]%n%x[slf5s.NDC]%n%t[slf5s.THREAD] n%c[slf5s.CATEGORY]%n%m[slf5s.MESSAGE]%n%n

########################
# Rolling File Appender
########################
log4j.appender.ROLLING_FILE=org.apache.log4j.RollingFileAppender
log4j.appender.ROLLING_FILE.Threshold=ERROR
log4j.appender.ROLLING_FILE.File=C:/Data/Log4JPaper/logs/rolling.log
log4j.appender.ROLLING_FILE.Append=true

log4j.appender.ROLLING_FILE.MaxFileSize=10KB
log4j.appender.ROLLING_FILE.MaxBackupIndex=1
log4j.appender.ROLLING_FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.ROLLING_FILE.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

####################
#  Socket Appender
####################
log4j.appender.SOCKET=org.apache.log4j.RollingFileAppender
log4j.appender.SOCKET.RemoteHost=localhost
log4j.appender.SOCKET.Port=5001
log4j.appender.SOCKET.LocationInfo=true

# Set up for Log Facter 5
log4j.appender.SOCKET.layout=org.apache.log4j.PatternLayout
log4j.appender.SOCET.layout.ConversionPattern=[slf5s.start]%d{DATE}[slf5s.DATE]
%n%p[slf5s.PRIORITY]%n%x[slf5s.NDC]%n%t[slf5s.THREAD]%n%c[slf5s.CATEGORY]%n%m
[slf5s.MESSAGE]%n%n

########################
# Log Factor 5 Appender
########################
log4j.appender.LF5_APPENDER=org.apache.log4j.lf5.LF5Appender
log4j.appender.LF5_APPENDER.MaxNumberOfRecords=2000

########################
# SMTP Appender
#######################
log4j.appender.MAIL=org.apache.log4j.net.SMTPAppender
log4j.appender.MAIL.Threshold=FATAL
log4j.appender.MAIL.BufferSize=10
log4j.appender.MAIL.From=log4j@novell.com
log4j.appender.MAIL.SMTPHost=mail.attbi.com
log4j.appender.MAIL.Subject=Log4J Message
log4j.appender.MAIL.To=jsackett@novell.com

log4j.appender.MAIL.layout=org.apache.log4j.PatternLayout
log4j.appender.MAIL.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

Log4j can also use XML sheets to define the configuration. Here is a sample of the same configuration, except this one is in XML format.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<param name="Threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d %-5p [%t] %C{2} (%F:%L) - %m\n"/>
</layout>
</appender>

<appender name="FILE" class="org.apache.log4j.FileAppender">
<param name="File" value="C:/Data/Log4JPaper/config/file.log" />
<param name="Append" value="false" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"

value="[slf5s.start]%d{DATE}[slf5s.DATE]%n%p[slf5s.PRIORITY]%n%x[slf5s.NDC]%n%t[slf5s.THREAD]%n%c[slf5s.CATEGORY]%n%m[slf5s.MESSAGE]%n%n"/>
</layout>
</appender>

<appender name="ROLLING_FILE" class="org.apache.log4j.RollingFileAppender">
<param name="Threshold" value="error" />
<param name="File" value="C:/Data/Log4JPaper/config/rolling.log" />
<param name="Append" value="false" />
<param name="MaxFileSize" value="10KB" />
<param name="MaxBackupIndelx" value="1" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d %-5p [%t] %C{2} (%F:%L) - %m\n"/>
</layout>
</appender>

<appender name="SOCKET" class="org.apache.log4j.net.SocketAppender">
<param name="RemoteHost" value="localhost"/>
<param name="Port" value="5001"/>
<param name="LocationInfo" value="true"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="[slf5s.start]%d{DATE}[slf5s.DATE]%n%p[slf5s.PRIORITY]%n%x[slf5s.NDC]%n%t[slf5s.THREAD]%n%c[slf5s.CATEGORY]%n%m[slf5s.MESSAGE]%n%n"/>
</layout>
</appender>

<appender name="MAIL" class="org.apache.log4j.net.SMTPAppender">
<param name="Threshold" value="fatal" />
<param name="BufferSize" value="10"/>
<param name="From" value="log4j@novell.com"/>
<param name="SMTPHost" value="mail.attbi.com"/>
<param name="Subject" value="Log4J Message"/>
<param name="To" value="jsackett@novell.com"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d %-5p [%t] %C{2} (%F:%L) - %m\n"/>
</layout>
</appender>

<appender name="LF5Appender" class="org.apache.log4j.lf5.LF5Appender">
<param name="MaxNumberOfRecords" value="2000"/>
</appender>

<category name="org.apache" additivity="false">
<priority value="warn" />
<appender-ref ref="CONSOLE" />
</category>

<root>
<priority value="debug"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="MAIL"/>
<appender-ref ref="ROLLING_FILE"/>
</root>
</log4j:configuration>

Dynamically Changing Log Properties

The following code allows the user to change logging properties "on the fly" by submitting an XML configuration file through a Web page.

/**
* Copyright 2002 Novell, Inc. All Rights Reserved.
* This software is for demonstration purposes only.  
* No warranty is implied or expressed.
*/

package com.novell.log4j.web;

import org.apache.struts.action.Action;
import org.apache.struts.action.ActionForward;
import org.apache.struts.action.ActionForm;
import org.apache.struts.action.ActionMapping;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.apache.log4j.xml.DOMConfigurator;
import org.apache.log4j.LogManager;

/**
* LoadLog4jAction - Loads a XML Configuration file to reconfigure logging
*/
public class LoadLog4jAction extends Action {

   private Log log = LogFactory.getLog(this.getClass());

   public ActionForward execute(ActionMapping mapping,
                           ActionForm form,
                           HttpServletRequest request,
                           HttpServletResponse response) throws Exception{
      try {
         System.out.println("Entered");
      Log4jForm data = (Log4jForm) form;
         // Get the file to load
      FormFile file = data.getFile();

      if(file != null){
               // Configure log4j.
            System.out.println("Configuring");
         DOMConfigurator configurator = new DOMConfigurator();
         configurator.doConfigure(file.getInputStream(), 
         LogManager.getLoggerRepository());
         }
         return mapping.findForward("success");
      } catch(Exception e) {
         if(log.isErrorEnabled()) {
            log.error("Actual Error", e);
         }
         throw e;
      }
   }
}

This servlet is a Struts-based action using the Struts framework. The URL for the input page is http://localhost/Applications/SimpleWAR/action/LoadXML. The appearance of the page is shown in Figure 2.

Figure 2: Appearance of the input page.

Analyzing Log Files Using LogFactor5

LogFactor5 is a GUI tool included with the latest release of log4j. It allows log files to be formatted in a format for LogFactor5, and allows error messages to be color-coded and displayed by level or category.. To start the GUI, run the following lines from the lf5/run_lf5.bat file:

@echo off
java -cp "..\lib\log4j-1.2.7.jar" org.apache.log4j.lf5.StartLogFactor5

Figure 3 shows the results of analyzing the file logs/lf5file.log in the attached sample code.

Figure 3: Results of log file analysis.

Using LogFactor5 for Remote Monitoring of Logs

You can also use LogFactor5 for remote logging. To configure the sending of messages to a remote server, load lf5/lf5NetConfig.xml using the Load Configuration page. Then run the batch file Run_LF5_Socket.bat to start the LF5 GUI. The relevant sections of the documents are shown below.

XML Config:

   <appender name="SOCKET" class="org.apache.log4j.net.SocketAppender">
      <param name="RemoteHost" value="localhost"/>
      <param name="Port" value="5001"/>
      <param name="LocationInfo" value="true"/>
      <layout class="org.apache.log4j.PatternLayout">
         <param name="ConversionPattern" 
value="[slf5s.start]%d{DATE}[slf5s.DATE]%n%p[slf5s.PRIORITY]%n%x[slf5s.NDC]%n%t
[slf5s.THREAD]%n%c[slf5s.CATEGORY]%n%m[slf5s.MESSAGE]%n%n"/>
      </layout>
   </appender>

Server Properties to Start LogFactor5 GUI:

log4j.rootCategory= DEBUG ,A1
log4j.category.org.apache.log4j.net=DEBUG

# A1 is set to be a LF5Appender which outputs to a swing
# logging console.

log4j.appender.L5Appender=org.apache.log4j.lf5.LF5Appender
log4j.appender.L5Appender.MaxNumberOfRecords=1000

Batch file to start GUI:

@echo off
java -cp "..\lib\log4j-1.2.7.jar" org.apache.log4j.net. SimpleSocketServer 5001 socketserver.properties

Using Nested Diagnostic Context (NDC)

In real-time multi-threaded environments, it is often helpful to measure a trace or timing of a single thread. To facilitate these types of measurements, log4j provides Nested Diagnostic Context (NDC). When NDC is used, an additional stamp is applied to each message that can be filtered when using LogFactor5.

Note: Currently the Commons Logging does not support NDC; therefore, all NDC calls must be initiated directly from the log4j library.

The following code shows how to initiate NDC.

/**
* Copyright 2002 Novell, Inc. All Rights Reserved.
* This software is for demonstration purposes only.  
* No warranty is implied or expressed.
*/
package com.novell.log4j.web;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.apache.log4j.NDC;

public class LogThread extends Thread {

   private Log log = LogFactory.getLog(this.getClass());

   public LogThread(String id){
      super(id);
   }

   public void run(){
      try{
      NDC.push(this.getName());
      log.info("Entering Thread");
      for(int i = 0; i < 10; i++){
         log.info("In loop:  " + i);
         SimpleMessage.printMessage();
         sleep((long)(3000*Math.random()));
      }
      NDC.pop();
      }catch(Exception e){
         log.error("Exception caught");
      }
   }
}

In this example, 10 threads are started, each printing 10 messages at random intervals between 0 and 3 seconds. The NDC.push(this.getName()) call turns on the NDC logging and the NDC.pop( ) call turns it off. It is up to the programmer to uniquely identify each context being logged. In this case, we used the thread name.

To run the sample, load lf5/lf5NDCconfig.xml using the remote logging page. Then start the lf5 remote server using the lf5/run_LF5_socket.bat file. The URL to start the actual test messages is http://localhost/Applications/ SimpleWAR/action/NDCTest.

Figure 4 shows what the output looks like without NDC filtering within LogFactor5.

Figure 4: Output without NDC filtering.

As you can see in the figure, it is hard to determine the timing of each individual thread. However, when the NDC filter is turned on, the output looks like that shown in Figure 5.

Figure 5: Output with the NDC filter turned on.

In this figure, it is easy to follow the exact timing of "LogThread 3". To turn on NDC filtering, simply use the Edit > NDC Sort drop-down menu and type the thread you want to follow.

Conclusion

This AppNote has shown how to integrate the log4j logging tool into the Novell exteNd Application Server. It has also covered how to use companion tools such as LogFactor5 and Nested Diagnostic Context (NDC).

* Originally published in Novell AppNotes

Disclaimer

The origin of this information may be internal or external to Novell. While Novell makes all reasonable efforts to verify this information, Novell does not make explicit or implied claims to its validity.

© Copyright Micro Focus or one of its affiliates