Logging in Tomcat using Log4j Core

Logging configuration in Java is complex, mostly due to the presence of multiple logging APIs and logging backends, that need to be configured separately.

On a Tomcat server we might find:

In this guide, we will present a way to bring all these logging APIs together and direct them to use a single instance of Log4j Core.

We assume that all the web applications running on Tomcat are wired to forward all other logging APIs to Log4j API. If that is not the case, see Installing bridges in Apache Log4j documentation.

Installation

To install all Copernik.eu plugins for Tomcat and Log4j:

Click here for a definition of CATALINA_BASE and CATALINA_HOME

A Tomcat installation can be split in two separate folders:

CATALINA_HOME

This is the folder that contains the code of the server and the default configuration of Tomcat instances.

CATALINA_BASE

This is the folder that contains the runtime configuration and working directories of a specific Tomcat instance.

The typical location of these folders varies between OSes:

Debian

On Debian and derived GNU/Linux distributions CATALINA_BASE is located in the /var/lib folder (e.g. /var/lib/tomcat10). CATALINA_HOME on the other hand is located in /usr/share (e.g. /usr/share/tomcat10).

Windows

If you installed Tomcat from the MSI package, both CATALINA_BASE and CATALINA_HOME point to the same subfolder of C:\Program Files\Apache Software Foundation, e.g. C:\Program Files\Apache Software Foundation/Tomcat 10.1.

  • Download the copernik-log4j-tomcat-3.0.0-beta1-overlay.zip distribution archive available on the GitHub Release page.

  • Unzip the contents of the overlay folder of the archive in the $CATALINA_BASE folder of you Tomcat installation.

  • Customize if necessary the $CATALINA_BASE/bin/setenv-log4j.sh file and move it to $CATALINA_BASE/bin/setenv.sh.

  • Customize if necessary the $CATALINA_BASE/conf/context-log4j.xml file and move it to $CATALINA_BASE/conf/context.xml.

Configuration

Depending on the flexibility of configuration you want on your Tomcat server:

Separate logging contexts

To use multiple logging contexts, you need to modify the conf/logging/log4j2.component.properties file in $CATALINA_BASE and enable one of the Tomcat context selectors:

Example log4j2.component.properties file
##
# Enable Tomcat context selector
log4j2.contextSelector = eu.copernik.log4j.tomcat.TomcatContextSelector

# To enable asynchronous loggers:
# 1. Add `com.lmax:disruptor` to `$CATALINA_BASE/bin/logging`.
# 2. Replace the `log4j2.contextSelector` property with:
#
# log4j2.contextSelector = eu.copernik.log4j.tomcat.TomcatAsyncContextSelector

After the context selector has been enabled, you will be able to add configuration files in the following locations of $CATALINA_BASE:

conf/logging/log4j2-tomcat.<extension>

This configuration file, if present, will be used by the Tomcat server itself.

conf/logging/log4j2/<engine>/<host>/<context>.<extension>

This configuration file will be used by the application with base name <context> in the host <host> of Tomcat’s <engine> engine.

conf/logging/log4j2.xml

This configuration file will be used as a fallback if a more specific configuration file does not exist.

Using multiple logger contexts will always correctly separate the logs issued by the web applications themselves.

The logs issued by shared $CATALINA_BASE/lib libraries might, however, end up in the global Tomcat logger context if the libraries were not written with log separation in mind. As far as we know, Tomcat’s libraries behave correctly when used with multiple logger contexts.

See Log Separation on Apache Log4j site for more information on the topic.

Separate logging contexts examples

Multiple files

To separate the logs of each application into their own log file, create a conf/logging/log4j2.xml file as the example below:

  • XML

  • JSON

  • YAML

Example log4j2.xml file
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="https://logging.apache.org/xml/ns"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="
                   https://logging.apache.org/xml/ns
                   https://logging.apache.org/xml/ns/log4j-config-2.xsd">
  <Properties>
    <Property name="logs.dir" value="${sys:catalina.base}/logs"/>
    <Property name="context.name" value="catalina"/>
  </Properties>
  <Appenders>
    <RollingFile name="FILE"
                 filePattern="${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log"> (1)
      <JsonTemplateLayout/>
      <DirectWriteRolloverStrategy>
        <Delete basePath="${logs.dir}">
          <IfFileName regex="${tomcat:context.name}\.\d{4}-\d{2}-\d{2}\.log"/>
          <IfLastModified age="P90D"/>
        </Delete>
      </DirectWriteRolloverStrategy>
      <TimeBasedTriggeringPolicy/>
    </RollingFile>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="FILE"/>
    </Root>
  </Loggers>
</Configuration>
Example log4j2.json file
{
  "Configuration": {
    "Properties": {
      "Property": [
        {
          "name": "logs.dir",
          "value": "${sys:catalina.base}/logs"
        },
        {
          "name": "context.name",
          "value": "catalina"
        }
      ]
    },
    "Appenders": {
      "RollingFile": {
        "name": "FILE",
        "filePattern": "${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log",
        "JsonTemplateLayout": {},
        "DirectWriteRolloverStrategy": {
          "Delete": {
            "basePath": "${logs.dir}",
            "IfFileName": {
              "regex": "${tomcat:context.name}\\.\\d{4}-\\d{2}-\\d{2}\\.log"
            },
            "IfLastModified": {
              "age": "P90D"
            }
          }
        },
        "TimeBasedTriggeringPolicy": {}
      }
    },
    "Loggers": {
      "Root": {
        "level": "INFO",
        "AppenderRef": {
          "ref": "FILE"
        }
      }
    }
  }
}
Example log4j2.yaml file
Configuration:
  Properties:
    Property:
      - name: "logs.dir"
        value: "${sys:catalina.base}/logs"
      - name: "context.name"
        value: "catalina"
  Appenders:
    RollingFile:
      name: "FILE"
      filePattern: "${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log"
      JsonTemplateLayout: {}
      DirectWriteRolloverStrategy:
        Delete:
          basePath: "${logs.dir}"
          IfFileName:
            regex: "${tomcat:context.name}\.\d{4}-\d{2}-\d{2}\.log"
          IfLastModified:
            age: "P90D"
      TimeBasedTriggeringPolicy: { }
  Loggers:
    Root:
      level: "INFO"
      AppenderRef:
        - ref: "FILE"

Single appender

If you prefer to send all logs to the console, make sure to mark the origin of each log event. You can do it with a configuration file like:

  • XML

  • JSON

  • YAML

Example log4j2.xml file
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="https://logging.apache.org/xml/ns"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="
                   https://logging.apache.org/xml/ns
                   https://logging.apache.org/xml/ns/log4j-config-2.xsd">
  <Properties>
    <Property name="context.name" value="catalina"/>
  </Properties>
  <Appenders>
    <Console name="CONSOLE"
             direct="true">
      <JsonTemplateLayout>
        <EventTemplateAdditionalField key="context.name" value="${tomcat:context.name}"/> (1)
      </JsonTemplateLayout>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="CONSOLE"/>
    </Root>
  </Loggers>
</Configuration>
Example log4j2.json file
{
  "Configuration": {
    "Properties": {
      "Property": [
        {
          "name": "context.name",
          "value": "catalina"
        }
      ]
    },
    "Appenders": {
      "Console": {
        "name": "CONSOLE",
        "direct": true,
        "JsonTemplateLayout": {
          "EventTemplateAdditionalField": { (1)
            "key": "context.name",
            "value": "${tomcat:context.name}"
          }
        },
        "TimeBasedTriggeringPolicy": {}
      }
    },
    "Loggers": {
      "Root": {
        "level": "INFO",
        "AppenderRef": {
          "ref": "CONSOLE"
        }
      }
    }
  }
}
Example log4j2.yaml file
Configuration:
  Properties:
    Property:
      - name: "context.name"
        value: "catalina"
  Appenders:
    RollingFile:
      name: "CONSOLE"
      direct: true
      JsonTemplateLayout:
        EventTemplateAdditionalField: (1)
          key: "context.name"
          value: "${tomcat:context.name}"
  Loggers:
    Root:
      level: "INFO"
      AppenderRef:
        - ref: "CONSOLE"
1 Use the Tomcat lookup to store the context name as additional context.name key in the JSON template.

Single logging context

To use a single logging context for the entire JVM, switch Log4j Core from its default context selector to either BasicContextLoggerSelector or BasicAsyncContextLoggerSelector by modifying the conf/logging/log4j2.component.properties file:

Example log4j2.component.properties file
##
# Use the `BasicContextSelector`
log4j2.contextSelector = org.apache.logging.log4j.core.selector.BasicContextSelector

# To enable asynchronous loggers:
# 1. Add `com.lmax:disruptor` to `$CATALINA_BASE/bin/logging`.
# 2. Replace the `log4j2.contextSelector` property with:
#
# log4j2.contextSelector = org.apache.logging.log4j.core.async.BasicAsyncLoggerContextSelector

Single logging contexts examples

Multiple log files

To separate the logs coming from each web application into its own file, you can use the Routing Appender as in the example below:

  • XML

  • JSON

  • YAML

Example log4j2.xml file
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="https://logging.apache.org/xml/ns"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="
                   https://logging.apache.org/xml/ns
                   https://logging.apache.org/xml/ns/log4j-config-2.xsd">
  <Properties>
    <Property name="logs.dir" value="${sys:catalina.base}/logs"/>
    <Property name="context.name" value="catalina"/>
  </Properties>
  <Appenders>
    <Routing name="ROUTING">
      <Routes pattern="$${tomcat:context.name}">
        <Route>
          <RollingFile name="${tomcat:context.name}"
                       filePattern="${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log">
            <JsonTemplateLayout/>
            <DirectWriteRolloverStrategy>
              <Delete basePath="${logs.dir}">
                <IfFileName regex="${tomcat:context.name}\.\d{4}-\d{2}-\d{2}\.log"/>
                <IfLastModified age="P90D"/>
              </Delete>
            </DirectWriteRolloverStrategy>
            <TimeBasedTriggeringPolicy/>
          </RollingFile>
        </Route>
      </Routes>
    </Routing>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="ROUTING"/>
    </Root>
  </Loggers>
</Configuration>
Example log4j2.json file
{
  "Configuration": {
    "Properties": {
      "Property": [
        {
          "name": "logs.dir",
          "value": "${sys:catalina.base}/logs"
        },
        {
          "name": "context.name",
          "value": "catalina"
        }
      ]
    },
    "Appenders": {
      "Routing": {
        "name": "ROUTING",
        "Routes": {
          "pattern": "$${tomcat:context.name}",
          "Route": {
            "RollingFile": {
              "name": "${tomcat:context.name}",
              "filePattern": "${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log",
              "JsonTemplatelayout": {},
              "DirectWriteRolloverStrategy": {
                "Delete": {
                  "basePath": "${logs.dir}",
                  "IfFileName": {
                    "regex": "${tomcat:context.name}\\.\\d{4}-\\d{2}-\\d{2}\\.log"
                  },
                  "IfLastModified": {
                    "age": "P90D"
                  }
                }
              },
              "TimeBasedTriggeringPolicy": {}
            }
          }
        }
      }
    },
    "Loggers": {
      "Root": {
        "level": "INFO",
        "AppenderRef": {
          "ref": "ROUTING"
        }
      }
    }
  }
}
Example log4j2.yaml file
Configuration:
  Properties:
    Property:
      - name: "logs.dir"
        value: "${sys:catalina.base}/logs"
      - name: "one.line.pattern"
        value: "%d{dd-MMM-yyyy HH:mm:ss.SSS} %p [%t] %C.%M %m%n"
      - name: "context.name"
        value: "catalina"
  Appenders:
    Routing:
      name: "ROUTING"
      Routes:
        pattern: "$${tomcat:context.name}"
        Route:
          RollingFile:
            name: "${tomcat:context.name}"
            filePattern: "${logs.dir}/${tomcat:context.name}.%d{yyyy-MM-dd}.log" #
            PatternLayout:
              pattern: "${one.line.pattern}"
            DirectWriteRolloverStrategy:
              Delete:
                basePath: "${logs.dir}"
                IfFileName:
                  regex: "${tomcat:context.name}\.\d{4}-\d{2}-\d{2}\.log"
                IfLastModified:
                  age: "P90D"
            TimeBasedTriggeringPolicy: { }
  Loggers:
    Root:
      level: "INFO"
      AppenderRef:
        - ref: "FILE"

Single appender

If you prefer instead to send all log events to a single appender, make sure to mark each log event with the name of the web application they come from. You can do it by enabling the Tomcat context data provider. You can do it by modifying your conf/logging/log4j2.component.properties to look like:

Example log4j2.component.properties file
##
# Use the `BasicContextSelector`
log4j2.contextSelector = org.apache.logging.log4j.core.selector.BasicContextSelector

# To enable asynchronous loggers:
# 1. Add `com.lmax:disruptor` to `$CATALINA_BASE/bin/logging`.
# 2. Replace the `log4j2.contextSelector` property with:
#
# log4j2.contextSelector = org.apache.logging.log4j.core.async.BasicAsyncLoggerContextSelector


##
# Enable the Tomcat context data provider
log4j2.tomcatContextDataEnabled = true

Once the context data provider is enabled, you can use a configuration file like this:

  • XML

  • JSON

  • YAML

Example log4j2.xml file
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="https://logging.apache.org/xml/ns"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="
                   https://logging.apache.org/xml/ns
                   https://logging.apache.org/xml/ns/log4j-config-2.xsd">
  <Properties>
    <Property name="context.name" value="catalina"/>
  </Properties>
  <Appenders>
    <Console name="CONSOLE"
             direct="true">
      <JsonTemplateLayout/>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="CONSOLE"/>
    </Root>
  </Loggers>
</Configuration>
Example log4j2.json file
{
  "Configuration": {
    "Properties": {
      "Property": [
        {
          "name": "context.name",
          "value": "catalina"
        }
      ]
    },
    "Appenders": {
      "Console": {
        "name": "CONSOLE",
        "direct": true,
        "JsonTemplateLayout": {},
        "TimeBasedTriggeringPolicy": {}
      }
    },
    "Loggers": {
      "Root": {
        "level": "INFO",
        "AppenderRef": {
          "ref": "CONSOLE"
        }
      }
    }
  }
}
Example log4j2.yaml file
Configuration:
  Properties:
    Property:
      - name: "context.name"
        value: "catalina"
  Appenders:
    RollingFile:
      name: "CONSOLE"
      direct: true
      JsonTemplateLayout: { }
  Loggers:
    Root:
      level: "INFO"
      AppenderRef:
        - ref: "CONSOLE"