Putting Tomcat into Production

Installing and configuring Tomcat is fairly straightforward if you follow the documentation that comes with it. But putting it into production requires more thought if you want to make sure you have restful nights and more productive days.

This guide is intended to help systems administrators and Java developers who are responsible for applications running on Tomcat in production to make sure their serves are stable, fast, and easy to manage.

Advanced Tomcat Setup Notes

It's easy to just unzip the Tomcat installation files, drop your webapp into place, and start up the server. But although that works fine for a quick start, once you get into production use, or more complex development scenarios, there are some basic things you can do to make your life easier.

Splitting your Tomcat installation for more power

The most basic setup of Tomcat involves unzipping the distribution files somewhere on your system, dropping your application into the webapps directory, and firing the server up. This is a fine quick start to run a single app on your own machine, but Tomcat offers a much more powerful way to organize production servers and complex developer setups.

If you read down a ways in the RUNNING.txt file in the tomcat installation bundle, you’ll find a section called “Advanced Configuration - Multiple Tomcat Instances”. This describes how to split your Tomcat installation into two directories, the $CATALINA_HOME directory for the Tomcat installation files, and the $CATALINA_BASE directory with the runtime files. Setting these two environment variables when you start up Tomcat controls where the server will look for its files.

The basic idea behind this is that to allow multiple Tomcat server instances to run on the same machine, using a single copy of the Tomcat installation files. There is a single $CATALINA_HOME directory with the installation files, and each server instance has its own $CATALINA_BASE directory, with its own set of webapps, configuration files, libraries, and logs. When you upgrade to a newer version of Tomcat, you simply install the files into a new directory and change the $CATALINA_HOME variable to point there when you restart each of your Tomcat servers. You don’t need to touch the server instance files in $CATALINA_BASE.

But this has additional advantages beyond multiple server instances. For one thing, you can have multiple versions of Tomcat installed on the same machine, and then easily switch which version is being used by a particular server instance by changing the $CATALINA_HOME variable for the instance. This is helpful in development, because you can test your application on different versions of Tomcat to ensure compatibility. It helps in production when different applications need different versions of Tomcat, and also for upgrades. When you upgrade to a newer version of Tomcat, you can switch over applications one by one, and if there is a problem, easily switch back.

Accounts and ownership

This mainly pertains to Unix and Linux systems. One of the cardinal rules is never run Tomcat as root, but I'm sorry to say I've seen this in live production systems.

Why not?

We'll walk you through setting your system up so you can run as a non-root user.

Potential separate roles:

  • Installation file owner
  • Server runtime user
  • Webapps user
  • Server administrator user

The installation files owner owns the CATALINA_HOME directory. It is often root, which is fine since no processes need to run as this user. The other user accounts shouldn't be able to write to these files.

The server runtime user is who the Tomcat process runs as. It needs to be able to read the installation files, the webapp files and everything else under CATALINA_BASE. It also needs to be able to write to the logs, temp, and work directories. It also needs to write the tomcat-users.xml file (look into whether this can be configured to live somewhere else).

The webapps administration user is an optional extra, it allows some users to deploy applications without being able to modify server configuration. The account should be able to write to the webapps directory, but shouldn't need much else.

The server administrator account is for people who need to be able to start and stop the application server, and review the logs. In most cases this can be the same as the webapps administrator, as this will be a developer who deploys the application, restarts the server, and checks the logs to debug any issues.

Tomcat Startup Scripts - Introduction

Making all this work needs control scripts to start and stop your server. The scripts that come with Tomcat are fine at a rough level. You set up a couple of environment variables and away you go. But they have some drawbacks.

The benefits of having your own scripts that call the catalina scripts are:

  • Make sure the server runs as the appropriate user
  • Easily have multiple different configurations, with different CATALINA_BASE directories, with different applications and/or configurations. You can also have different versions of the JVM and Tomcat with the flip of a switch. Useful for servers with multiple applications, and also for development setups where you need to be able to switch the application quickly.
  • Avoid starting Tomcat when a previous instance is still running
  • Manage common tasks, such as rotating logs, and optionally cleaning out temp files and work directories (although be sure you know what you're doing in this case

What we'll do here is outline a decent script to do all of this. The basic version of the script will be intended for fairly straightforward setups, a single server instance, where you can change the options discussed by changing hard-coded variables in the script.

A more advanced version will cater for running multiple server instances on the same machine, with potentially different versions of Java and Tomcat. This will use configuration files to work out which options to use when running.

Ensuring the user

Here's a trick I use to ensure the server runs as the correct user. If the script is run as root, it simply uses the su command to re-run the script as the appropriate user. If it runs as a non-root user other than the intended one, it will try calling sudo command to re-run the script as root. If the user who runs the script has appropriate sudo permissions the script will be run as root, and then again as the appropriate user.

Insert code example here

...

Tomcat Log Management

One of the drawbacks of the default Tomcat setup is that the logfiles aren't written to the sensible location on Unix and Linux systems. This isn't hard to correct.

Here's how. (Examples of configuration settings in server.xml and so on go here).

While you're at it, you should consider tweaking what logs are written.

Different types of logs you can have.

Recommendations for paring it down for production.

Tweaks to make for debugging.

What you may want on development.

Tweaking the log4j configuration to control what you log.

Retention policy. Decide how long you want to keep your logs. Important logs, e.g. where you have applications write data that need to be collated and reported on for business reasons, should be archived.

(example of logrotate configuration file for tomcat logs)

The catalina.out log. This is a bit of a pain, because unlike the other logs you can't have it rotate automatically. The only way to rotate it is to rename the file and restart the server.

(Research this, is there a better way? Could get very fancy with a named pipe or something)

Include catalina.out rotation in the start/stop script.

Tightening the Default Tomcat Configuration

Basic things to do to tighten up the Tomcat configuration. Out of the box it's not really set up for production use.

Remove unnecessary webapps

Strip down the server.xml

Take out connectors you don't need. Get rid of comments to make it easier to follow. Strip the examples code. What user account functionality do you need for your apps, and for the server admin/management tools if you use them?

Some basic tuning

Reloadability and such. Logging. Connector tweaking.

Configuring the Tomcat manager webapp

I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. To do this, make a file called manager.xml file in the webapps directory of your Tomcat instance. One I like to use is this:

    <Context path="/manager"
        docBase="/usr/local/tomcat/server/webapps/manager"
        debug="0"
        privileged="true">

    <ResourceLink name="users"
            global="UserDatabase"
            type="org.apache.catalina.UserDatabase"/>

    <Valve className="org.apache.catalina.valves.RemoteAddrValve"
            allow="127.0.0.1,192.168.100.100"/>

</Context>

The key bit is the docBase attribute, which needs to point to the webapp in the Tomcat installation directory. I add a RemoteAddrValve to keep evil people from trying to break into the manager.

You'll also need to add a user account with permission to use the manager. Put a file called tomcat-users.xml into the conf directory of the Tomcat instance, which should have something like the following:

<tomcat-users>
  <role rolename="manager"/>
  <user username="admin" password="hard2Guess" roles="manager"/>
</tomcat-users>

Finally, your server.xml file needs to have a UserDatabase configured. This is in the example configuration files from the Tomcat installation.

Running multiple Tomcat instances on one server

Here's a brief step by step guide to running more than one instance of Tomcat on a single machine.

Step 1: Install the Tomcat files

Download Tomcat 4.1 or 5.5, and unzip it into an appropriate directory. I usually put it in /usr/local, so it ends up in a directory called /usr/local/apache-tomcat-5.5.17 (5.5.17 being the current version as of this writing), and make a symlink named /usr/local/tomcat to that directory. When later versions come out, I can unzip them and relink, leaving the older version in case things don't work out (which rarely if ever happens, but I'm paranoid).

Step 2: Make directories for each instance

For each instance of Tomcat you're going to run, you'll need a directory that will be CATALINA_HOME. For example, you might make them /var/tomcat/serverA and /var/tomcat/serverB.

In each of these directories you need the following subdirectories: conf, logs, temp, webapps, and work.

Put a server.xml and web.xml file in the conf directory. You can get these from the conf directory of the directory where you put the tomcat installation files, although of course you should tighten up your server.xml a bit.

The webapps directory is where you'll put the web applications you want to run on the particular instance of Tomcat.

I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. See my instructions for configuring the Tomcat manager webapp.

Step 3: Configure the ports and/or addresses for each instance

Tomcat listens to at least two network ports, one for the shutdown command, and one or more for accepting requests. Two instances of Tomcat can't listen to the same port number on the same IP address, so you will need to edit your server.xml files to change the ports they listen to.

The first port to look at is the shutdown port. This is used by the command line shutdown script (actually, but the Java code it runs) to tell the Tomcat instance to shut itself down. This port is defined at the top of the server.xml file for the instance.

<Server port="8001" shutdown="_SHUTDOWN_COMMAND_" debug="0">

Make sure each instance uses a different port value. The port value will normally need to be higher than 1024, and shouldn't conflict with any other network service running on the same system. The shutdown string is the value that is sent to shut the server down. Note that Tomcat won't accept shutdown commands that come from other machines.

Unlike the other ports Tomcat listens to, the shutdown port can't be configured to listen to its port on a different IP address. It always listens on 127.0.0.1.

The other ports Tomcat listens to are configured with the <Connector> elements, for instance the HTTP or JK listeners. The port attribute configures which port to listen to. Setting this to a different value on the different Tomcat instances on a machine will avoid conflict.

Of course, you'll need to configure whatever connects to that Connector to use the different port. If a web server is used as the front end using mod_jk, mod_proxy, or the like, then this is simple enough - change your web server's configuration.

In some cases you may not want to do this, for instance you may not want to use a port other than 8080 for HTTP connectors. If you want all of your Tomcat intances to use the same port number, you'll need to use different IP addresses. The server system must be configured with multiple IP addresses, and the address attribute of the <Connector> element for each Tomcat instance will be set to the appropriate IP address.

Step 4: Startup

Startup scripts are a whole other topic, but here's the brief rundown. The main different from running a single Tomcat instance is you need to set CATALINA_BASE to the directory you set up for the particular instance you want to start (or stop). Here's a typical startup routine:

JAVA_HOME=/usr/java
JAVA_OPTS="-Xmx800m -Xms800m"
CATALINA_HOME=/usr/local/tomcat
CATALINA_BASE=/var/tomcat/serverA
export JAVA_HOME JAVA_OPTS CATALINA_HOME CATALINA_BASE

$CATALINA_HOME/bin/catalina.sh start