Splitting your Tomcat installation for more power

The most basic setup of Tomcat involves unzipping the distribution files somewhere on your system, dropping your application into the webapps directory, and firing the server up. This is a fine quick start to run a single app on your own machine, but Tomcat offers a much more powerful way to organize production servers and complex developer setups.

If you read down a ways in the RUNNING.txt file in the tomcat installation bundle, you’ll find a section called “Advanced Configuration - Multiple Tomcat Instances”. This describes how to split your Tomcat installation into two directories, the $CATALINA_HOME directory for the Tomcat installation files, and the $CATALINA_BASE directory with the runtime files. Setting these two environment variables when you start up Tomcat controls where the server will look for its files.

The basic idea behind this is that to allow multiple Tomcat server instances to run on the same machine, using a single copy of the Tomcat installation files. There is a single $CATALINA_HOME directory with the installation files, and each server instance has its own $CATALINA_BASE directory, with its own set of webapps, configuration files, libraries, and logs. When you upgrade to a newer version of Tomcat, you simply install the files into a new directory and change the $CATALINA_HOME variable to point there when you restart each of your Tomcat servers. You don’t need to touch the server instance files in $CATALINA_BASE.

But this has additional advantages beyond multiple server instances. For one thing, you can have multiple versions of Tomcat installed on the same machine, and then easily switch which version is being used by a particular server instance by changing the $CATALINA_HOME variable for the instance. This is helpful in development, because you can test your application on different versions of Tomcat to ensure compatibility. It helps in production when different applications need different versions of Tomcat, and also for upgrades. When you upgrade to a newer version of Tomcat, you can switch over applications one by one, and if there is a problem, easily switch back.

Configuring the Tomcat manager webapp

I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. To do this, make a file called manager.xml file in the webapps directory of your Tomcat instance. One I like to use is this:

    <Context path="/manager"
        docBase="/usr/local/tomcat/server/webapps/manager"
        debug="0"
        privileged="true">

    <ResourceLink name="users"
            global="UserDatabase"
            type="org.apache.catalina.UserDatabase"/>

    <Valve className="org.apache.catalina.valves.RemoteAddrValve"
            allow="127.0.0.1,192.168.100.100"/>

</Context>

The key bit is the docBase attribute, which needs to point to the webapp in the Tomcat installation directory. I add a RemoteAddrValve to keep evil people from trying to break into the manager.

You'll also need to add a user account with permission to use the manager. Put a file called tomcat-users.xml into the conf directory of the Tomcat instance, which should have something like the following:

<tomcat-users>
  <role rolename="manager"/>
  <user username="admin" password="hard2Guess" roles="manager"/>
</tomcat-users>

Finally, your server.xml file needs to have a UserDatabase configured. This is in the example configuration files from the Tomcat installation.

Running multiple Tomcat instances on one server

Here's a brief step by step guide to running more than one instance of Tomcat on a single machine.

Step 1: Install the Tomcat files

Download Tomcat 4.1 or 5.5, and unzip it into an appropriate directory. I usually put it in /usr/local, so it ends up in a directory called /usr/local/apache-tomcat-5.5.17 (5.5.17 being the current version as of this writing), and make a symlink named /usr/local/tomcat to that directory. When later versions come out, I can unzip them and relink, leaving the older version in case things don't work out (which rarely if ever happens, but I'm paranoid).

Step 2: Make directories for each instance

For each instance of Tomcat you're going to run, you'll need a directory that will be CATALINA_HOME. For example, you might make them /var/tomcat/serverA and /var/tomcat/serverB.

In each of these directories you need the following subdirectories: conf, logs, temp, webapps, and work.

Put a server.xml and web.xml file in the conf directory. You can get these from the conf directory of the directory where you put the tomcat installation files, although of course you should tighten up your server.xml a bit.

The webapps directory is where you'll put the web applications you want to run on the particular instance of Tomcat.

I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. See my instructions for configuring the Tomcat manager webapp.

Step 3: Configure the ports and/or addresses for each instance

Tomcat listens to at least two network ports, one for the shutdown command, and one or more for accepting requests. Two instances of Tomcat can't listen to the same port number on the same IP address, so you will need to edit your server.xml files to change the ports they listen to.

The first port to look at is the shutdown port. This is used by the command line shutdown script (actually, but the Java code it runs) to tell the Tomcat instance to shut itself down. This port is defined at the top of the server.xml file for the instance.

<Server port="8001" shutdown="_SHUTDOWN_COMMAND_" debug="0">

Make sure each instance uses a different port value. The port value will normally need to be higher than 1024, and shouldn't conflict with any other network service running on the same system. The shutdown string is the value that is sent to shut the server down. Note that Tomcat won't accept shutdown commands that come from other machines.

Unlike the other ports Tomcat listens to, the shutdown port can't be configured to listen to its port on a different IP address. It always listens on 127.0.0.1.

The other ports Tomcat listens to are configured with the <Connector> elements, for instance the HTTP or JK listeners. The port attribute configures which port to listen to. Setting this to a different value on the different Tomcat instances on a machine will avoid conflict.

Of course, you'll need to configure whatever connects to that Connector to use the different port. If a web server is used as the front end using mod_jk, mod_proxy, or the like, then this is simple enough - change your web server's configuration.

In some cases you may not want to do this, for instance you may not want to use a port other than 8080 for HTTP connectors. If you want all of your Tomcat intances to use the same port number, you'll need to use different IP addresses. The server system must be configured with multiple IP addresses, and the address attribute of the <Connector> element for each Tomcat instance will be set to the appropriate IP address.

Step 4: Startup

Startup scripts are a whole other topic, but here's the brief rundown. The main different from running a single Tomcat instance is you need to set CATALINA_BASE to the directory you set up for the particular instance you want to start (or stop). Here's a typical startup routine:

JAVA_HOME=/usr/java
JAVA_OPTS="-Xmx800m -Xms800m"
CATALINA_HOME=/usr/local/tomcat
CATALINA_BASE=/var/tomcat/serverA
export JAVA_HOME JAVA_OPTS CATALINA_HOME CATALINA_BASE

$CATALINA_HOME/bin/catalina.sh start