How To Cluster SpringSource tc Server™

I decided to pull together this HowTo while working on a tc Server screencast for Chariot Solutions' booth at SpringOne 2GX. For the unfamiliar, tc Sever is an enterprise version of Apache Tomcat provided by SpringSource™.

There are a number of differences between tc Server and Apache Tomcat, however, at their core they are the same server. As SpringSource puts it, tc Server is "hardened for enterprise use". There is plenty of information about tc Server on SpringSource's website so I won't go into the details here. The main focus of this post is to describe how to get a small cluster (session replication) of tc Servers up and running.

Clustering

tc Server clustering provides session replication and context attribute replication. Session replication is basically copying the HttpSession object to all members of a cluster each time it has changed. Typical web applications use the HttpSession object to temporarily store user specific information like shopping cart items, security credentials, etc. When the user's web session is terminated (e.g. by the user, server crash, etc.) then all the session data is lost, which can cause serious usability and availability problems. The following steps will provide you with a simple (3) node tc Server session replication cluster, which provides a stable platform configured for high availability.

NOTE: If you have a large cluster with many nodes then SpringSource recommends a more advanced cluster configuration that replicates session information to a single backup member.

What You Need

• tc Server installation - I used a server-only platform neutral distribution (tcServer-6.0.20.B-GA-appserver.tar). You can download an eval from SpringSource's website.
• JDK 1.5 or 1.6 - I used Sun JDK 1.6
• Mac, Unix or Linux platform - you can install on Windows however, the following instructions are specific for a Unix platform.

High-level Steps

• Install & Test tc Server
  
• Create tc Server instances
• Configure Clustering
• Build and Deploy a Test Web Application
• Test the cluster

Install & Test tc Server

1. Install a JDK.

NOTE: If you are installing on a 64-bit UNIX platform then you must explicitly set the JAVA_HOME environment variable. This also helps later when creating a tc Server instance using the tcserver-instance.sh script.

2. Unpack the tc Server distribution in a temp directory using the following command.

$ tar -xf tcServer-6.0.20.B-GA-appserver.tar

3. Change to the temp directory containing the unpacked tc Server installation files.

4. Run the installer program using the following command.

$ ./install.sh



a. Enter a 1 and hit return.



b. Specify the full path to the location where you want to install tc Server and hit return.

NOTE: The installer will not create a directory for you so it must exist.



c. Enter a 2 and hit return to exit the installer.

tc Server is now installed with an ASF layout of a tc Server instance. The ASF layout is the Apache Tomcat directory structure layout. There is also a tc Server layout that we will use when clustering the servers. There are some differences between the layouts.

5. Test your installation. First change directory to the bin directory of the default tc Server instance INSTALL_DIR/tcServer-6.0/tomcat-6.0.20.B/bin.

6. Start the tc Server instance.

$ ./tcserver-ctl.sh start

OR

$ ./tcserver-ctl.sh run

The 'start' command starts tc Server as a daemon process. The 'run' command starts tc Server as a foreground process.

7. Open a web browser and point to http://localhost:8080/ and you should see the follow home page for the server.



You can now shutdown the server. If you started the server using the 'run' command then you can use ctrl+c to shutdown. If you started the server as a daemon process then use the following command to shutdown:

$ ./tcserver-ctl.sh stop

Congratulations! You've successfully installed and tested tc Server in a few short steps.

Creating tc Server instances

I've decided to use the tc Server layout for the instances so that means we need to create them with a tc Server script.

1. Change directory to the INSTALL_DIR/tcServer-6.0

2. Run the tcserver-instance.sh with the required -s (servername) and -v (tc Server version) options.

$ ./tcserver-instance.sh -s tcserver1 -v 6.0.20.B



3. Repeat step #2 two more times for servername tcserver2 & tcserver3. You should have a directory structure similar to this:



4. Test each instance (tcserver1, tcserver2 & tcserver 3) one at a time to confirm that they start up ok. Since all 3 instances are using the same ports right now you have to test one at a time.

Configure Clustering

1. For each tc Server instance (tcserver1, tcserver2 & tcserver3) add the following <Cluster> child element of the <Engine> element in the INSTALL_DIR/tcServer-6.0/tcserver1/conf/server.xml file.

<?xml version='1.0' encoding='utf-8'?>
<Server port="-1" shutdown="SHUTDOWN">
...
<Service name="Catalina">
...
<Engine name="Catalina" defaultHost="localhost">
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
...
</Engine> 
</Service>
</Server> 

2. Change the TCP/IP listen ports on each tc Server instance to be unique to avoid port conflicts. This is done by editing the INSTALL_DIR/tcServer-6.0/tcserver1/conf/catalina.properties file and changing the following ports to be unique in each server instance. I just incremented the http, jmx and ajp ports for tcserver2 & tcserver3.

http.port=8080
jmx.port=6969
ajp.port=8009

3. Start all instances and confirm that there are no port conflicts. You'll notice that clustering adds a little overhead to the startup time.

Build and Deploy a Test Web Application

Well we could just stop right here because you now have a running cluster however, I like to test it out just to be sure. In order to do this you need to create a basic web application that can read & write to the HttpSession object. This application will then be deployed to each tc Server instance to test out the cluster. The following are steps using Maven to create a simple web application with a single JSP.

1. Install Maven

2. Create a directory for the web application.

3. Change directory to the web application directory and run this Maven command:

mvn archetype:create -DgroupId=com.test -DartifactId=ct-web -DarchetypeArtifactId=maven-archetype-webapp

This command will build a basic web application structure.

4. Edit the PROJECT_DIR/ct-web/src/main/webapp/index.jsp file. Replace the index.jsp contents with the following:

<html>
<head>
<title> TC Server - Cluster Test Web Application</title>
</head>

<body>
<h2> TC Server - Cluster Test Web Application</h2>
<table width="100%" border="1" cellspacing="0" cellpadding="0">
<tr>
<td width="35%" style="font-weight: bold;"> WEB-INF Real Path - application.getRealPath("WEB-INF"):</td>
<td width="65%"><%=application.getRealPath("WEB-INF")%></td>
</tr>
<tr bgcolor="#CCCCCC">
<td style="font-weight: bold;"> Session ID - session.getId() :</td>
<td><%=session.getId()%></td>
</tr>
<tr>
<td style="font-weight: bold;"> App Server Info - application.getServerInfo():</td>
<td><%=application.getServerInfo()%></td>
</tr>
</table>
</body>
</html>

This code will display the physical path to the WEB-INF directory from the ServletContext, HttpSession ID and server information.

5. Edit the PROJECT_DIR/ct-web/src/main/webapp/WEB-INF/web.xml file. Add in the <distributable /> child element of the <web-app> element.

<web-app>
<display-name>Archetype Created Web Application</display-name>

<distributable />

</web-app>

NOTE: This configuration will indicate that the application is suitable for a distributed environment. Another approach would be to edit the INSTALL_DIR/tcServer-6.0/tcserver1/conf/context.xml and add in <Context distributable="true"> for one or all web applications to be distributed.

6. Run the following command to build and package the web application.

$ mvn package



7. You should now have a ct-web/target/ct-web.war file. Copy the WAR file to INSTALL_DIR/tcServer-6.0/tcserver1/webapps directory for each instance.

8. Start all of the server instances.

Test the cluster

To test out the cluster simply open a web browser and create tabs (or windows) for each of the following addresses:

http://localhost:8080/ct-web/



http://localhost:8081/ct-web/



http://localhost:8082/ct-web/



The WEB-INF real path should point to the correct physical directory of each server instance. The session ID should be the same among all three instances indicating that the cluster is working. If it was not working then you would see different session IDs for each tab or window. To further prove this out simply:

1. Shutdown all of the tc Servers.

2. Remove the ../webapps/ct-web.war file and ../webapps/ct-war directory from each tc Server instance.

3. Edit the PROJECT_DIR/ct-web/src/main/webapp/WEB-INF/web.xml in the ct-web application and remove or comment out the <distributable /> element.

4. Run $ mvn clean package to clean out the old build and package up a new ct-web.war file.

5. Start all the tc Server instances.

6. Copy the new ct-web.war file out to all the tc Server instances.

Now the ct-web application is not configured for clustering and when you re-test you should see different session IDs for each server instance.

You now have proved out your fully functioning (3) node tc Server cluster that is replicating session data. The next step would be to put a load balancer in front of the cluster to make it production ready. Again, if you are dealing with a large number of tcServer nodes then a more advanced cluster configuration maybe necessary.