HA Singleton with JBoss 7 Cluster

Document  Version 1.0

   Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
JBoss 7  HA  Service, HA Singleton, JBoss 7 Cluster, load migration, load distribution, HA Service deployment

Start 2 standalone JBosss 7 server in clustered mode  [Draft]

Download JBoss AS 7.1.1.Final and extract the archive to a tmp location, lets say " JBoss_AS_7_1_1_Final"

Create 2 new folders:

jboss7_node1

jboss7_node2

Copy all folders and files under "JBoss_AS_7_1_1_Final" into jboss7_node1.

Copy all folders and files under "JBoss_AS_7_1_1_Final" into jboss7_node2.

Now we have 2 JBoss nodes ready to be started.

Start "jboss7_node1" in clusted mode using command:

./standalone.sh -Djboss.node.name=node1 --server-config=standalone-ha.xml

The node1 is started correctly in clusted mode when following line is shown in console:

[org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-3) Listening to proxy advertisements on 224.0.1.105:23364

....

When you are familiar with older JBoss (4x, 5x version), you may expect to see log info about cluster information or  node information. But JBoss 7 will not show such info yet.

Start "jboss7_node2" in clusted mode using command:

./standalone.sh -Djboss.node.name=node2 --server-config=standalone-ha.xml -Djboss.socket.binding.port-offset=100

The node2 is started correctly in clustered mode when following line is shown in console:

[org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (MSC service thread 1-3) Listening to proxy advertisements on 224.0.1.105:23364

....

 Here we still can not see any log info to clusters and nodes.

You could just write a samll "Hello World" web applicaiton and try to deploy it in cluster.
I will just take the "TestWebSec20" web application, add "<distributable/>" to the web.xml file and try put the "TestWebSec20.war" into the folder "deployments" of jboss7_node1.

The console of node1 shows:


21:19:36,185 INFO  [org.jboss.as.clustering.impl.CoreGroupCommunicationService.web] (MSC service thread 1-1) JBAS010206: Number of cluster members: 1
....
21:19:37,378 INFO  [org.jboss.as.server] (DeploymentScanner-threads - 2) JBAS018559: Deployed "TestWebSec20.war"

Now cluster info is shown, but where is the node2?

Now try to deploy the "TestWebSec20.wa" in node2, put the war file in the folder "deployments" of jboss7_node2.
The console of node2 shows:


21:26:35,227 INFO  [org.jboss.as.clustering.impl.CoreGroupCommunicationService.web] (MSC service thread 1-3) JBAS010206: Number of cluster members: 2
....
21:26:37,207 INFO  [org.jboss.as.server] (DeploymentScanner-threads - 1) JBAS018559: Deployed "TestWebSec20.war"

Lets take look again the console of node1:


21:26:31,272 INFO  [org.jboss.as.clustering.impl.CoreGroupCommunicationService.lifecycle.web] (Incoming-1,null) JBAS010247: New cluster view for partition web (id: 1, delta: 1, merge: false) : 

[node1/web, node2/web]

21:26:31,326 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,null) ISPN000094: Received new cluster view: [node1/web|1] [node1/web, node2/web]

Now lets shutdown node1, and console of node2 shows:

21:31:10,903 INFO  [org.jboss.as.clustering.impl.CoreGroupCommunicationService.lifecycle.web] (Incoming-6,null) JBAS010247: New cluster view for partition web (id: 2, delta: -1, merge: false) : [node2/web]

21:31:10,961 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-6,null) ISPN000094: Received new cluster view: [node2/web|2] [node2/web]

HA Singleton, Cluster Wide Singleton as MBean in JBoss 5, part 3/3

Document  Version 1.0

  Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
JBoss HA  Service, HA Singleton, JBoss Cluster, load migration, load distribution, HA Service deployment

In "HA Singleton, Cluster Wide Singleton as MBean in JBoss 5", part1-2, we have successfully configured a 2-nodes JBoss cluster, and have reated a "TestHASingleton" MBean. This bean is ready to be deployed as a JBoss HA singleton. If you have followed the part2, you should now have a "TestHASingleton.sar" file. In this part 3 of the serial, we will deploy the  "TestHASingleton" Mbean in JBoss cluster. We will also do some experiments to make sure it is really a HA singleton, i.e. it runs ONLY once in the cluster.

Deployment of "TestHASingleton" MBean

Put the "TestHASingleton.sar in "JBOSS_HOME/server/node1/farm/".

You will see following information in the console of node1 :

10:28:09,752 INFO  [STDOUT] ### Starting JbHaSingletonSvcSample Singleton Service..

From the above text we can see that the "startSingletonService" method of "JbHaSingletonSvcSample" is called, i.e. the singleton is started on node1.

We check the console output of node2, nothing about the singleton is shown. This is correct.

Shutdown node1

Now, we will shut down node1. Now we will see following info in the console of node2:

10:40:05,383 INFO  [STDOUT] ### Starting JbHaSingletonSvcSample Singleton Service..

Find out service on which node will the singleton be active?

step1. shutdown node1 and node2

step2.  remove the "TestHASingleton.sar" from farms of both nodes

step3.  start node1

step4.  start node2

step5. deploy "TestHASingleton.sar" in farm of node2.

The console of node2 shows:

10:48:34,880 INFO  [STDOUT] ### Starting JbHaSingletonSvcSample Singleton Service..

10:48:37,657 INFO  [STDOUT] ### Stopping JbHaSingletonSvcSample Singleton Service..

The console of node1 shows:

10:48:37,658 INFO  [STDOUT] ### Starting JbHaSingletonSvcSample Singleton Service..

This is because, via default, a HA singleton will only active on the "master" node of a cluster (JBoss mainains a sorted list of all active nodes, sorted by the time it joins the cluster. The first node in the list will be choosen as master... ). A HA singleton is active only on the MASTER node.

We could also customize the HA singleton behavior when deciding service on which node should be made active. In such case customized "HA selection policy" is needed... 

HASingleton, Cluster Wide Singleton as MBean in JBoss 5, part2/3   part1/3

HA Singleton, Cluster Wide Singleton as MBean in JBoss 5, part 2/3

Document  Version 1.0

  Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
JBoss HA  Service, HA Singleton, JBoss Cluster, load migration, load distribution, HA Service deployment

Write and deploy an JBoss Mbean

We will write a JBoss MBean called "JbHaSingletonSvcSample"

Create a simple Java project "TestHASingleton" in Eclipse. 

An MBean needs an interface, an implementation class, and a "jbosss-service.xml" file for MBean description/deployment.

The interface: 

package test.ha;

import org.jboss.system.ServiceMBean;

public interface JbHaSingletonSvcSampleMBean extends ServiceMBean{

}

The implementation class:

package test.ha;
import org.jboss.system.ServiceMBeanSupport;
/**
 * The service itself shoul not be written as singleton.
 * @author ws
 *
 */
public class JbHaSingletonSvcSample extends ServiceMBeanSupport implements
JbHaSingletonSvcSampleMBean {
// The lifecycle
public void startSingletonService() throws Exception {
System.out.println("### Starting JbHaSingletonSvcSample Singleton Service..");
}
public void stopSingletonService() throws Exception {
System.out.println("### Stopping JbHaSingletonSvcSample Singleton Service..");
}
}

Create a "META-INF" folder directly under project root:

TestHASingleton/

                            src/

                            META-INF/

Create a "jboss-service.xml" file in "META-INF" folder with following content:

<?xml version="1.0" encoding="UTF-8"?>

<server>

  <mbean code="test.ha.JbHaSingletonSvcSample" name="myexample:service=testHaSample"/>

    <mbean code="org.jboss.ha.singleton.HASingletonController" name="myexample:service=SingletonServiceControllerA">

        <attribute name="HAPartition"><inject bean="HAPartition" /></attribute>

         <attribute name="Target"><inject bean="myexample:service=testHaSample" /></attribute>

        <attribute name="TargetStartMethod">startSingletonService</attribute>

        <attribute name="TargetStopMethod">stopSingletonService</attribute>

    </mbean>

</server>

MBean will be deployed as ".sar" archive. An ".sar" is nothing else that a ".jar" file. To make an ".sar" file, we just need to export the project binaries as "jar" file with Eclipse, i.e. "TestHASingleton.jar",

and then rename it to "TestHASingleton.sar".

HASingleton, Cluster Wide Singleton as MBean in JBoss 5, part3/3   part1/3

HA Singleton, Cluster Wide Singleton as MBean in JBoss 5, part 1/3

Document  Version 1.0

   Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
JBoss HA  Service, HA Singleton, JBoss Cluster, load migration, load distribution, HA Service deployment

What is HA Singleton?

We know that singletons are the kind of instances or services that exists only once in an application context. When you write a singleton, deploy it in your server, you will get only one instance of this singleton on the server.

The singleton mentioned above is actually class-loader wide singleton,  i.e. one instance per class-loader. In case of high availabile cluster, when the above singleton is deployed in cluster, there will be one singleton instance in each cluster-node.

But there are cases where we need cluster-wide singleton. For example, in a clustered auction system,
bid orders can come from different node, but the process which deals with the final trading settelment should run only once in the whole cluster. When the node on which the singleton service fails, another node will start  an singleton service automatically. Such singleton is so called cluster-wide singleton, i.e. HA singleton.


We will now take JBoss as example to show how to implement a HA singleton.


JBoss supports deployment of singleton as HA singleton. There are generally 2 ways to deplyoment HA singleton on JBoss

• option1: just put deployment archive under "../deploy-singleton/ ", and the deployment service bekomes a HA singleton. Disadvantages of this way are, no hot-deployment support, in case of node failure, service startup time takes longer...
• option 2: deploy service as MBean. MBean support hot deployment. Singleton MBean will be deployment on all nodes,  but provides service only on one node. 

We take the second option, and show how to deploy a MBean as HA singleton:
We will configure and run a 2-nodes JBoss cluster
We will write a simple MBean.
We will deploy the MBean as HA singlleton.


Configure and run a 2-nodes JBoss cluster

Download JBoss  5.1.0_GA from www.boss.org (http://sourceforge.net/projects/jboss/files/JBoss/JBoss-5.1.0.GA).  Extract the file to some location, we will name it JBOSS_HOME hereafter.

In the extracted JBoss directory goto JBOSS_HOME/server/, create 2 new folders directly here:

node1
node2

Copy all files in JBOSS_HOME/server/all into node1.
Copy all files in JBOSS_HOME/server/all into node2.

Now we have a JBoss cluster with two nodes ready to run.

Start node1:

./run.sh -c node1 -Djboss.service.binding.set=ports-01 -Djboss.messaging.ServerPeerID=1



When you have followed the above steps, you will see in the console like these:


09:57:56,551 INFO  [GroupMember] I am (127.0.0.1:56944)
09:57:56,551 INFO  [GroupMember] New Members : 1 ([127.0.0.1:56944])
09:57:56,551 INFO  [GroupMember] All Members : 1 ([127.0.0.1:56944])
09:57:56,556 INFO  [STDOUT] 
.......

09:58:03,364 INFO  [Http11Protocol] Starting Coyote HTTP/1.1 on http-127.0.0.1-8180

09:58:03,377 INFO  [AjpProtocol] Starting Coyote AJP/1.3 on ajp-127.0.0.1-8109

09:58:03,382 INFO  [ServerImpl] JBoss (Microcontainer) [5.1.0.GA (build: SVNTag=JBoss_5_1_0_GA date=200905221634)] Started in 25s:474ms

The node1 is now started.

Start node2:

./run.sh -c node2 -Djboss.service.binding.set=ports-02 -Djboss.messaging.ServerPeerID=2


You will see in the console lines like these:

|1] [127.0.0.1:56944, 127.0.0.1:55687], old view is null
10:01:57,587 INFO  [GroupMember] I am (127.0.0.1:55687)
10:01:57,587 INFO  [GroupMember] New Members : 2 ([127.0.0.1:56944, 127.0.0.1:55687])
10:01:57,587 INFO  [GroupMember] All Members : 2 ([127.0.0.1:56944, 127.0.0.1:55687])
10:01:57,629 INFO  [STDOUT] 
.....

10:01:59,745 INFO  [Http11Protocol] Starting Coyote HTTP/1.1 on http-127.0.0.1-8280
10:01:59,756 INFO  [AjpProtocol] Starting Coyote AJP/1.3 on ajp-127.0.0.1-8209
10:01:59,762 INFO  [ServerImpl] JBoss (Microcontainer) [5.1.0.GA (build: SVNTag=JBoss_5_1_0_GA date=200905221634)] Started in 17s:400ms

The second node is started, and it joined the cluster.

HA Singleton, Cluster Wide Singleton as MBean in JBoss 5, part2/3 part3/3

                  

SSL Offloading with mod_jk part 6

Document  Version 1.0

   Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
SSLOffloading SSL-Offloading, SSL Termination, Apache, Tomcat, mod_jk configuration, multiple vhosts, multiple SSL certificates one ip

Configure Tomcat to accept SSL handling of mod_jk


We will now configure "SSL Termination" for "TestWebSec20" application.
We need to: 

1. Generate a self-signed SSL certificate.
   This includes creating a ".key" file, a ".csr" file and ".crt file".
2. Configure Apache virtual host i.e. host "ahaha.com" to use SSL
3. Configure Tomcat to accept SSL handling of Apache and mod_jk.


6.1 Generate a self-signed SSL certificate

The following link is the best Tutorial I have ever found for creating self-signed SSL certificate:
http://www.akadia.com/services/ssh_test_certificate.html

Please follow the link to create 3 files in following folders:

• "/etc/apache2/ssl.key/ahaha.com.key", here "ahaha.com.key" is the key file name. 
• "/etc/apache2/ssl.csr/ahaha.com.csr",  here "ahaha.com.csr" is the csr file name.

"/etc/apache2/ssl.crt/ahaha.com.crt",   here "ahaha.com.crt" is the crt file name.

6.2 Configure Apache virtual host for SSL

Configure Apache virtual host for "ahaha.com" to use SSL.
Add a "vhost-ssl.conf" file to Apache's "vhost.d" folder, and add following content to the file:

####START vhost-ssl.conf
NameVirtualHost *:443
<IfDefine SSL>
<IfDefine !NOSSL>

<VirtualHost *:443>
#  General setup for the virtual host
DocumentRoot "/srv/www/vhosts/ahaha.com"
ServerName www.ahaha.com
#ServerAdmin webmaster@example.com
ErrorLog /var/log/apache2/error_log
TransferLog /var/log/apache2/access_log

#   SSL Engine Switch:
#   Enable/Disable SSL for this virtual host.
SSLEngine on

#  SSL protocols
#  Supporting TLS only is adequate nowadays
SSLProtocol all -SSLv2 -SSLv3

#   SSL Cipher Suite:
SSLCipherSuite ALL:!aNULL:!eNULL:!SSLv2:!LOW:!EXP:!MD5:@STRENGTH

#   Server Certificate:
SSLCertificateFile /etc/apache2/ssl.crt/ahaha.com.crt

#   Server Private Key:
SSLCertificateKeyFile /etc/apache2/ssl.key/ahaha.com.key

CustomLog /var/log/apache2/ssl_request_log   ssl_combined
       JkMountCopy On
       JkMount / worker1
       JkMount /* worker1

</VirtualHost>                                  

</IfDefine>
</IfDefine>
####END vhost-ssl.conf

With this vhost configuration for SSL, a SSL request to "ahahacom" will be first processed by mod_jk. mod_jk will take care of the SSL communication:

•    providing client with the corresponding SSL certifiacte 
•    do the SSL hand shake
•   do encryption and decryption
•   and forward it to Tomcat

The communication between mod_jk and Tomcat are in "plain" text,  NO SSL.  

6.3 Configure Tomcat  to accept SSL handling of mod_jk.

With the SSL configuration in "vhost-ssl.conf", mod_jk will take care of the SSL conmunication to browser, and forward client request to Tomcat. Tomcat need to be configured to believe the request from mod_jk is "secured".

Remembe in part5, we configured tomcat for a "ajp connector". 

...
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443"/>
...

To make Tomcat accept mod_jk SSL handling is easy, we just need to change the redirectPort of the above ajp connector to 433:

...
<Connector port="8009" protocol="AJP/1.3" redirectPort="443"/>
...

No restart Apache and Tomcat. Try accessing "secure/HalloSec" again:
http://ahaha.com/TestWebSec20/secure/HalloSec

Your browser will be redirected to HTPPS, and now the server certificate is shown  corectly, and SSL port is not shown any more:

part1 part2 part3 part4 part5

SSL Offloading with mod_jk part 5

Document  Version 1.0

  Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
SSLOffloading SSL-Offloading, SSL Termination, Apache, Tomcat, mod_jk configuration, multiple vhosts, multiple SSL certificates one ip


SSL Port Problem

In part4 of this serial, we have successfully configured, a virtual host "ahaha.com", we have configured Apache and mod_jk, so that when a request comes form browser(clien), mod_jk will try to communicate this request to Tomcat. mod_jk talks "ajp language" to Tomcat. Now we need to configure Tomcat so that, he can also understand "ajp".
This could be accmplished by uncommenting following lines of "server.xml" of tomcat:

...
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443"/>
...

Now restart Tomcat and try accessing the "TestWebSec20"applicaiton again:

"http://www.ahaha.com/TestWebSec20/HalloNormal

And now you will see: 

Now you could access the "TestWebSec20" web applicaiton through a "real" domain name.
And you do NOT see the port 8080 in browser input any more!

Let's try accessing the secured resource, namly:

http://www.ahaha.com/TestWebSec20/secure/HalloSec

And you will see in you browser:

Cool! Isn't it?

The brower was redirected to HTTPS, and after confirmation of th SSL certificate, we see the protected resource!


But, wait a minute! There is still something wrong!

• The  browser was redirected to HTTPS, but we see now another port numer 8443 (which is configured  for Tomcat https),  but we actually don't want to show users the port number in browser. We are expecting something like this

           https:///www.ahaha.com/TestWebSec20/secure/HalloSec

• When we take a closer look at the SSL certificate details, we will see that the certificate here is the certificate of Tomcat, not the certificate of "www.ahaha.com".

To sum it up, we do NOT want to show the SSL port to brower; we want to show the user our "real" certificate that we created for domain "ahaha.com". 

To solve these two problems, we need to introduce a new "term", i.e. so called "SSL offload" or "SSL Termination". This is actually the main purpose of  the  "SSL Offloading with mod_jk" serial I am writing!


So what is SSLTermination / Offloading? 

SSL Termination is used take care of  everything about SSL in a centralized position in a system, at the border between secured and unsecured areas. "SSL Terminator" takes care of the SSL things between client and Servers, and all the traffic behinde "SSL Terminator" is not encrypted but thought to be secure.

To make it simple,  in our case we want apache / mod_jk to take care of the SSL handling, and by NOT Tomcat.


SSL Offloading with mod_jk part 6
part1 part2 part3 part4

SSL Offloading with mod_jk part 4

Document  Version 1.0

  Copyright © 2012-2013 beijing.beijing.012@gmail.com

Keywords:
SSLOffloading SSL-Offloading, SSL Termination, Apache, Tomcat, mod_jk configuration, multiple vhosts, multiple SSL certificates one ip

Fronting Tomcat with Apache and mod_jk

Now, we will  try to front Tomcat with Apache and mod_jk. The target OS is Linux -OpenSuse12.1. The procedure might be slightly different between different Linux systems.


4.1 Install Apache2

Normmally Apache2 is included in OpenSuse12.1 package. If Apache is installed , there will be an apache2 folder under "etc":
/etc/apache2/

When you can't find this folder, just try to install Apache2 and mod_jk with "Yast".
After installation, start apache2:

#  cd /etc/init.d
# ./apache2 start

Now when you type following URL in brower input "http://localhost", you will land on  the apache2 default page.

Just now we have also installed mod_jk ( Yast, search mod_jk, and install). But mod_jk is not loaded by apache via default. 


4.2 Configure Apache to load/use mod_jk

Configuration of Apache to use mod_jk, three things need to be done:

1.  Add a file "mod_jk. conf" to Apache. This will load mod_jk module and specify "worker.properties" file for mod_jk
2.  Add a "worker.properties" file to configure mod_jk workers
3.  Add vritual host which will use mod_jk


4.2.1 mod_jk.conf

Create a "mod_jk.conf" file in folder "/etc/apache2/conf.d/".
You could also name the file "sample.sss.conf", but important is, the file name must have ".conf" at the end, and the is put in "conf.d" folder. In this way, the file will be found and loaded by apache.

Content of the "mod_jk.conf" file:

#### START mod_jk.conf
# mod_jk configuration for Apache
# Load mod_jk module
LoadModule jk_module /usr/lib/apache2/mod_jk.so

# Tell Apache where to find workers.properties. We assume Tomcat runs on a differerent machine than # Apache. and put workers.properties file near to apache
JkWorkersFile /etc/apache2/conf.d/workers.properties

# mod_jk log configuration
JkLogFile /var/log/apache2/mod_jk.log
JkLogLevel debug
jkLogStampFormat "[%a %b %H:%M:%S %Y]"

# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat set the request format
JkRequestLogFormat "%W %V %T"

# Send everything for context /TestWebSec20 to worker ajp13
JkMount /TestWebSec20 worker1
JkMount /TestWebSec20/* worker1

# Send everything for context /sampple to worker ajp13
#JkMount /sample worker1
#JkMound /sample/* worker1

##### END mod_jk.conf

Explanation to the "mod_jk.conf" file

1. LoadModule tells Apache to load  mod_jk module.
2. JkWorkersFile specifies the worker file location. Workers file tells mod_jk, where to find the real 
    application (i.e. ip and port of the Server / application). 
3. JkMount, have 2 entries, one with another without "*"


4.2.2 workers.properties file

Create a "workers.properties" file in conf.d folder with following content:

##### SATART works.properties
# Define a worker named "worker1"
# Several worker names are separated by ","
worker.list=worker1

# Set properties for worker1 to use ajp13 protocol and run on port 8009
worker.worker1.type=ajp13
worker.worker1.host=localhost
worker.worker1.port=8009
worker.worker1.lbfactor=50
worker.worker1.cachesize=10
worker.worker1.cache_timeout=600
worker.worker1.socket_keepalive=1
worker.worker1.socket_timeout=300

##### END works.properties

Explanation to the workers.properties file:

1. We just configured one worker. In case of more workers, worker names are separated by ",". 
   For example: 
   worker.list=worker1, worker2
 2. Workers are configured to use / communicate to certain host and port using "ajp" protocol.
     When a request comes to apache /mod_jk  via http or https, mod_jk will redirect the request  NOT to   
     HTTP or HTTPS ports, but to AJP ports.(We will configure Tomcat to use AJP connector).



4.2.3 Create virtual host and configure the virtual host to use mod_jk

Assume we will configure a new virtual host "ahahacom", two steps are needed:

   Step 1. 
    Add virtual host to "hosts" file. Edit "hosts" file under "/etc/", add following lines:
   127.0.0.1  ahaha.com www.ahaha.com

   Step 2. 
   Create a vhost.conf file under "/etc/apache2/vhost.d/" with following content

#### STAART vhost.conf
   <VirtualHost *:80>    
         ServerAdmin info@ahaha.com
         ServerName ahaha.com

        # DocumentRoot: The directory out of which you will serve your
        # documents. By default, all requests are taken from this directory, but
        # symbolic links and aliases may be used to point to other locations.
        DocumentRoot /srv/www/vhosts/ahaha.com

        # if not specified, the global error log is used
        ErrorLog /var/log/apache2/ahaha.com_error.log
        CustomLog /var/log/apache2/ahaha.com_access.log combined

        JkMount / worker1
        JkMount /* worker1  
   </VirtualHost>  
   #### END vhost.conf

With vhost configured, the "TestWebSec20" web application could be accessed "later" using following URL:

http://www.ahaha.com/TestWebSec20 

Now restart apache:

./apache2 restart

Try accessing the "http://www.ahaha.com/TestWebSec20/HalloNormal" with your browser.

The browser will show an error "Service Temporarily Unavailable"

Check the mod_jk error log we just configured "
/var/log/apache2/ahaha.com_error.log"

You will see a new error log entry like this:

[Mon Sep 03 20:36:32 2012] [error] [client 127.0.0.1] (2)No such file or directory: cannot access type map file: HTTP_SERVICE_UNAVAILABLE.html.var 

When you see this error, your virtual host configuration and  mod_jk configuration at Apache/mod_jk side are correct!

This above error says that mod_jk can not find the worker, i.e. it can not find the Tomcat server. Remember, mod_jk tries to talk to Tomcat with "ajp" protocol, to certain host name and port (as configured in "workers.properties"), so the question is now, does Tomcat know about the "ajp" thing? 
No not yet!

Configure Tomcat to communicate with mod_jk, using AJP
SSL Offloading with mod_jk part 5
part1 part2 part3 part6