Profiling Virgo with JProfiler

The information below has been tested with JProfile 5 on Linux 64bit, and assumes that the JProfiler Agent is installed in folder /opt/jprofiler5.
While this post refers to a rather old JProfiler version, I am sure the principles apply also to more recent JProfiler version. 
 
The following environment variables must be set before starting Virgo:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/jprofiler5/bin/linux-x64
export JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/a:/opt/jprofiler5/bin/agent.jar -agentpath:/opt/jprofiler5/bin/linux-x64/libjprofilerti.so=port=8849" 

If your JProfiler installation is not located in /opt/jprofiler5 change the above paths accordingly.

A convenient method for setting the environment variables consists in creating a setenv.sh file in the Virgo bin folder. If such file exists, it will be invoked by the startup.sh script.

 

Additionally, it is necessary to modify the java6-server.profile file contained in the <VIRGO_HOME>/configuration folder to ensure that the property org.osgi.framework.bootdelegation contains the com.jprofiler package, as in the example below.
Note: do not replace the file content, just add the com.jprofile.*,\ line.

org.osgi.framework.bootdelegation = \
 org.eclipse.virgo.osgi.extensions.*,\
 org.eclipse.virgo.osgi.launcher.*,\
 org.eclipse.virgo.kernel.authentication,\
 com.sun.*,\
 com.jprofiler.*,\
 javax.xml.*,\
 org.apache.xerces.jaxp.*,\
 org.w3c.*,\
 org.xml.*,\
 sun.*

After applying the above changes, start Virgo and connect using the JProfiler GUI.

Read More

How to use Hyperic to monitor Eclipse Virgo 3.6

Few days ago I started looking for a monitoring tool to track the health status of our Virgo Server for Apache Tomcat instances.

As I could not find anything Virgo specific, I looked for an open source tool that supported Apache Tomcat 7.0.

First try was Moskito, but it does not seem to support the  OSGi-aware Tomcat that is embedded in Virgo. In any case I did not like the deployment approach, which requires to add Mosquito to every monitored Web App.

So I searched for monitoring tools that used Tomcat JMX support, as I believed such tools should be unaffected by the OSGi runtime, and I found Hyperic.

Hyperic 5.8 

Hyperic is developed by SpringSource and there exists both an enterprise distribution (vCenter Hyperic) and a free-to-use distribution.

Hyperic consists of the Hyperic Server and the Hyperic Agent.

The server is a Tomcat6-based Web application that uses PostgreSQL for storing historical data and which provides the monitoring console.

The agent is a stand-alone Java application that integrates with the O.S. via the well known Sigar native library (used by JMeter, Elasticsearch etc), and which features a number of plug-ins for different servers and database systems (WebSphere, WebLogic, JBoss, Tomcat, Apache, Oracle, DB/2,  PostgreSQL, MySQL and many more).

A screenshot taken from the SourceForge download page

Installation and usage

You must first install the Hyperic Server, typically on a dedicated machine that you will be using for monitoring. Then you must install the Hyperic Agent on every server machine that you want to monitor.

The Hyperic Agent will automatically discover running database systems and application servers on the machine where it is installed, and it will offer them as observable resources to the Hyperic Server.

From the Hyperic Server console the administrator can select what to monitor (e.g. Apache Tomcat, MySQL, but also operating system CPU, SWAP, file system, I/O, network etc)

The agent will then start sampling the requested items and will push the information to the console. which will be able to display historical data, paint graphs, and which can be configured to raise alerts when certain conditions are met (e.g. low disk space, high CPU usage, Tomcat process crashed etc etc).

Tweaking the Virgo installation to pretend it's Tomcat 7

Even if Virgo was created by SpringSource, Hyperic does not seem to support Virgo OOTB, or at least the open source edition does not seem to support Virgo.

It however supports Tomcat 7.0 via an agent plug-in called tomcat-plugin.jar.
A quick look at the plug-in sources reveals the Tomcat 7.0 discovery process:

1. The plug-in uses a Sigar query to identify running processes that are potential Tomcat servers. The query just looks for running Java processes that include parameter -Dcataliba.base=<some_path> in the command line.
2. The version of Tomcat is determined by checking the existence of some files that are specific to each Tomcat version. For Tomcat 7, the file is <catalina.base>/lib/tomcat-api.jar (see here).
3. Once Tomcat is identified, the agent will connect to the process to collect samples, provided that JMX support is enabled. Luckily enough JMX is enabled per default in Virgo.

To have Virgo recognized as Tomcat 7.0:

1. Set JAVA_OPTS to -Dcatalina.base=<VIRGO_INSTALLATION_FOLDER> before starting Virgo:
   

   export JAVA_OPTS=-Dcatalina.base=/home/giamma/virgo

   ./startup.sh

2.  Create an empty file named tomcat-api.jar in the Virgo lib folder:
   

   cd /home/giamma/virgo 

   touch lib/tomcat-plugin.jar

Now start Virgo and after few seconds it will show up as a Tomcat 7 server in the Hyperic console.
Note that the Hyperic template for Tomcat 7 will use wrong defaults for log files etc, but you can change them after enrolling the Virgo server in the Hyperic console.

In case you wonder, the catalina.base system property has no side effect on Virgo, see here.

Patching the Agent to monitor global data sources

The current Hyperic Agent plugin for Tomcat is capable of monitoring only data sources that are defined at the Web App level.

If you are using Tomcat global data sources as we are doing (see here for details), they will not be monitored by Hyperic.

However, you can easily patch the tomcat-plugin.jar as explained here. Just remember that after patching the plug-in you must replace it both in the Hyperic Agent and in the Hyperic Server.

./agent-5.8.2/bundles/agent-5.8.2/pdk/plugins/tomcat-plugin.jar

./server-5.8.2/hq-engine/hq-server/webapps/ROOT/WEB-INF/hq-plugins/tomcat-plugin.jar

Read More