Showing posts with label Tomcat. Show all posts
Showing posts with label Tomcat. Show all posts

Mar 10, 2017

Virgo 3.7.0 Released

 Today Florian announced the availability of Virgo 3.7.0. 

Virgo 3.7.0 brings several improvements (most notably Tomcat 8.5 and Spring Framework 4.2) and is paired by release 1.5.0 of the Virgo Tools, the Eclipse IDE integration.

This is the first official Virgo release since I joined the project as a committer, and it's also my first open source release in general. I am very happy and very proud of what we achieved.

Download Virgo and get Virgo Tools from the update site!

Aug 30, 2014

How to use Hyperic to monitor Eclipse Virgo 3.6

Few days ago I started looking for a monitoring tool to track the health status of our Virgo Server for Apache Tomcat instances.

As I could not find anything Virgo specific, I looked for an open source tool that supported Apache Tomcat 7.0.

First try was Moskito, but it does not seem to support the  OSGi-aware Tomcat that is embedded in Virgo. In any case I did not like the deployment approach, which requires to add Mosquito to every monitored Web App.

So I searched for monitoring tools that used Tomcat JMX support, as I believed such tools should be unaffected by the OSGi runtime, and I found Hyperic.

Hyperic 5.8 

Hyperic is developed by SpringSource and there exists both an enterprise distribution (vCenter Hyperic) and a free-to-use distribution.

Hyperic consists of the Hyperic Server and the Hyperic Agent.

The server is a Tomcat6-based Web application that uses PostgreSQL for storing historical data and which provides the monitoring console.

The agent is a stand-alone Java application that integrates with the O.S. via the well known Sigar native library (used by JMeter, Elasticsearch etc), and which features a number of plug-ins for different servers and database systems (WebSphere, WebLogic, JBoss, Tomcat, Apache, Oracle, DB/2,  PostgreSQL, MySQL and many more).

A screenshot taken from the SourceForge download page

Installation and usage

You must first install the Hyperic Server, typically on a dedicated machine that you will be using for monitoring. Then you must install the Hyperic Agent on every server machine that you want to monitor.

The Hyperic Agent will automatically discover running database systems and application servers on the machine where it is installed, and it will offer them as observable resources to the Hyperic Server.

From the Hyperic Server console the administrator can select what to monitor (e.g. Apache Tomcat, MySQL, but also operating system CPU, SWAP, file system, I/O, network etc)

The agent will then start sampling the requested items and will push the information to the console. which will be able to display historical data, paint graphs, and which can be configured to raise alerts when certain conditions are met (e.g. low disk space, high CPU usage, Tomcat process crashed etc etc).

Tweaking the Virgo installation to pretend it's Tomcat 7

Even if Virgo was created by SpringSource, Hyperic does not seem to support Virgo OOTB, or at least the open source edition does not seem to support Virgo.

It however supports Tomcat 7.0 via an agent plug-in called tomcat-plugin.jar.
A quick look at the plug-in sources reveals the Tomcat 7.0 discovery process:
  1. The plug-in uses a Sigar query to identify running processes that are potential Tomcat servers. The query just looks for running Java processes that include parameter -Dcataliba.base=<some_path> in the command line.
  2. The version of Tomcat is determined by checking the existence of some files that are specific to each Tomcat version. For Tomcat 7, the file is <catalina.base>/lib/tomcat-api.jar (see here).
  3. Once Tomcat is identified, the agent will connect to the process to collect samples, provided that JMX support is enabled. Luckily enough JMX is enabled per default in Virgo.
To have Virgo recognized as Tomcat 7.0:
  1. Set JAVA_OPTS to -Dcatalina.base=<VIRGO_INSTALLATION_FOLDER> before starting Virgo:
    export JAVA_OPTS=-Dcatalina.base=/home/giamma/virgo
    ./startup.sh
  2.  Create an empty file named tomcat-api.jar in the Virgo lib folder:
    cd /home/giamma/virgo 
    touch lib/tomcat-plugin.jar
Now start Virgo and after few seconds it will show up as a Tomcat 7 server in the Hyperic console.
Note that the Hyperic template for Tomcat 7 will use wrong defaults for log files etc, but you can change them after enrolling the Virgo server in the Hyperic console.

In case you wonder, the catalina.base system property has no side effect on Virgo, see here.

Patching the Agent to monitor global data sources

The current Hyperic Agent plugin for Tomcat is capable of monitoring only data sources that are defined at the Web App level.

If you are using Tomcat global data sources as we are doing (see here for details), they will not be monitored by Hyperic.

However, you can easily patch the tomcat-plugin.jar as explained here. Just remember that after patching the plug-in you must replace it both in the Hyperic Agent and in the Hyperic Server.

./agent-5.8.2/bundles/agent-5.8.2/pdk/plugins/tomcat-plugin.jar

./server-5.8.2/hq-engine/hq-server/webapps/ROOT/WEB-INF/hq-plugins/tomcat-plugin.jar

Oct 31, 2012

Global JNDI support in Virgo Server 3.5 for Tomcat


This rather long post analyses four solutions to the problem of using data sources, and more in general JNDI in Virgo, the 4th being the one I recommend and decided to use, which consists in leveraging Tomcat's built-in JNDI provider in Eclipse Virgo Server for Apache Tomcat.

If you are not interested in the reasons why I dropped the first three, jump directly to the fourth.

Even if the post is mostly focused on JDBC data sources, once Tomcat JNDI provider is exposed to the application it can be used for any type of resource, not only data sources.

Most of the credits for this solution go to my colleague Stefano Malimpensa.

1. JDBC data sources in OSGi

The most correct approach to obtain a JDBC data source in a pure OSGi enterprise application consists in using the OSGi JDBC Service (see the official OSGi JDBC specification).  In Virgo, that means using Gemini DBAccess.

However, in my humble opinion Gemini DBAccess is not an optimal solution for a number of  reasons:

  • Gemini DBAccess is currently available only for Derby, and to use a different database you need to write your own implementation. Not a big issue but some extra effort anyway.
  • To integrate DBAccess with EclipseLink you should probably use Gemini JPA, which is affected by a bug that prevents connection pooling from working and is therefore not usable in production https://bugs.eclipse.org/bugs/show_bug.cgi?id=379397
  • When using DBAccess with Gemini JPA you need configure connection parameters in each bundle's persistence.xml. I find this inconvenient because it is necessary to repackage the bundles every time the connection parameters change, and due to the modular nature of OSGi one complex application may include several bundles with persistence units.
  • If DBAccess is used without GeminiJPA, DBAccess will provide only a data source factory, and it will be the responsibility of your code to instantiate and configure the data source (e.g. pass in user name, password etc). In such case you would need to support a configuration file to let system administrators easily change connnection parameters, which is again extra effort.
  • DBAccess requires the OSGi registry, which means it would not work with legacy code or third party libraries written for J2EE. 
As the name implies, Gemini DBAccess is tailored to data base resources. If you want a single, unified approach for looking up any type of resource, then it's not a good fit for you.

2. Full JNDI in OSGi

 At this point you may want to try Gemini Naming,  which implements the OSGi JNDI service specification. Even Gemini Naming is in my humble opinion not an optimal solution:
  • There is no configuration console, nor a configuration file: the only way you can bind resources in the JNDI namespace is programmatically. This implies a lot of boring initialisation code, and you probably need to support a configuration file to let system administrators easily change configuration parameters, which is again extra effort.
  • Gemini Naming requires the OSGi registry, which means it would not work with legacy code or third party libraries written for J2EE.

3. Local JNDI declared inside a Web App


Virgo supports JNDI lookups for data sources inside a Web App. To achieve this you have to:
  • Include in the Web application the JDBC driver(s) and the pool implementation (e.g. Apache DBCP or Tomcat JDBC) as jars in your WEB-INF/lib folder
  • Configure web.xml to list the usual JNDI resource-refs
  • Include a Tomcat context.xml file in the Web App and configure it as explained here and here
The above will work for JDBC data sources but has the following draw backs:
  • You must include the JDBC driver and pool in every Web App of yours. This means that each Web App will have its own pool, even if they connect to the same database, and that you must repackage the WAR if you need to update the JDBC driver
  • JNDI lookup will work only in a thread originated by a HTTP request. This means that application bundles that are not WARs will not be able to obtain the data source via a JNDI lookup, unless their code is executed by a thread started by the Web container. In fact, the JNDI lookup will fail from threads created by Equinox: this is for example the case of code that observes OSGi framework lifecycle events (BundleListener) and need access the database when a bundle is installed or uninstalled.
In my case the main show stopper to this solution is the thread issue described above, because my application must access the database (and therefore the data source) from threads that are not always started by the Web container. If you are interested in the historical roots of this apparently strange limitation, I recommend reading this post by Neil Bartlett.

4. Tomcat global JNDI registry


Another option is available, which is not affected by any of the above issues and limitations and it consists in using Tomcat's global JNDI support. You gain a general purpose well tested and well documented JNDI registry capable of deploying any type of resource, not only data-sources (can even be extended to support custom resource types http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html#Adding_Custom_Resource_Factories).

Please note that this solution will make the global JNDI namespace available, disabling the Web App java:comp/env context. In order words, you cannot use the java:comp/env prefix in your JNDI lookups. This should be an acceptable limitation, given that the java:comp/env prefix in any case would work only within a Web App and not from a plain OSGi bundle.

In order to use the Tomcat  JNDI registry for data sources in Virgo the following mandatory steps are required:

  1. Create a bundle fragment for Catalina to extend Virgo's Tomcat with connection pooling support. The Virgo distribution contains in fact a stripped down version of Tomcat that removes the libraries required for JDBC pooling. Luckily enough, you can create a fragment to contribute the libraries back to Catalina. You just need to make sure that your fragments are placed in the bundle repository folder, not in pickup.
  2. Create a bundle fragment for Catalina to make the required JDBC driver(s) available to the server and the application
  3. Create a fragment that gets the JNDI context from Tomcat and that registers in the JVM a global InitialContextFactoryBuilder
  4. Edit tomcat-server.xml and define your global resources

All the above fragments are a convenient method for extending Tomcat/Catalina to use third party libraries whose Java packages were not originally imported by the Virgo bundles. For further details refer to this post by Glyn Normington, the project lead of Virgo. If you are working with Virgo his personal blog is a must read!



1. The pool bundle fragment

Here is a sample for Apache DBCP. The MANIFEST.MF below is added to the DBCP JAR, that's why there is no Bundle-Classpath. Mind the fragment host header.

Manifest-Version: 1.0
Fragment-Host: com.springsource.org.apache.catalina
Bundle-ManifestVersion: 2
Bundle-Name: Tomcat DBCP
Bundle-SymbolicName: org.apache.tomcat.dbcp
Bundle-Version: 7.0.27
Bundle-Vendor: apache
Bundle-RequiredExecutionEnvironment: JavaSE-1.6
Import-Package: javax.management,
 javax.management.openmbean,
 javax.naming,
 javax.sql,
 org.apache.juli.logging


2. The JDBC driver bundle fragment

Here is a sample for PostgreSQL. The MANIFEST.MF below is added to the PostgreSQL driver JAR, that's why there is no Bundle-Classpath. Mind the fragment host header.

Manifest-Version: 1.0
Fragment-Host: com.springsource.org.apache.catalina
Bundle-ManifestVersion: 2
Bundle-Name: PostgreSQL JDBC Driver
Bundle-SymbolicName: org.postgresql.jdbc.catalina
Bundle-Version: 9.2.1000
Bundle-Vendor: postgresql
Bundle-RequiredExecutionEnvironment: JavaSE-1.6


3. JNDI bridge fragment

Create a fragment with the following MANIFEST.MF that includes the two classes below and place it in the repository folder.

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: JNDITomcatBridge
Bundle-SymbolicName: jndi.tomcat.bridge
Bundle-Version: 1.0.0.qualifier
Bundle-Vendor: org.example
Fragment-Host: com.springsource.org.apache.catalina;bundle-version="7.0.26"
Bundle-RequiredExecutionEnvironment: JavaSE-1.6
Import-Package: org.apache.catalina.mbeans;version="7.0.26"

The code below consists of two classes.

The first, GlobalJNDILifecycleListener, is a GlobalResourcesLifecycleListener subclass that downcasts the server instance to get the global JNDI context and that registers in the JVM NamingManager a custom InitialContextFactoryBuilder which wraps the JNDI context obtained from Tomcat.

Other options are of course possible, including doing everything in the custom implementation of InitialContextFactoryBuilder without subclassing the listener, but whatever approach you adopt, it is important to make sure that the invocation to NamingManager.setInitialContextFactoryBuilder occurs only once in the life of a Virgo server instance, because the method will fail and raise a runtime exception if it is called more than once.


import javax.naming.Binding;
import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;
import javax.naming.spi.NamingManager;

import org.apache.catalina.Lifecycle;
import org.apache.catalina.LifecycleEvent;
import org.apache.catalina.Server;
import org.apache.catalina.mbeans.GlobalResourcesLifecycleListener;
import org.apache.juli.logging.Log;
import org.apache.juli.logging.LogFactory;

public class GlobalJNDILifecycleListener extends GlobalResourcesLifecycleListener {
    private static final Log log = LogFactory.getLog(GlobalJNDILifecycleListener.class);

    @Override
    public void lifecycleEvent(LifecycleEvent event) {
        super.lifecycleEvent(event);
        if (Lifecycle.START_EVENT.equals(event.getType())) {
            Server server = (Server) event.getLifecycle();
            Context ctx = server.getGlobalNamingContext();
            ContextFactory factory = new ContextFactory(ctx);
            try {                
                NamingManager.setInitialContextFactoryBuilder(factory);
                log.info("Published Global Naming as default InitialContext");
                logJNDIEntries(ctx, null);
            } catch (NamingException e) {
                log.error("Naming Exception:", e);
            }
        }

    }

    private void logJNDIEntries(Context context, String prefix) throws NamingException {

        NamingEnumeration<Binding> namingEnumeration = context.listBindings("");

        while (namingEnumeration.hasMoreElements()) {
            Binding binding = namingEnumeration.next();
            String nameEntry = binding.getName();
            String fullName = (prefix == null || prefix.equals("") ? nameEntry : prefix + "/" + nameEntry);
            String entryClassName = binding.getClassName();
            if (Context.class.getName().equals(entryClassName)) {
                Context ctx = (Context) binding.getObject();
                logJNDIEntries(ctx, fullName);
            } else {
                log.info("Found: " + fullName);
            }
        }
    }

}
import java.util.Hashtable;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.naming.spi.InitialContextFactory;
import javax.naming.spi.InitialContextFactoryBuilder;

/**
 * Simple implementation of {@link InitialContextFactory} that returns the global {@link InitialContext}
 * obtained from Tomcat
 *
 * @author giamma, stefano
 *
 */
public class ContextFactory implements InitialContextFactoryBuilder, InitialContextFactory {

    private Context context;

    public ContextFactory(Context context) {
        this.context = context;
    }
   
    @Override
    public InitialContextFactory createInitialContextFactory(Hashtable environment) throws NamingException {
        return this;
    }

    @Override
    public Context getInitialContext(Hashtable environment) throws NamingException {
        return context;
    }

}

4. tomcat-server.xml

 

Replace the listener with yours and declare your JNDI resources:
 
 <-- Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />  -->
 <Listener className="com.example.catalina.mbeans.GlobalJNDILifecycleListener"/>
 
<GlobalNamingResources>
 <Resource name="jdbc/db"
           type="javax.sql.DataSource" username="user" 
           password="password" 
           factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory" 
           driverClassName="org.postgresql.Driver" 
           url="jdbc:postgresql://127.0.0.1/db" 
           maxActive="20" maxIdle="10"/>
</GlobalNamingResources>


You can now happily perform plain old new InitialContext().lookup() from every method of every class of your OSGi application deployed in Virgo, regardless of the origin of the thread.