Tricking Out Spring Bean Factories – How to Embed and Programmatically Define Beans in Spring

I recently had the task of writing an integration test utility that would setup a complex Java Content Repository (JCR) for use by integration tests.  This turned out to be much more fun than I thought all because of Spring ;).  As I’ve worked with Spring over the years, I’ve found that staying in of the normal mode of declaring Spring xml files and letting it rule your application works great.  However, once you start treating Spring like a 2nd class citizen and telling it what to do, it will make your life pretty difficult.  One thing I will say though, is just about anything is possible; you will just have to work at it.  In my case, trial, error, and luckily happening upon useful javadoc eventually got me the answer.

Here is what I set out to do…

  1. I needed to control the loading of Spring in an encapsulated way, inside my utility class.  I did not want to require every JUnit test that uses my utility to have to annotate with a  @RunWith(SpringJUnit4ClassRunner.class) and a @ContextConfiguration(locations = { “classpath:/some.spring.xml” }).  I wanted the JUnit test writer to not care about how the JCR subsystem was configured and started.
  2. The test utility needs to be able to read more than one Spring beans xml file for configuration.  Two Spring beans files are required to configure the test JCR environment: the production-ready one, plus one with a few overrides specific to the test environment.
  3. Lastly, and the requirement that was the most difficult to get working, I needed to programmatically configure/override one of the beans that was declared prior by one of Spring xml files.  Spring was managing the jcrRepository factory bean, which writes data to a specified dir on your filesystem.  In my integration test environment, I needed that folder to be unique for each test run.  That’s why declaring the “home” dir in a Spring xml file just wouldn’t do.  The test utility needed to generate a unique dirname.

After many different attempts at using ApplicationContexts and BeanFactories in various interesting ways, I came upon the answer.  DefaultListableBeanFactory is the best suited Spring bean factory for complex configuration.  Once I discovered it’s ability to both programmatically define beans *and* read in more than one Spring xml bean file, the problem was solved.  Here is a snippet showing how, in my test utility class, I was able to instantiate a DefultListableBeanFactory.  The code illustrates using XmlBeanDefinitionReader to configure beans based on Spring xml files, then how to programmatically define an additional bean (with the unique dir name as a property).

    DefaultListableBeanFactory beanFactory = new DefaultListableBeanFactory();
    XmlBeanDefinitionReader bdr = new XmlBeanDefinitionReader(beanFactory);
    bdr.loadBeanDefinitions(new ClassPathResource("repository.spring.xml"));
    bdr.loadBeanDefinitions(new ClassPathResource("repository-test-override.spring.xml"));
    BeanDefinitionBuilder beanBuildr = BeanDefinitionBuilder
        .rootBeanDefinition("org.springframework.extensions.jcr.jackrabbit.RepositoryFactoryBean");
    beanBuildr.setScope(BeanDefinition.SCOPE_SINGLETON);
    beanBuildr.addPropertyValue("configuration", new ClassPathResource("/jackrabbit-test-repo.xml"));
    beanBuildr.addPropertyValue("homeDir", new FileSystemResource(tempDir));
    beanFactory.registerBeanDefinition("jcrRepository", beanBuildr.getBeanDefinition());
Posted in Uncategorized | Tagged , , , , , | Leave a comment

How to build the Pentaho BIServer

From time to time, I get asked (on the ##pentaho irc channel in freenode) just how to build the Pentaho BIServer.  In this post, I’ll outline two methods for constructing a full Tomcat archive bundle which has the Pentaho BIServer installed.

First things first, checkout the source of what is known as the “BI Platform”.  These are the core pieces that comprise a BIServer.   Note: replace <path> with either “trunk”, or “tags/3.7.0-GA” or the like.

$ svn co svn://source.pentaho.org/svnroot/bi-platform-v2/<path> bi-platform

Now that you have the source, you have two choices, full build or fast build.  There are advantages and disadvantages that I’ll outline below…

Full build (as release does it)..

Use this method if you wish to build a Tomcat archive that is as close to a released version as possible.  This is the method that most closely reflects the way that Pentaho cuts releases of the BIServer.  The disadvantage to this method it is more time consuming than “fast build” when doing iterative development.

  1. Change directory to the bi-platform-build project
    $ cd bi-platform-build
  2. Create an override.properties by copying dev_build.properties and editing the properties.  Pay particular attention to “tomcat.dir”
  3. Build projects (cycles through individual projects and builds and publishes the jars locally)
    $  ant publish-all
  4. Assemble the compiled and published jars into the pentaho webapp installed to a Tomcat archive
    $ cd ../bi-platform-assembly
    $ ant -f assembly.xml clean-all resolve assemble

    The Tomcat archive will be located at the location specified by “stage.dir” in your override.properties.

Fast build, aka “dev build” (recompile changed classes only)..

Use this method if you have made code changes and want to see them quickly reflected in a Tomcat archive.   Changed java files will be compiled to class files which will be placed in <tomcat>/WEB-INF/classes.

  1. Change directory to the bi-platform-build project
    $ cd bi-platform-build
  2. Create a dev_override.properties by copying dev_build.properties and editing the properties.  Pay particular attention to “tomcat.dir”
    which must point to a vanilla extracted Tomcat (version 6)
  3. Build the Tomcat archive from scratch, placing classes in WEB-INF/classes of the pentaho webapp
    $ ant -f dev_build.xml -Djavac.deprecation=false dev-rebuild
  4. After this has completed, you can make changes to individual projects and just build the changes with the “dev-update” target:
    $ ant -f dev_build.xml dev-update

    The Tomcat archive will be located at the location specified by “stage.dir” in your dev_override.properties.  It is a good idea to restart Tomcat after making changes to class files.

  5. Just a few notes regarding this method.  There are a few known problems, I’ll list the ones I know about here:
    • If you have made dependency changes to your project, i.e. changes to ivy.xml, do not use the fast build as your new dependencies will not be brought into tomcat; instead, use the full build.
    • If you are using this method while changing non-Java resource files, beware that sometimes your changes will not be moved to Tomcat during a dev-update
    • There is a current bug in this method that causes web applications to fail to compile due to a missing servlet-api.jar.  To get around this, grab the servlet-api.jar from your target Tomcat and place it or link it to WEB-INF/lib so your code will compile.  Alternatively, you can modify the classpath path in the build.xml script to reflect the location of your servlet-api.jar

This is not a comprehensive guide to build Pentaho BIServer by any stretch but I hope it does help developers to understand what build options are available.

Community Build Framework

There is a third option, CBF, which is a consultant/solution developer oriented abstraction that is built on the “fast build” scripts.  I won’t go into details here, just link you to http://wiki.pentaho.com/display/COM/CBF+-+Community+Build+Framework

/phyto

Posted in Uncategorized | Tagged , , , , | 4 Comments

Demystifying the JBoss5 jboss-classloading.xml file

I have recently been given the task of getting the Pentaho BIServer to deploy within JBoss5 AS.  Classloading has aparently been completely reworked in v5 vs v4 and prior.  The new way to customize classloading behavior is to write up a jboss-classloading.xml file and drop it in various spots within your application to be deployed.  After many hours of being frustrated reading blog and forum posts and not finding a single source of information on exactly what the jboss-classloading.xml is and does and where to put it, I figured I’d try to save someone else (and myself later after I forget this) the same pain I’ve experienced.  Below I answer some of the questions I had on my journey.

Where do I place a jboss-classloading.xml file?

To control classloading behavior of an..

  • ..EAR, place it in the META-INF of the ear, e.g. myapp.ear/META-INF/jboss-classloading.xml
  • ..WAR, place it in the WEB-INF of the war, e.g. myapp.war/WEB-INF/jboss-classloading.xml
  • ..JAR, place it in the META-INF of the jar, e.g. mylib.jar/META-INF/jboss-classloading.xml  (I’ve not personally tried this but you can find info on this in JBoss forums)

What are all the attributes and what do they mean?

  • name - typically the name of war or ear, e.g. “myapp.war”
  • import-all – (“true”/”false”) – true means make all classes exported from other applications visible to this application
  • export-all – value “NON_EMPTY” – all classes are exposed to other applications
  • domain – a classloading domain, can be an arbitrary name but typically you will want to name it the name of your application, e.g. “myapp.ear” or “myapp.war”. If a domain by that name already exists, your app will join that classloading domain.
  • parent-domain – default is “DefaultDomain” which is shared with all other applications that don’t specify a domain.  If you prefer to delegate to an explicit parent domain when a class is not found in yours, then specify an existing domain here, as in delegating to an EAR’s domain, e.g. “myapp.ear”
  • top-level-classloader – used to enable embedded apps, such as a war within an EAR, to participate in a top-level domain (see JBCL-125)
  • parent-first (“true”/”false”) – if false, the app is in non-j2ee compliance mode, where the app’s domain is searched first for a class before searching the parent domain.  The default for top-level deployed WARs is “false”, however, I have found that the default behavior for WARs within EARs is “true”

How do I deploy a WAR within an EAR in which the WAR can see all jars in the EAR, but the EAR’s classes trump JBoss’s?

The original problem I had was the Pentaho BIServer application uses a more recent version of commons-collections.jar, more recent than the one in JBoss’s own library.  When deploying the Pentaho WAR outside of an EAR, I did not have a problem since as of JBoss5, WARs are deployed with a classloading scheme that looks up classes within the WAR first, before delegating to the parent domain.  However, when I deployed the Pentaho EAR I saw the opposite, as if JBoss’s copy of commons-collections.jar was found first, not the EAR’s classes first.  I needed to override this behavior such that EAR classes are preferred over JBoss’s.   The situation I started with was a pentaho.war within a pentaho.ear and no jboss-classloading.xml files present.  As I mentioned earlier, once I deployed this, JBoss libs were preferred over mine.  So I had to go to work authoring some custom classloading.  Here’s what I wound up with:

Contents of jboss-classloading.xml in pentaho.ear/META_INF:

<classloading xmlns="urn:jboss:classloading:1.0"
domain="pentaho.ear"
export-all="NON_EMPTY"
import-all="true"
parent-first="false">
</classloading>

The important parts here: (1) we are specifing that the pentaho.ear application participate in it’s own domain “pentaho.ear”.  This is so in the embedded war, we can declare this domain as it’s parent. (2) by setting parent-first to false, we are essentially declaring that EARs classes be preferred over JBoss’s (in the DefaultDomain presumably).

One funny side-effect to mention here is that I was actually able to deploy an EAR in which javax.servlet.HttpServlet was not found during deployment.  Amazing that this is even possible, but it happened.   The jboss-classloading.xml file that made this happen included only a parent-first=”false” attribute, nothing else.  I haven’t eliminated the attrs one-by-one to find out which is responsible for this; I suspect perhaps import-all.

Contents of jboss-classloading.xml in pentaho.ear/pentaho.war/WEB-INF:

<classloading xmlns="urn:jboss:classloading:1.0"
domain="pentaho.war"
parent-domain="pentaho.ear"
export-all="NON_EMPTY"
import-all="true">
</classloading>

The most important bit here is the pentaho.war is in it’s own domain “pentaho.war”, but delegates to the EAR’s CL domain, “pentaho.ear”.  This enables the WAR to see classes within jars deployed at the EAR level.  The pentaho.war in the case of an EAR deployment has no libs in WEB-INF/lib, so it must rely on the EAR for jars.

Result? Once I dropped these files in, the app “just worked”.  My app found my commons-collections.jar and not JBoss’s.

FYI, the most helpful article on the subject I found during this effort was http://java.dzone.com/articles/jboss-microcontainer-classloading

/phyto

Posted in Uncategorized | Tagged , , , , , , , , , , , | 30 Comments