Tree Walking, the art of navigating an object graph or relational model through the use of the Visitor (GoF) or any other arbitrary algorithm. The concept is relatively trivial, in that you start at a node in the interconnected system and then make navigation decisions based on some logic or rules. Most times, the structure of the tree is known a-priori and therefore the navigation is analogous to navigating a map.

Object To Relational mapping, or better known as ORM is one of the application that we can see Tree Walking in practice. For example, given a domain model which defines associations between entities can be instantiated to reveal a complex tree. Here is a typical example:

In the above example, we can see the domain model where the associations and cardinality of each entity is illustrated. Although the above UML represents classes, this is not an instance view. The instance view is significantly different and more complex to represent, given that there are some unbounded association properties of the model. Therefore, the representation of the instance view can be complex for the given model and is best left to the imagination.

Continuing with our ORM discussion, we can map the above model onto the database tables using metadata and achieve the ORM solution. Up till now, a majority of the implementation details have been inherited from the ORM tool selection, e.g. using annotations or XML to implement the metadata. What’s left now, is the answers to the “hard” questions, typically relating to resource management, performance views, dependency management, transaction management and a whole list of other Application Architecture concerns.

One particular Application Architecture concern that l have seen poorly defined and designed time and time again, is the handling of object graphs that are central to ORM. To further elaborate, object graphs are used all throughout the client and business tiers of an application. They can be constructed from the view and passed to the business tier for persistence, or can be generated from the business tier for consumption (i.e. rendering) by the web tier. The business tier, is where my arch nemesis, the Tree Walker often inhabits. For example, ORM implied constraints like lazy loading, can be a large enough force for some Application Architects to mandate a form of Tree Walking to mitigate N+1 selects problems, specifically to improve application performance. The choice seems obtuse to me, as the complexity associated to Tree Walkers often leads to architectural debt, which is more than not, remediated later in project maintenance cycles; mostly by means of the removal of the Tree Walker.

A recent sighting of a Tree Walker got me gasping as the unnecessary complexity it introduced into the persistence layer. After inspecting the anatomy of the Tree Walker, as with all my other Tree Walker sightings, l was able to dismiss it’s requirement in the solution very easily. This particular application architecture was simple, JSF, Hibernate and Spring all contained in a single WAR deployed in a Servlet container (i.e. no EJB). I was able to dismiss its requirement citing:

• Hibernate session is in scope of the View and does not need to a shortened connection lifecycle.
• Lazy collections can be hydrated from the database when the view is walking the object graph in the rendering phase of the request.
• Why not use already proven and mature Hibernate HQL to fetch the required object graph shape?

The main purpose of the Tree Walker in this instance, was to (a) optimise the database access by ensuring all associations in the object graph are lazy when first loaded into memory from the database and (b) reduce the time connections to the database is kept open; hence avoiding the Hibernate LazyIniitializationException.

Even if the database session was not in the scope of the View (i.e. in the business tier and behind an EJB), point 3 from the above list would still hold strong. This made me think about my previous blog entry on a similar topic; N+1 has leaked into my service interfaces. Does the Tree Walker stop the ORM constraints from leaking in to my service interfaces? Well, no. A developer still needs to define the directions for the Tree Walker which it will use to navigate the object graph. Although the service interfaces may appear to be more simple, the fact that you need to define a parameter to accept the directions for the Tree Walker means you have not avoided service interface pollution.

Time and time again, I often think what drives Application Architects, Designer and Developers to develop a solution that is, simply, not required. Is the art of simplicity something that can be learned? How can one reject thoughts that lead us to develop complex architectures when they are clearly not required? This brings me to a belief that I have held for a very long time; intelligence alone cannot buck the forces of over-engineering; it is wisdom that guides a truly skilled architect to a solution that’s both elegant and simple.

Recently, l encountered a very interesting fact about using Hibernate and EJB. Our design was simple, Container Managed Transactions and a remote interface provided by EJB, Spring wired business objects, and Hibernate as our data access layer. Simple and neat. Well, sort of.

It had been smooth sailing, we delivered a large number of application functionality and time came to integrate a third party application. We provided a specific integration API (through EJB) to abstract the complexity of our business processes. Our API was clean and simple to use. We spent a while “hooking up” the third party application and just as we we thought we were going to pull it off; Unmarshalling Exeptions. What the?

We looked for the usual candidates; had we packaged the incorrect version of our domain objects that we delivered to the vendor? Nope. Did we have an incorrect version of the domain objects? Nope. Well, what could it be? Looking a little closer at the exception stack trace revealed the truth; incompatible hibernate proxy objects! Oh no, hibernate is in my POJO.

A quick check of the vendor libraries revealed a truth we were hoping would not be true; the vendor was using hibernate as well, and no prizes for who can guess what the problem was; The vendor had a different version of hibernate.

This made me reflect on the “discussions” we’d had with Architecture about the use of Data Transfer Objects (DTO’s). An Object Oriented purist would strongly argue that DTO’s are, in fact, not objects as they have no behavior, and that DTO’s increase maintenance costs due to the impact they have on change. But, if your architecture has mandated isolated classloaders (EJB’s facades), one must pass “pure” objects from one classloader to the next. What does that mean? Its simple, the binary dependency should only be on the objects that are defined in the API and nothing more.

So, my next question was; where the hell is hibernate hiding in my domain objects? Collections. To be more specific; hibernate Collections. Due to the many pitfalls of ORM, hibernate has “solved” the problem by creating proxies for Collections of objects. This is the hibernate teams solution to the N+1 selects problem. Only for me, this particular aspect relating to ORM had leaked into our architecture and exposed a hole that was not covered. Now we have a submarine with a leaky hull.

I remember reading about Spring two years ago, and although Dependency Injection has been in my bag-of-tricks for a while now, l was excited about the semantics Spring provided when defining Java beans. It was simple, a few XML files scattered with a number of beans, and you were well on your way to making your application more testable, configurable and all the other wonderful things that using Spring provides.

Id been using it for about a year on small to medium sized projects with lots of success. Then came the monolith. I joined a large scale internet banking project which made such heavy use of Spring that the Application Context files had become a large aspect of the system. Sure, each component of the system was Spring wired to provide all the benefits of using DI, but overall, as each component depended on another, the Spring aspect became more and more pervasive. This made life very difficult for a lot of developers on the team. Let me explain why in a little more detail.

Each project in the Eclipse workspace was, loosely speaking, a component in the application. Typically, each project exposed some beans for other projects to consume. Each project had, on average, 5 spring XML files which defined the beans implemented in the component (project). If you wanted to use another bean from another project, you would; (a) add the project as a dependency in the ivy.xml file (we used Ivy as the dependency management tool), (b) import the specific spring application context xml files from the project you just added to the ivy.xml file and (c) wire up the imported bean to your component. Simple right?

Well, let me tell you, it wasn’t. The best way to describe this pattern is by giving an example of something thats familiar to all Java developers; JAR files. A single JAR file contains many classes. When you need a class or set of classes from a library, do you unzip the JAR and import a single class file into your project? The answer to this question (for all those Java newbies out there) is NO. You import the whole JAR and use what you need. What we were effectively doing was exactly that; picking a spring XML file(s) from a JAR file and importing only the ones we selected; trying to ignore all the other spring xml files in the library. As you can imagine, trying to determine the dependency of the spring file (even within the same JAR) is very difficult and error prone, especially when there is a complex dependency relationships in the bean definitions. This was a monolithic abstraction leakage; a leakage comparable to the Great Mississippi Flood of 1927.

After some time thinking about the problem, l realised that the answer to the problem was in the fundamentals of Object Oriented Principles; Abstraction. The solution turned out to be very simple. Each project was to present a single set of standard spring application contexts to the bean factory for loading. I named these standard files the “Public Interface Context”.

The solution was to create two files, the componets.xml and resources.xml files. The components.xml file contained all spring beans that are loosely defined as components of the project, e.g. services, helpers, processors etc. The resources.xml file contained beans for resource related objects, e.g. Hibernate Session Factories, JMS Connection Factories etc. Each of the projects that compiled to a JAR must have a /META-INF/components directory where the public interface context files are to be saved. These public interface context files imported referenced all the other spring xml files in the project; this time, through the public interface context xml files.

After the magic of Ivy, when all EAR’s are build, all the assembled JAR’s contain the public interface contexts are saved and available for import in a standard location: META-INF/components . Therefore, when constructing your bean factory, the application contexts are loaded like:

pubic class FooSingleton {
    private XmlBeanFactory beanFactory = null;
    private FooSingleton() {
        Enumeration compCtxtCfg =
            getClass().getClassLoader().getResources("META-INF/components/components.xml");

        Enumeration resrcCtxCfg =
            getClass().getClassLoader().getResources("META-INF/components/resources.xml");
            /* Loop through all the spring xml files and load bean factory. */

          }
}

Now, once you have build all your EAR’s, WAR’s and JAR’s, your application classapath will determine which spring application contexts will be loaded! The elegance of abstraction.

Lesson learned: Always question patterns that seem overly complicated and cumbersome.