Archive for October, 2007

Optimise your Web Tier using Sematic Markup

October 27th, 2007 by Oscar Huseyin

Asking anyone about the importance of getting your applications business tier design and implementation right will draw no objections. A well designed and implemented business tier increases the flexibility and maintainability of your application and shields it from complex and difficult refactoring imposed by changing requirements.

Over the years, l’ve seen well designed and implemented business tiers by the use common design patterns such as Command, Data Access Objects, Service Facade etc. If well implemented, these design patterns can increase the maintainability of the application and increase the business agility to evolving business requirements. From my experience, the same cannot be said about the View.

Ive seen a number of projects fail dismally in the view implementation where changes are typically complex and difficult to implement. Frameworks like Spring MVC and Struts are great to help organise application layers and apply constraints to help avoid abstraction leakage (like business logic bleed into the view) and provide a clean MVC framework, but is still not sufficient for Web Tier agility. The technology ecosystem for the Web Tier is vast with new and constantly evolving frameworks which increase the richness of the web experience. AJAX, JSP, Velocity, Tapestry, Java Server Faces are all excellent tools in implementing your view and all present a number of rich API’s to help in the rendering and behavior of the view.

That said, what I’ve found missing in Web Tier implementations is a consistent lack of organisation. So what do l mean by “organisation”? Using the business tier as an example, typically most database access detail are hidden behind a Data Access Object API. This way, as long as business services are coupled to the DAO API’s, then changes to the data abstraction code is localised in one area; the implementation of the Data Access Objects. This is the classic Separation Of Concerns. In my experience, concerns of the View have not always been adequately separated.

Most projects that l’ve worked on have not able to even articulate the View concerns. So, what are the concerns of the View? Well, simple. Markup, Style and Behavior. Separating these layers in the view is crucial to increase flexibility and maintainability of your view. Separating the Markup, Style and Behaviour of an application requires diligence and strict adherence to a few rules:

(1) - Ensure that your markup is limited to structural definitions and conforms to XHTML (Strict, Transactional or Frameset). The rule is to exclude any style or behaviour code. Here’s is an example of bad markup:

<html>
    <body style="background: #fff; alignment: left;" onclick="doSomeWork()">
        <ul style="list-style-type: none">
	    <li>A List Item</li>
        </ul>
    </body>
</html>

The above example mixes all of the View concerns. Firstly, there is style definitions in the <body> tag where the background is set to some arbitrary value and the alignment is set to left. Some style information has now leaked into the markup and will force the developer to search for and change each instance of the <body> when the application is being “re-skinned”. This same problem is repeated in the unsorted list definition, where the style of the list has been hard coded into to markup.

Secondly, the <body> tag has been defined to handle the onclick event from the document body. Similar to the first problem, refactoring the behaviour of the <body> will force the developer modify every <body> in the application when there is a change required in this area.

(2) - Ensure that Style is separated from the markup. This enables your applications view to be changed with minimal effort. When marking up the documents block elements, ensure that you use id and class attributes to decouple the markup from the style. This way, with the use of CSS selectors, style can be added to the markup and can be changed across the whole site using a single definition.

(3) - Ensure that Behaviour is separated from the markup. Once implemented, there should be only one JavaScript function call in your markup, which is typically <body onload="onload()"/>. Given that rich web view implementations require JavaScript, avoiding it is not practical, therefore attachment of events should be defined in a JavaScript file and called from the onload() call path.

/**
 * The onload() method that separates the markup from the behaviour
 */
function onload() {
    attachEvents();
}

/**
 * Attatchement of all events in the document
 */
function attachEvents() {
    document.getElementById("foo").attachEvent("onclick", bar);
    // Attatch all the events for document
}

/**
 * Event handlers.
 */
function bar() {
    doSomethingForOnClickEvent();
}

There you have it. Clean separation of all your View concerns. If you ensure that Markup, Style and Behaviour have been adequately separated, then you will have a more agile View, where skinning the application can be performed in far less time.

Posted in Patterns | 1 Comment »

N+1 has leaked into my service interfaces

October 11th, 2007 by Oscar Huseyin

About two years ago, l (like many others) bought and read Hibernate In Action. It was definitely the most decisive book on the Object to Relational Mapping tool that l could find. Apart from being really well written, the book was a great reference text which could be used in the trenches to configure and use Hibernate.

Now, a few years after using Hibernate in angst, l have realised the decision to use ORM has large drawbacks; larger than l initially envisioned. Notably, the infamous N+1 selects problem can cause increased development times as developers spend a large portion of time “ironing out” the performance issues related to the ORM implied constraints.

In first appearances, the N+1 problem presented performance issues that would often cripple the JVM for memory whist fetching and hydrating objects from the database. The solution seems trivial in that defining lazy associations would restrict the loading of object until they were needed, consequently restricting the number of objects in the object graph. However, this is not the be-all-and-end-all of problems relating to N+1.

Looking back, l had a sense of victory after we performed a first pass to detail the domain associations with a view to implement a more performant database abstraction. However, as the system functionality increased, the requirements on the domain model unfolded to create more and more “scenario based” object associations. To illustrate this point, let me give a Hello World! example. If my domain object Customer has associations to Order, Item and Address then my association could be represented as:

Customer  (1) ---------- (*) Order  (1) --------- (*) Item
                |
                --------------(*) Address

Now, if l don’t define any lazy associations, then when l load my Customer from the database, Hibernate will retrieve all Order’s, Item’s and Addresses. Adding lazy associations to each relationship, l can now control the loading of Order’s, Item’s and Addresses as l need them. Typically, this is achieved by “touching” the Collection that l need to load from the Customer. Simple right?

In my example, one possible “scenario” is the non-lazy one; e.g. when you load a Customer, all associated objects are also loaded. Another scenario is the lazy one, Customer’s with Order’s and no Addresses. I’ve described two scenarios here. Can you see anymore? I can. Customer with Orders only (e.g. no Item’s). Any more? Yep, Customer with Addresses only. l can go on and on. So, the number of possible object loading scenarios is a function of the number of associations in the domain model. Now, that can be a very big number! Exponential actually.

Given this aspect of ORM must be solved to increase the performance and scalability of the domain, how is this typically implemented? That’s the focus of this blog; N+1 leaking into the service methods.

To continue with the above example, if l create a service to retrieve Customer’s, l could (without considering the lazy associations) create a service named CusomterManager with a single method named getCustomer(int customerId). However, the service interface is certain to be non-performant as my Customer will be loaded with Order’s, Item’s and Addresses. Now, if l want to specialise my object graph that is returned, l need to add more methods to the CustomerManager service, getCustomerWithOrders(int customerId), getCustomerWithOrdersAndItems(int customerId), getCustomerWithOrdersItemsAndAddresses(int customerId), and so forth.

So, from a single method in my CustomerManager service to four! Thats what l call an abstraction leakage. My clients are now exposed to the shortcomings of ORM and l have severely polluted my service interface.

Avoiding this service pollution is not a concern of development. This responsibility rests squarely with the application architect. Constraints applied by application architecture are the primary cause of the abstraction leakage which were mandated by the use of, say EJB. Retrieving object graphs from Stateless Session Beans will directly present the ORM shortcomings for clients to deal with. However, services deployed in, say, the Servlet container will remove the need to pollute service interfaces, but create other issues like holding onto resources (such as a database connection) for lengthy periods of time to allow the service implementations to “retrieve” the lazily loaded associations as needed.

In conclusion, living with Hibernate is costly. The semantic definitions of your interfaces will resemble the object graphs that are being fetched and returned. Ive found this to be really messy and will force unwanted constrains on otherwise simple service definitions.

Posted in Architecture, Hibernate | 9 Comments »

Dynamic LDAP group membership; increasing the flexibility of your security model

October 3rd, 2007 by Oscar Huseyin

How’s this for a problem; having users that can switch between security context in a J2EE construct and maintain a linear number of user to group associations in LDAP. What does this mean? To put it simply, if l’m user Foo and l can (from a security perspective) take on the exclusive roles of an administrator, accountant, auditor respectively, this must ultimately equate to group memberships; more specifically LDAP group memberships. This problem is not solved by simply assigning the user Foo into all the groups, because then Foo would have access to an auditor’s functions after he logged in as an accountant, and not all accountant’s are auditors. If we were to attempt a solution modeling the permissions using LDAP, we would have to create users Foo, Foo' and Foo'' where users are given group memberships to administrator, accountant and auditor respectively. From an LDAP perspective this solution is unmanageable, because as the requirement for users permissioning levels change, then the number of unique user to group combinations would exponentially increase, resulting in unmanageable LDAP system.

A simple solution to this problem is in the use of dynamic groups. One small problem though, a dynamic group is only a concept and, technically speaking, cannot be modeled in OpenLDAP, eTrust and IBM Directory Server. Therefore, dynamic groups functionality must be built into the application security architecture.

Firstly, all the defined groups must be defined a-priori and stored in an LDAP server, eg. cn=mycompany-group-banking, ou=groups,ou=policies,o=mycompany. All users in the system are not assigned any group memberships in LDAP itself which means, form an LDAP perspective, they dont have any group memberships. Now, to achieve dynamic groups, the “system access profile” needs to be read from a store (which is typically the application database as the user to profile relationships are highly normalised) and then used to build the users security credential. For example, the database is queried to read the user group memberships (which map one-to-one with the LDAP group) and are then used to construct the platform specific security credential; the JAAS Subject in a J2EE construct.

Confused? Perhaps it time for some concrete examples. WebSphere Application Server has an interface which can be configured to allow credential construction from a “trusted” source. This mechanism is called the Trust Association Interceptor (TAI). You could also use a custom Login Module but l find them a little cumbersome to work with. IBM Developerworks has a great article on this exact topic. If you want to assert the identify of a user who has not been authenticated using WebSphere, you can configure the TAI so that it fires when a client attempts to access a secured resource. At this time, the TAI has the ability to construct a security credential and assert the identity to WAS.

/*
 * Trust Authentication Interceptor interface method that will called by WebSphere.
 */
public TAIResult negotiateValidateandEstablishTrust(
                            HttpServletRequest req,
                            HttpServletResponse resp) throws WebTrustAssociationFailedException {
    String userid = req.getParameter("userid");

    String password = req.getParameter("password");

    try {
        // Compute identity information
        InitialContext ctx = new InitialContext();

        UserRegistry reg = (UserRegistry) ctx.lookup("UserRegistry");

        // Check the users identity in LDAP, eg. course grained authentication
        boolean validUser = authenticateUsingLDAP(userid, password, reg);

        // Load users group memberships from the database

        if (validUser) {
            Collection groups = readGroupsFromDatabase(userid);
        } else {
            throw new UnauthenticatedException("User could not be authenticated.");
        }

        // Having read the groups from the database for the authenticated user
        Subject subject = constructSubject(groups);

        // Now, assert the identity to WebSphere Application Server
        return TAIResult.create(HttpServletResponse.SC_OK, "notused", subject);

    } catch (SomeException e) {
        Log.ingo("Exception caught whilst attempting to authenticate user");
    }
}

Now that we have a security credential, for all invocations to secured EJB’s, Servlet’s, JSP’s, etc. WebSphere will automatically propagate the Subject from the authenticated caller to the callee. For example, when an EJB is invoked in another WAS instance in the Cell, the WebSphere EJB container (on the callee side) will intercept the EJB call, check the presence of the LTPA token for the call, check the local WAS Security Store for the (potentially) cached Subject, then make a call to retrieve the Subject (via JMX) if the Subject is not present in the Security Store to the caller WebSphere.

Constructing the subject is also simple. The key to the design, as mentioned before, is to ensure your user group memberships are in the application domain and not in LDAP. Here’s an example:

private Subject constructSubject(String userId, Collection groups) {
    // Create the JAAS Subject
    Subject subject = new Subject();

    // Data structure to store credentials
    Hashtable hashtable = new Hashtable();
    hashtable.put(AttributeNameConstants.WSCREDENTIAL_UNIQUEID, userId + System.currentTimeInMillis());
    hashtable.put(AttributeNameConstants.WSCREDENTIAL_SECURITYNAME, userId);

    // This is the point which implements the Dynamic Groups.  WebSphere reads the
    // group memberships using the WSCREDENTIAL_GROUPS key.
    hashtable.put(AttributeNameConstants.WSCREDENTIAL_GROUPS, groups);

    // Add all credential information to the Subject to complete Subject construction
    subject.getPublicCredentials().add(hashtable);

    return subject;
}

One point to be careful about. The groups that you insert into the public credential set of the Subject should exist and match the ones in LDAP. An example of a group is:


cn=mycompany-group-accounting, ou=groups,ou=policies,o=mycompany

Posted in Security | 1 Comment »