Building a “SOApp” with Mule (Open Source ESB)

Ok, I know that we're all up to our ears with technical acronyms, terms, buzzwords, etc. that all appear to have some marketing spin to them in hope that they will eventually work their way into industry acceptance. So, why should this be any different...just kidding, kind of :) Anyway, bear with me through this first paragraph and you'll understand how this “SOApp” thing came into existence in my little world. In 2007, I was fortunate enough to be part of a development team at Chariot Solutions responsible for designing and developing a product for the Material Handling and Distribution Industry. After a few requirements gathering sessions with the client, we discovered that the project had a number of integration points with new and existing systems...the age old integration challenge. The existing systems ranged from a COTS warehouse management system to a C-based controller that interfaced with physical hardware devices (e.g. lights, switches, buttons, etc.). Steve Smith (Chariot Architect) was involved early on in the initial discovery sessions and was quick to point out that all these integration points (files, database tables, TCP messages, etc.) we were faced with are very similar to challenges faced while implementing an SOA but on a much smaller scale. So, why not take advantage of the benefits that an ESB provides (i.e. protocol translation, message transformation and routing) to speed development and deliver a maintainable solution? Hence the “SOApp” concept was born. Well, the term “SOApp” (Service Oriented Application) was a play on the SOA acronym about 3 months ago when I was considering writing up a post (this post) to discuss a unique way of using ESB technology. A close second was “Save Our A...”, well you get the picture, anyway, back to the interesting technical stuff.

Ok, so we had a conceptual architecture centered around an ESB. The team was all in agreement that we would benefit from NOT having to write custom integration code and simply focus on developing business logic in POJOs. The decision to use Mule (v. 1.4.1) for the ESB was an easy choice for us because we had plenty of experience using it in our SOA Lab. There were many interesting aspects of the project however, this post focuses on how we dealt with the various integration challenges. Let's take a closer look. The diagram below illustrates what the architecture looks like with a focus on various system interfaces.



Warehouse Management System (WMS) – File-based Interface
The WMS generated flat files in a proprietary format on some random schedule. This data is essential to the overall operation of the entire system. These flat files needed to be processed as soon as the would appear in a directory. A Mule file connector was configured to poll a directory, pickup a file, pass the data onto a POJO for processing and eventually update the PostgreSQL database via Hibernate. Nothing fancy here just basic file processing.

PC Controller – TCP Sockets Interface
This controller serves as a central integration point for various switches, sensors, etc. A basic use case was when a sensor was triggered, the controller would send a proprietary delimited length text message was sent to a particular POJO handling that transaction in Mule. The POJO would typically query the PostgreSQL database and return the resulting data back to the controller. There was a non-functional requirement for the entire transaction to take less 200 ms. At first, we used a Mule TCP connector to communicate (both inbound and outbound) with the controller however, due to some limitations on the controller side (and time constraints) we had to switch outbound communications to custom socket code. Here was a case where we only benefited from using Mule's connectors for one communications direction however, since the socket code was easy enough to write it didn't take long to develop a workaround. Oh, the transaction performance ended up to be less than 100ms on average.

Pick-To-Light Controller – JDBC Connector Interface
This controller was a java-based controller that interfaced physical devices via the PostgreSQL database. Though this is not the “ideal” method for interfacing it was a constraint that we had to deal with. What was interesting about this interface is that we configured a Mule JDBC connector to process incoming requests when information appeared in the database tables. The POJOs that processed the data then used our Hibernate persistence layer to respond back to the controller via database tables.

Java Swing Clients – MuleClient / TCP Interface
There were two Java Swing clients that we developed for administrators and end users to interface with the system. These swing clients used the MuleClient to simplify the TCP communications with Mule. Each POJO was considered a “service” with a unique endpoint deployed in Mule. What is interesting about this is that the swing client only had to specify a single uri (e.g. tcp://localhost:) along with a payload and let the MuleClient handle the rest. There was no need to identify a specific endpoint in the URI, which made this design very flexible and maintainable. Here we leveraged the content-based routing capabilities of Mule by configuring various FilteringOutboundRouter's to call internal POJOs for processing. The outbound routers inspect the payload type and routed accordingly based on configuration.

CLI (Command Line Interface) Java Application – Telnet & MuleClient / TCP Interface
Well I saved the most interesting interface for last, IMHO :) One of the requirements was to have a handheld Windows CE device, capable of scanning barcodes, perform CRUD operations on the PostgreSQL database over a wireless 802.11b network. We had part of this problem solved by using the MuleClient to communicate with our POJOs deployed in Mule. Ok, so, how do we implement the MuleClient on a Windows CE device? Well, John Shepard (Chariot Architect) came up with a clever design to use Telnet and a basic Java application to solve the problem. This was a great example of leveraging proven technology along with a creative way of configuring Unix accounts. The end user accounts were configured to start the CLI java application after a successful login to the Unix machine instead of a bash shell. This work very well and performed beautifully. There was no need to install and manage client applications or worry about performance of the handheld device. The built-in Telnet capability of the Windows CE device was all that was needed on the client. This was very cool to see in action especially knowing that only a basic java application was behind it.

The purpose of this post was to give you an overview of how to use an ESB in new and creative ways aside from their original intent. In the future, I may expand more on these interfaces to provide more detail. There were a number of other interesting aspects to the project like using annotations and XDoclet to automatically generate the Mule configuration, etc. that is good material for future posts. Wrapping this up, we did take on some risk of using Mule in this capacity, however, luckily there were no major problems we ran into. Besides we always had Java to fall back on if Mule came up short anywhere. Could this be the beginning of a new movement in application development...the “SOApp”? Maybe, however I can already hear the snickering of my colleges as they read the end of this post :)