Like a Swiss Army Knife with many blades and tools, the enterprise service bus (ESB) is many things to many people....
It is easy to see why. Within the ESB arsenal is an array of middleware tools. These can support SOA or EAI approaches to integration. Used unwisely with either SOA or EAI, ESBs can bring performance liabilities. ESBs seem to need someone to tell them to stand up and reveal their true identity.
Ken Johnson, Middleware product manager at Red Hat, which makes JBoss Enterprise Middleware, says many people think of an ESB as simply a piece of software, although it is better to view it as “a way to architect a solution.” Furthermore, despite the “service” in the name, ESBs aren’t always used with services. ESBs can assume a variety of functions that one might not expect. For instance, Rob Davies, CTO of FuseSource and co-founder of Apache ServiceMix, ActiveMQ and Camel, says the difference between ESB and EAI is primarily in their use cases. “Under the covers, ESBs can communicate to each other using a messaging layer,” he says.
For high throughput: ''Keep the message structure as simple as possible and the payload light,"
Simon Bilton, GodelTech
Neeraj Satija, Software Development Manager, at Two Degrees Mobile Limited, a customer of WSO2, another player in the ESB space, describes an ESB usage pattern that has become integral to IT. It is a bus “that exists right in the middle of front-end channels [such as Portals, IVR and SMS] as well as back-end systems [such as a CRM system, network elements, and billings and recharge [systems].”
Satija says at his current organization, as well as previously, he has used ESBs from vendors such as IBM, Oracle and Mule. Now, WSO2 is the platform his company’s SOA is built on. “We have started using a significant part of the WSO2 portfolio of middleware technologies. A major portion of our IT architecture is heavily reliant on the SOA paradigm and I believe we are doing some exceptional, innovative work because of this architecture,” he says. And, he adds, his company doesn’t use EAI. It does use SOAP.
“Ours is a typical SOA platform and both the service consumer [channels] and service provider [back-end systems] talk to each other in XML-based protocols,” he explains. In fact, at Two Degrees Mobile, different transactions are fulfilled using different messaging patterns, such as synchronous request-response, asynchronous request-response, publish-subscribe or event-listeners -- with the business needs determining the pattern selection, explains Satija.
Satija says he has found a number of patterns and practices that can be useful in employing an ESB, such asWSO2, including:
- Service abstraction (use of ESB proxies to decouple the actual end point)
- Standardized service contracts
- Data format transformation
- Protocol conversion
- Business logic orchestration using a workflow engine
- Data format transformation and data model transformation.
Although some have expressed concern about ESB throughput, Satija says Two Degrees Mobile has used WSO2 components such as the ESB and the Workflow engine in some of its busiest transactions – some of which clock up to 50 transactions per second during peak hours. In general, he recommends three “best practices” to achieve good throughput:
- Payload size and content optimization
- ESB and workflow manager federation, based on business requirements
- Preference to develop light-weight atomic middleware components over bulky applications that contains business logic
Also speaking to performance challenges, Robin Bloor, Ph.D., chief analyst & co-founder, Bloor Research, says when it comes to throughput/performance, ESBs tend to scale up to a certain level, but not all of them scale out well.
“It depends on how well they do distributed processing,” he adds. Most SOA implementations involve an ESB because of the heavy messaging load and the need to recover from failures, he explains. Furthermore, it is difficult to achieve a high level of reuse of applications/services without an ESB.
Where they are used for EAI they tend to be used when point-to-point integration is insufficient. “You can do point-to-point EAI without an ESB by hard wiring, but when it gets complex enough to mandate a hub and spoke architecture, ESBs tend to be used as the hub,” says Bloor.
“The ESB is just a pipe and hopefully it's a high performance pipe, so latency isn't likely to be an issue unless the ESB is inadequately resourced,” he adds.
Meanwhile, another WSO2 customer offers a slightly different view of the role ESBs are currently playing. “We find that there is a fine distinction between a bus-based EAI architecture and true ESB/SOA solution,” notes Simon Bilton at GodelTech a-UK based software development company.
Colleagues Vladimir Klevko and Alex Karimov echo that sentiment and point out that historically, enterprises have taken what appears on the surface to be a logical approach to linking disparate applications or services by implementing a central transportation bus mechanism that allows these applications or services to deliver or receive data without direct point-to-point connectors.
However, in general, these central buses have largely been proprietary and each application or service has had to have an integration engine written for it. “While this may deliver a solution to the individual enterprise in question, it doesn’t easily allow connectivity to additional services – say an external customer’s service – without yet another bespoke integration piece to be written,” says Bilton. So, he explains, “We believe that these implementations do not meet one of the key features of SOA, that of de-coupling services.”
On the other hand, an ESB uses open standards and the integration points are defined in “contracts” (such as WSDLs). And although each application or service still needs an appropriate adapter to be developed, this adapter, he notes, must meet the definition in the contract and can then plug straight in to any similar standards-based ESB. Furthermore, this allows easy integration with additional services – be they internal or external – with no redevelopment costs.
“What we at GodelTech have seen...in many cases, is that enterprises have deployed an ESB product as the transportation mechanism for an EAI architecture. [Thus,] eliminating the proprietary bus, [and] proxying services between end points,” says Bilton. However, Bilton and his colleagues say this is really a “halfway house,” since the services are not necessarily fully reusable, not necessarily de-coupled and therefore do not deliver a true SOA solution based on core SOA principles.
Bilton and his colleagues also offer some advice for those harnessing ESBs for messaging: keep the message structure as simple as possible and the payload light, because that is the most efficient way to achieve high throughput.
“In SOA, a message does not expect to receive a response, but it may expect to receive a reaction, maybe as a result of an event being triggered by the originating message,” notes Bilton. Typically, each of these message exchanges are executed asynchronously and in isolation, and therefore do not constitute a conversation.
SOA doesn’t necessarily lend itself to transaction-based processing, he asserts, but is well suited to event-based activities. Complex event processors are becoming more prevalent to control these actions, and to allow the normally asynchronous, stateless message exchanges to approach more transactional characteristics, he adds.