The relationship of complex event processing (CEP) to service-oriented architecture (SOA) remains, in a word,
Vendors, including two making CEP related announcements today, are taking different approaches, standards are yet to emerge, and at least one analyst argues that it all could be simpler. Oracle Corp., announcing a new version of its Business Process Analysis Suite designed to allow tech savvy business users to have direct input into SOA, continues to champion event-driven architecture (EDA). Coral8 Inc., a pure play CEP vendor announcing the formation of a Developer Council for CEP advancement today, has developed its own language for event processing. Meanwhile, Software AG, which will have announcements on the merger of its Infravio and CentraSite governance products at the end of this month, is working in its labs on a neural network approach that focuses more on correlation of events.
When Jason Bloomberg, senior analyst ZapThink LLC, was asked about the status of CEP in SOA, he began by saying: "The relationship between complex event processing and SOA is a loaded question, because CEP is often associated with EDA, as though EDA were an alternative to SOA. However, our perspective is that SOA should fundamentally be event-driven, so there's no particular reason to distinguish SOA from EDA."
In Bloomberg's view the event-driven nature of SOA is hammered home by CEP.
"In SOA, services communicate by sending and/or receiving messages, and messages are essentially software events," he said. "That is how the system reflects a business event. So in a fully realized SOA implementation, the traffic you'd expect to see on the network will consist of services and service consumers madly exchanging messages -- or in other words, large numbers of events in what you might call an event cloud. Furthermore, those messages contain all the information you'd ever want about the behavior and state of your business, if only you could make sense of them."
Bloomberg argues that CEP's role in SOA is extracting intelligence from the event clouds in real-time.
"The broader your SOA implementation, the more effective your CEP can be, because an increasing percentage of meaningful traffic will consist of such messages/events," he said. "Another way of putting this is that SOA enables deep, real-time business intelligence and CEP is the technical mechanism for realizing this benefit."
However, Coral8 president and CTO Mark Tsimelzon argues that to provide real-time business intelligence CEP needs to go beyond SOAP messages.
"Traditional SOA applications exchange either XML documents or they exchange SOAP calls," the Coral8 executive said. "CEP is a way to do very different applications, applications that arise when you have many different events, including financial services transactions or events from RFID readers or on large Web sites where users are generating events."Tsimelzon said that for event intensive applications there is too much overhead in SOAP-based SOA.
"If you have a SOAP call for each event all you'll be doing is parsing those SOAP messages," he said. "You need to have a different set of standards to exchange events in those applications.
CEP requires a whole different language and standards to compute business metrics in real time or compute running averages, or track different processes in real time, Tsimelzon said. To this end, Coral8 has developed a CEP programming language called Continuous Computation Language (CCL), which Tsimelzon said is designed to program applications that handle continuous flows of events.
Despite the analyst criticism, Amlan Debnath, vice president of server technologies at Oracle, is championing EDA. He said within EDA, CEP needs its own standards, which Oracle and other vendors are in the early stages of formulating with the help of computer scientists from the academic world. The specifications they develop will eventually be turned over to a standards body for final codification, he said.
But Oracle is not waiting for standards to advance its vision of EDA. The new Business Process Analysis tools the company is announcing today allow business users to design SOA implementations that will provide them with the events data they need for analysis of business trends, he said.
"The BPA portion that we are announcing is responsible for the design of the business processes and linking the high level business models into the SOA and EDA systems," he said. "The model gets created and sent to the execution system, which is a combination of SOA and EDA. It is able to implement that design and execute that design. When it is in execution mode the system is generating events in the complex event processors and the BAM [business activity monitoring] systems are able to get those events, process them, analyze them and create high-level intelligence.
Peter Kürpick, president and chief product officer for the webMethods Business Line at Software AG, is looking at yet another way to handle event processing through neural networks that would identify event correlations to provide business intelligence. He said he has spent "quite some time" with analysts looking into the best way to do CEP in SOA. He is skeptical of CEP engines because he argues that regardless of how thorough the business users are in setting the criteria, the rules may miss a correlation in the data that could be important.
"I see two ways to approach this," he said of CEP. "One is you capture events and then you have this black box in the middle that in a rule-based way will correlate events, which is pretty much a big if-then-else machinery and is almost a business logic machine. The downside of such an approach is what if your rule doesn't capture a correlation because it's fully deterministic? If the rule doesn't capture the correlation then you just don't see that correlation."
So Software AG is working on neural networks that would be less deterministic and theoretically more capable of identifying correlations in data that business users might not even anticipate when setting up a rules-based systems.
This is not an approach for the general CEP use case which is financial systems handling millions of transactions, but is more aimed at providing information that can be analyzed for market trends. It is in theory a system that can think outside the box.
"Of course, you can set thresholds for correlations," Kürpick said, "but the system over time would determine correlations of events. This would be especially helpful where you are working with large numbers of events. So we're already working on a prototype in the lab that allows you to view complex events basically based on neural networks."
Asked for an example of how this approach would turn events into business intelligence, Kürpick said the classic one is pre-Christmas sales analysis. The neural network-based CEP system would detect high sales volume for a product by early shoppers trying to beat the Yuletide buying frenzy. That event would tell sales and marketing people that this was a product that will do well in December so they had better get the shelves stocked.
This is not a futuristic scenario Kürpick explained. "On a time axis you can already, today with neural networks, identify patterns," he said. "Typically, if sales of a gift item would go up because everybody wants to grab it before Christmas. It automatically generate a pattern of buying behavior."
However, if you want a unified theory of CEP for Christmas, good luck.