Along with Roy Schulte, K. Mani Chandy (left) ‘wrote the book’ on designing event-driven systems. Dr. Chandy has received multiple awards, including the CMG Michelson Award, the IEEE Kobayashi award and the Babbage Award, to name a few. He is also a member of the National Academy of Engineering. We talked with Chandy about event processing after the publication of his new book, Event Processing: Designing IT Systems for Agile Companies. In June 2010, SearchSOA.com Site Editor Jack Vaughan talked with Chandy for this issue of the BPM Quarterly E-Zine.
Could you explain event processing for us in a nutshell?
Yes well, there are four parts to it. One is you need to continuously acquire data and there are various ways in which this is done. Some of it is done through sensors. Companies and enterprises all over are finding that they have a lot more sensors in their manufacturing and logistics and they're generating a lot more statistics than they're actually using. So number one is the acquisition of data, and two is the fusion of data - correlating data from multiple sources of information. Once you fuse the data you have an idea of what the world is like outside and how it's changed. And so the third part is to plan. You want to plan a response to this change. And the fourth part is to execute the response. The combination of all four parts is what makes event processing very useful to the business.
What advice do you have for readers who are interested in implementing event processing?
What we recommend, especially for enterprises starting out, is taking a sequence of incremental steps and evaluating each step for its business benefits. One key step is often for key executives to get real-time or near real-time business performance indicators. The first part of that focuses on the very first of the four things I talked about and also partly on the second. Namely, the first being data acquisition and the second is very simple correlation in the sense of putting the data in the form that is most useful for an executive to see on a screen. This is sometimes called business activity monitoring or BAM.
Starting with this fairly small step, you make sure that it's useful in the sense that it helps the executive make better sense of what's going on and hopefully make better decisions. Once they see the benefit of that, then take the next step which is to do more real-time correlation— more real-time fusion. Start trying to detect more interesting patterns than just showing the executive a screen with the data in it. You can start really digesting it for the executive so that you provide much more actionable information.
You are advised to take small steps. Start really digesting it for the executive so that you provide much more actionable information.
Then take the latter steps after that. Move on to the planning step. I think you need to do these things incrementally, starting with asking yourself the question “What part of the Enterprise software stack do I already have that can help get me to the first step?” I think the hurdle that people imagine is that they've got to throw away their SOA systems and buy something completely new and very expensive. But they can get started with building on top of what they already have and move on to something more radical later, if necessary.
How do events play into a service-oriented architecture?
I want to answer the question in two parts. Services are often thought of as something invoked by the client. The client invokes a service and the service responds to the specific invocation of the client. The client says, “I want you to execute this service at these parameters.” Now, events just flip the whole thing around in the sense that what the client does is to specify a long-term profile. The client isn't requesting the invocation of a specific service.
The client is saying, “I'm interested in the following kinds of things and I will remain interested in them for the foreseeable future.” The contract between the client and the server for services is often one service. When the service is over, the contract is done, whereas the contract between the client and the server for event processing is usually long term and less specific. It's more about the collection of interest than the specific service in specific parameters. That's really the basis of the difference [between services and events].
When it comes to implementation, SOA is sometimes thought of as not only services but SOA itself, as Roy Schulte talks about it, SOA is the principles of modularity. And the principals of modularity are certainly just as applicable to events as they are to services. So in that sense, that SOA means modular principals, SOA is equally applicable to events as services.
And how does all this apply to the “real world”, so to speak?
Well, this is the stuff of life. It can even apply to something as basic as water, because you need sensors in water to make sure it's flowing right and that it's the right quality. There are increasing issues of contention over water rights and how water is distributed. All of that requires continuous monitoring and response.
It's also important in getting food “from farm to fork”, as they say. And in pharmaceuticals and drugs, there's also concern about ensuring that if, for example, drugs have to be withdrawn because the batch wasn't quite right, that they can withdraw those drugs quickly. It's important that you can sense that something went wrong and respond quickly.
There is also a lot of interest now in social networks and responding to rapid change within social networks. And there is a great deal of interest in analyzing text in news stories and blogs and again responding rapidly to any knowledge that is gained from this general text. So I think this is fairly the stuff of life. And that's what I think is going to be more important.
So when you look at the stuff of life, you see events?
Well I call it sense and response. Sensing and responding to the world around us. Events are a part of that.
Any final thoughts?
One more thing I would add is that now there is this event processing technical society. It's a group of corporations, both vendors and users of event processing, as well as academics and government representatives. It's called EPTS and it has several goals, one of which is to explain what event processing is, what it is not, and especially to give a balanced view of its strengths and weaknesses. And then do things like help to develop standards. We will talk about the kind of standards that one would like to develop. We won't suggest a particular standard, but will show how to get there.
[This material originally appeared in the SearchSOA BPM PDF ezine, Vol 1. #1, 2010.]
For more on CEP: updated and related event processing background books and docs
Event processing technical papers - DEI.Poimi.it (.PDF)
The event processing manifesto – Dagsthul.de (.PDF)
Book Excerpt: Event Processing: Designing IT Systems for Agile Companies – SearchSOA.com