Web services can reduce risk
My attention was drawn to a recent UK news item that reported on the enormous task facing the National Health Service and its IT systems. The background is that the NHS is a state controlled institution that has been progressively failing over many years and administrations. The UK government has committed an act of faith to inject vast amounts of money into the service, yet there is widespread debate and disagreement on the fundamental practices and systems.
A respected neurological surgeon Mr. Marios Papadopoulos recently made the comment (reported on the BBC) that in a mathematical sense, the service is on "the edge of chaos, neither ordered, nor completely chaotic". As such, he says, "it is highly resistant to change" - meaning that even big changes in investment may have little discernable effect.
High probability of project failure
The news item that caught my attention referred to the vast amounts of investment being committed, and caused me to speculate on whether the money would be spent wisely and effectively. My research showed the UK public sector has a poor track record with IT projects. Let's look at a few:
- Benefits Agency and Post Office Benefit Payments Project - abandoned after 3 years and USD1.5bn
- Lord Chancellors Case Processing Project - core case working software failure, cost USD267m
- Various Inland Revenue projects - cost over run of USD 2.1bn
- Classified Ministry of Defence project - abandoned with security and compatibility problems at a cost of USD60m
- Swanwick Air Traffic Control Centre - USD500m cost over run, twice original budget, 6 years overdue, and now perceived as not meeting the needs of the near term, yet alone its envisaged 30 year life.
The Swanwick project contains some important lessons for us. According to the BBC, plans for a new Air Traffic Control system were originally laid in early 1990 in response to the seemingly inexorable growth in holiday and business flights. The decision was taken to start from scratch. The two existing operations would be brought under one roof to form the biggest purpose-built air traffic control centre in the world. Although the package would be based on one that was already being developed for the United States, it meant designing new hardware and writing wholly original software. With hindsight, an independent air traffic control expert Philip Butterworth-Hayes commented, this was probably the wrong tack. "The basic stumbling block was not to get off-the-shelf components and software. When you plan a new operation like this, you want it to be good for the next 20 to 25 years. But software is advancing at a tremendous pace, so it becomes obsolete every 18 months."
Across the ten year development period, the project management and accountability changed hands three times. Since implementation things have not gone smoothly. There have been three major disruptions in two months to UK air traffic control impacting tens of thousands of passengers as a direct result of system malfunction.
All of these projects were run by well known multinational systems integrators and services companies.
Though these cases focus on the UK, sadly these cases merely confirm experience around the world that large scale projects have a very low probability of success. It seems that with large scale projects in private or public sectors, the sheer complexity of the task is often beyond the capability of the business people to specify and manage and the technical project team to implement.
The Service Bus - a radical change in project approach
Many people and organizations have addressed this issue of project failure. There is much excellent work for example from the Software Engineering Institute, that provides guidance on process and practice which has been shown to deliver good results, albeit with significant investments of time and money.
Over two years ago I wrote a report which addressed this issue from a different perspective. I introduced the concept of the Business Service Bus, which is analogous to a middleware bus, but defining sets of services that comply with common semantics and can be managed together. I suggested that the Service Bus is a great way to introduce new functionality on a progressive basis, particularly where there is great complexity in both existing application portfolios and ownership structures. The service concept is sharply differentiated from prior middleware strategies because the functionality being provided and consumed offers a complete business function. And because the Service Bus is implemented on Web service standards (SOAP, WSDL) the middleware issues are massively simplified.
The key point is that EDI and integration are focused on coexistence and data exchange. This is fine if it is addressing a tactical requirement. But for major replacement and upgrade projects you need a different approach that allows complex systems to incorporate new functionality on a progressive and unplanned basis. Unplanned? Yes, because over the life of a major system the functionality needs to be able to evolve on a constant basis. The service being closely aligned with process usage, is an ideal unit of scope and life cycle management.
Remember wrap and replace
Once again we can learn from component theory. (No apologies here, good services are simply components but with standards based interop and published meta data). Remember the wrap and replace theory? This has been used extensively throughout the industry, and has been proved as a practical approach to progressive, multi release upgrade of complex applications. Expose and publish the API, upgrade consumers to the new standard, then progressively upgrade the supporting functionality in a manner that is transparent to the consumer.
Now this is equally applicable to the provision of common services into a complex marketplace, where providing organizations can offer standard services based on agreed XML schemas, and consuming organizations can plan to acquire services that do not need to be organization specific, by upgrading to the common API.
Application rationalization not messaging
I looked at the NHS public material and was pleased to see that there is an XML strategy in place. However I was disappointed to see that the (apparent) direction was focused on XML and Web services as a better form of EDI. Just think about it. The NHS has hundreds of hospitals and works with thousands of general practices. Now the opportunities for common services are going to be huge. But in this day and age we all understand the difficulties and risks of application rationalization. A better strategy is to rationalize at the business service level, or better still with sets of services, that are designed to operate collaboratively with others on the same bus structure.
I was talking to Microsoft people last week and aside from a general strategic briefing on .NET, which I will be reporting on in the upcoming CBDI Journal, I asked them about how they are moving to integrate and rationalize the two recently acquired CRM applications, Great Plains and Navision. These applications are very large, complex and naturally deployed across huge and disparate customer bases. The answer Microsoft gave was obvious - they are exposing services which rationalize the underlying functionality on a progressive basis. This process will take several years and at least several releases of the core applications, and will involve some considerable upgrading and restructuring of the underlying applications. However by using the service approach the impacts on the customer base can be minimized.
We discussed the strategy and I was intrigued to hear that they are actually taking a strongly tactical approach. No time for grand visions here! What they have is an overall and consistent approach, which is of course dictated by the platform to a significant extent, which is then implemented progressively in accordance with specific release functionality priorities.
Now the NHS is without question much more complex than the Microsoft CRM situation. But the same techniques apply. If you have a chaotic system you need to introduce order gradually, and a sensible way to do this is by picking off common services one by one. In that way you get immediate feedback, you don't waste huge efforts on infrastructure, and you can ensure that you don't waste millions of billions before you realize it. THIS is the new RAD.
CODA: It's not always a technology problem
As a practitioner I often came across the issue that project failure was caused by a fundamental conflict between the commercial and technical project systems being used. In a public situation such as the NHS, this is highly likely to prevail. In this case the Service Bus approach provides an interesting way of creating a tighter scoping, contracting, delivery and measurement system that ought to be attractive to the provider and the consumer.
Please send your feedback to firstname.lastname@example.org.
Copyright CBDi Forum Limited 2002. The CBDi Forum is an analysis firm and think tank, providing insight on component and Web service technologies, processes and practices for the software industry and its customers. To register for the weekly newswire click here.
For More Information:
- Looking for free research? Browse our comprehensive White Papers section by topic, author or keyword.
- Are you tired of technospeak? The Web services Advisor column uses plain talk and avoids the hype.
- For insightful opinion and commentary from today's industry leaders, read our Guest Commentary columns.
- Hey Codeheads! Start benefiting from these time-saving XML Developer Tips and .NET Developer Tips.
- Visit our huge Best Web Links for Web services collection for the freshest editor-selected resources.
- Visit Ask the Experts for answers to your Web services, SOAP, WSDL, XML, .NET, Java and EAI questions.
- Discuss this article, voice your opinion or talk with your peers in the SearchWebservices Discussion Forums.