Software development pros know that testing is the critical transition between development and deployment, and
Application testing is typically a three-level process. First, components of the application are "unit tested" by developers to ensure they function according to their local specifications. Second, these components are integrated to complete the application and then "system tested" or "integration tested" to ensure that workflow among applications and functional relationships perform as expected. Finally, the applications are "load tested" or "pilot tested" to simulate actual deployment conditions. SOA applications can follow this same model, but with special provisions in each phase to accommodate the specific nature of loosely coupled component-based applications.
Loose coupling in SOA demands that application components function the way "as-a-service" components do by presenting themselves almost as complete applications. It is critical to treat SOA components as applications for purposes of unit testing, which means combining the three test phases above and executing them on each SOA service/component.
Service interfaces are the key to SOA component testing. Be sure that unit testing exercises the components in their true service form, via the SOAP/WSDL that will describe the service in production. Users report that basic software tests often miss the service interface, leaving it for later integration testing, and this delays the project by allowing issues with service specification to accumulate until everything is assembled. Proper service interface testing often requires a test generator operating through the SOA/SOAP interface to ensure that all the possible conditions are tested.
The service interfaces are the key to SOA component testing.
At this point, add basic load testing to the validation to be sure the component's performance meets overall goals. Testing the performance of a whole string of orchestrated SOA components is more complicated since all possible paths may be harder to verify. If each component is load-tested individually, then the performance of the application can be predicted under various situations by plotting the workflows and summing the delays for each path. This then serves as a benchmark to compare against actual load-testing results.
With SOA, integration testing has three goals, not one. SOA applications differ from ordinary applications in that they normally introduce a new component in the orchestration software, the message/service bus technology employed to link SOA components. In many SOA applications, message management among components is controlled by a Business Process Execution Language (BPEL) template, and this is also a new element to be tested. Only when both these "new" elements have been tested can you test normal inter-component integration. The fact that "integration" testing tests more than the components that were previously unit-tested shows why it's important to raise the standards for component validation as outlined earlier in this article.
Most SOA applications have their BPEL at least prototyped during early application development by the application architects. If the basic main-path BPEL and associated test data are available from this process and from unit testing, they can be used to validate the functionality of the message/service bus orchestration software. It's important to insure that the BPEL correctly drives component sequencing and that the interfaces between the message/service bus software and the components are correct. This is also where component directory access and SOAP message formats can be re-tested; they should have been validated at the basic level in the unit testing process. Secondary logic paths can be tested through this same BPEL-and-component-sequencing test when the primary test is complete.
Load testing in SOA applications is also complicated by the message/service bus and BPEL introduction because performance of these elements will impact application quality of experience (QoE). Testing a fully orchestrated SOA application for performance under load is more likely complicated by component placement relative to the points of worker support. Unless all SOA components are hosted in a common data center with a short network connection path, QoE is likely to vary somewhat depending on the location of the worker.
Multi-location data injection is the only reliable way to test any network-connected application and it's especially important with SOA applications. The challenge in this model of testing is to correlate problems with causes, since timing of events generated from multiple points in the network are difficult to repeat to help with problem isolation. Monitoring of the test flows with accurate event time-stamps is essential. Users report that relying on data logs can be problematic since logging impacts application performance and also the timing of event-handling.
Users committed to Agile development practices aimed at commissioning new code faster to respond to bugs or changes may find SOA both helpful and a challenge. Because components are more isolated in SOA, it may be possible to commission a new version of a component quickly and redeploy it without repeating major portions of integration and load testing. However, interactivity among SOA components is naturally higher and so testing SOA-level integration for redeployments may be time-consuming.
The best strategy for SOA redeployment testing is to design for very structured component interfaces and message flows, avoiding free-form parameters and loosely bound of variables. Let components validate their parameters thoroughly and/or have WSDL schemas express parameter limits explicitly. If individual components can be made "self-integrating," then the need for wider testing on redeployment is reduced and the speed of commissioning changes and bug fixes is increased.
SOA testing is a special case of integrated application testing, special because SOA typically deploys more components and special because workflow exchanges are at least potentially more structured. By taking advantage of structure, it's possible to reduce the risks created by extensive componentization and to test thoroughly without testing excessively.
Follow us on Twitter at @SearchSOA.
This was first published in March 2013