How should enterprise architects go about finding problems within their SOA architecture?
It comes down to almost the same thing as finding any problem -- testing and monitoring. You need to have:
- monitoring throughout the application lifecycle from development to all testing phases and into your live system; and
- the ability to understand what normal and abnormal behavior looks like.
Any performance problem is always caused by a resource bottleneck of some kind, whether that's CPU utilization somewhere or disk utilization. It really comes down to working out what those bottlenecks are and then identifying what you can do to move or change them. There will always be another bottleneck; otherwise the application would still run infinitely fast.
The key thing about testing is to make sure the tests are not conducted as a pass or fail exercise.
The key thing about testing is to make sure the tests are not conducted as a pass or fail exercise. By having monitoring in place, you can look at how close things work to failing and what the hot components were in the system. A good example of poor testing is something I heard of when talking to a supermarket, which had done some testing on its Oracle service architecture. After about an hour of testing, the database administrators were telling us the SAN storage used for the database was at 98%. If it had gone on for another 15 minutes or so, they would've run out SAN storage, and the whole database would've come crashing down and not been able to write anything.
What that experience told them is that they can take about an hour and 15 minutes of that load before everything collapsed. The example shows testing and monitoring have to be done hand-in-hand in order to understand what's going on inside the system. There are various tools you can use for that and I tend to take a bottom-up approach. You start using operating system tools to look at CPU usage, disk usage and memory usage. Then, you can drill down using JVM tools to look at the JVM memory use and statistics.
Editor's note: This answer is excerpted from an interview with Matt Brasier, a professional services consultant, conducted by SearchSOA associate site editor Maxine Giza.
This was first published in September 2013