|
Quite frankly, I don't put a lot of stock in vendor-provided benchmarks. I prefer to do my own or to use third party tools. Most of the time different vendors will be better tuned for different tasks. If you have time, it can be useful to test your application on different platforms.
Fortunately, Sun, through the Java Community Process has come up with a
performance and scalability benchmarking kit called ECperf (http://java.sun.com/j2ee/ecperf/index.html). It should be disclosed that they too are an EJB vendor. However, they have released the specification for the benchmark, so you can analyze if it is adaquate for your project.
This brings up an issue about benchmarking. As often as not, there is a
factor that is ignored in the analysis of the application server. To my way of thinking, the most important benchmark is development time. If I can find a platform that enables me to build my application 20% faster, I am much more likely to license it, even if it runs 20% slower. Why? Because the 20% extra time I have can be put into profiling and benchmarking my application. Usually in that time I can get much more than a 20% speed increase with that much time dedicated toward optimizations.
This is not to say that raw performance is not an important factor--it just is not the only factor. When choosing between products, find out what tools they come with that can make your process faster and more efficient; find out what 3rd party tools are available. This is usually be more important than the benchmark.
If all we were interested in was speed and resource usage, we would
still be writing in assembly ;-)
|
