Web capacity planning: A delicate balancing act
By Paul Korzeniowski
During the last few years, companies like Amazon.com Inc., eBay Inc., and Yahoo Inc. have quickly been transformed from start-ups to well established multi-million dollar enterprises. However, the transition has not always been smooth. Regularly, users have encountered delays -- or even been entirely unable -- to access dot-com companies' Web servers.
Such problems often stem from poor capacity planning and the lack of robust performance monitoring tools. While vendors have been able to put the infrastructure in place so companies can build alluring Web sites, the products for monitoring system and network usage, which firms need to insure that systems and services are readily available, have been a step or two behind.
The disparity comes from a number of factors, including the Internet's unprecedented growth. "Since the Internet is so new, no one really has a good understanding about how traffic patterns will evolve," said Mike Birmingham, a support analyst with Millennia Vision Corp., a Redwood City, Calif., application service provider that has seen the number of servers supporting its business increase from about 15 in the fall of 1999 to approximately 70 now.
Keeping ahead of dramatic usage spikes -- traffic can double or even triple from month to month -- is difficult because response time problems can stem from a number of components. An ill-configured database management system, an overloaded application server, a slow Web server, an overutilized data center LAN, a maladjusted load balancing switch, or an overworked Internet service provider connection are just a few of the places where bottlenecks can arise.
Companies need tools that can examine all of the possible troublespots and then pinpoint the culprit. Software products designed to monitor, diagnose, and help corporations manage enterprise networks and systems have recently been honed to examine Web system performance. Established management systems suppliers, such as BMC Software Inc., Computer Associates International Inc., Hewlett-Packard Co., and IBM have joined recent start-ups like Manage.com Inc. and NetIQ Corp. in trying to solve Web availability problems.
The Motley Fool Inc., Alexandria, Va. is a typical customer and has seen it Web server farm grow from about 90 systems at the beginning of the year to 125 now. Keeping pace with the growth has been difficult because the financial information supplier had relied on a series of autonomous tools to monitor system and network usage.
As it prepared to move into a new data center, the company decided to pare down its tools and selected BMC Software's Patrol for server monitoring and IBM's Tivoli for trouble ticketing. "As a dot-com company, we had been conducting system upgrades by feel and now have a more organized, systematic way of gauging our infrastructure requirements," said Kevin Book, senior director of technology at the company, which moved into the new data center in March.
In addition, services are emerging that help companies gauge availability. Here, vendors ship automated software agents across the Internet, contact sites, determine whether preselected pages are available, calculate how long they take to load, and collect this information for analysis. These services do more than simply check to see if a server is up; they gather a range of metrics, such as the time needed to identify a server's location, connection setup, first byte received, redirect delays, base-page download and content download.
Site managers can use these systems in two ways. The first is immediate troubleshooting where e-mails or pager notifications tell network technicians that system performance is not meeting preset thresholds. The second is capacity planning. By collecting the performance data, IS managers can deduce usage trends and decide to upgrade server hardware, divide applications among a couple of DBMS, change backbone configurations or move content closer to repeat users.
Because Web monitoring is so complex, the services focus on various areas. Freshwater Software Inc., I/Pro Inc., Keynote Systems Inc., Service Metrics Inc., and WebPartner Inc. focus on network monitoring and deploy measurement software across the Internet that collect availability data. Their services can help site managers reposition content on the Web for better performance or work with their ISPs to establish adequate service-level guarantees.
Evity Inc., Mercury Interactive Corp., and Optimal Networks Corp. concentrate on examining how long it takes a server to process a transaction by sending synthetic traffic to a site. Transactions can be extremely difficult to track because there are so many ways to personalize content that simply delivering a browser-based URL recorder to a site manager to record a given transaction doesn't always work. Consequently, some manual intervention may be required to interpret the results.
Others have an end user focus. Customer Insites Inc. and WebCriteria Inc. examine human factors in site performance, such as the visitor perception that a site has stopped responding when one of the site's pages takes significantly longer to load than others. Companies, such as Vividence Corp., demographically select volunteers and then electronically track these human visitors to sites using a combination of questions and site heuristics to assess site performance, errors and ease of use.
Many of these tools and services are new so few companies have put them in place. Industry analysts estimate that less than five percent of all Web sites rely on such products, although that number is expected to increase dramatically during the next few years.
Change will come because online marketers and merchants are discovering that they can't sustain vibrant businesses if their sites are plagued by availability problems: major outages, slow performance, content errors and broken transactions.
"Availability is our top priority, even higher than speed and features," said The Motley Fool's Book.
That's because competitors' sites are just a click away. Market research firm Jupiter Communications Inc. found that 46 percent of e-commerce customers left sites with products they desired because of poor performance. And 24 percent of those customers returned to the site only after establishing a relationship with a competing company.
So, dot-com's want to identify and fix availability problems ASAP.
EBags Inc., a Greenwood Village, Colo., e-commerce supplier, was experiencing server and network problems a few months after introducing its service in the spring of 1999. The firm examined established management tools like HP's OpenView, found them expensive, and selected Freshwater Software's SiteScope. "Since we began using the service, we gained more insight into how are systems are working, been able to stay ahead of processing load changes, and haven't experienced any significant performance problems," said Mike Frazzini, vice president at eBags. These new monitoring tools can help that become a sentence every Web site administrator can utter.
Copyright 2001, availability.com. Reprinted by permission.
FOR MORE INFORMATION: