The unfortunate news for those organizations that have inherited a mess due to organic growth is that only hard work and money is going to help re-orient their IT systems toward driving future growth. The good news for these businesses is that today's technologies and practices allow it to be done incrementally, while still instituting an enterprise architecture.
For businesses that have emerged within the past 15 years, the likely explanation for not having agile systems is a lack of emphasis on an Architecture-first mentality toward their IT investments. In these situations, there's plenty of fault to go around, but recrimination only acts to create a hostile environment and does little to help correct the situation. It is, however, beneficial to examine the factors that obstruct Architecture-first mandates.
1.No one wants to invest in the intangible.
The biggest issue with an architecture-first mandate is that it produces artifacts, but typically these artifacts mean little to those responsible for funding. Here's a perfect example, if you need a water heater and one plumber brings you a blueprint of the system they will build for you so that you will never run out of hot water again while also lowering your energy costs and another plumber shows up with a hot water heater in the bed of their truck, who would you grant the job? It takes a lot of effort to sit down with the first plumber and understand what they are selling and how it might be better than a Home Depot special, perhaps, ultimately, even lowering your energy costs while providing a never-ending supply of hot water; especially when you haven't showered in a couple of days. Leading an architecture-first mandate requires the ability to sell vision; a very rare skill. In fact, the ability to sell vision is most often found in entrepreneurs, which you will not typically find inside mid- to large-sized corporations' IT departments. So, businesses have an impedance mismatch with regard to their needs and their employed skills. This can be overcome by using outside resources, but that first requires that IT admit their weaknesses and relinquish this task to an outsider. A concept that may help this along is not to see the outsider as a threat, but think of it as hiring an external advertising firm to help sell your product to business.
2.It's easy to think you have enough experience to know the right answers, even without supporting evidence.
When it comes to architecture, experience speaks volumes. The more an architect has seen the greater his/her field of vision when it comes to solutions. However, there's a downside to having a lot of experience—architects being to listen to their intuition without first capturing enough supporting evidence. For example, a new requirement might emerge that really requires redesign of an existing data model. However, because it seems similar to something the architect has worked on before, their impulse is to simply amend the current data set. Within months of the amendment, the system starts to perform sluggishly. After a few more weeks of analysis and monitoring, it's discovered that the cause of the poor performance is the change that was introduced weeks earlier. An Architecture-first mandate requires that the amendment be modeled and that the supporting queries be reviewed in tandem with the current queries and perhaps even required some load modeling on the amended data set.
3.Time is always of the essence.
When was the last time you heard the words, "don't worry how long it takes, we prefer to do it right rather than fast." While it's true that a well-designed system usually will take less time to build and usually has a higher quality with less bugs, design takes time. Part of this problem resides with Factor #1; betting on the intangible versus "show me now." Agile development methodologies help to balance between design time and tangible output, but running a successful agile project takes diligence and discipline. Rapid application development techniques also help with the psychological factor of dealing with the intangible, while allowing the user interface aspects of the application to be produced and approved. Still, these efforts work well at the application scope. Cobble a few well-designed applications together into a portfolio without any higher-level design mandates and you have a mess. Thus, IT management has a difficult task to constantly balance producing and designing. Besides, an all up-front design is not always the best course of action. It reeks of big-bang and doesn't get proven out until after the design effort is done. By then, the only course of action is to deal with the faults and try to patch as you go or retreat to the drawing board for version 2.0. Agile methods up and down the stack from enterprise to application provides a solid approach to dealing with the time issue.
Leading an architecture-first mandate in an organization that does not currently subscribe to the value in designing before doing is a difficult task for even the most skilled politician. However, most times, the business does not get involved in the "how" and "what" of IT, it simply wants something by a certain date. While not always popular, incorporating the proper time in to the plan may be enough to get the practice in motion. Often times, even if the business pushes back on the time, walking them through areas of complexity and incorporating them into the QA cycle may be enough to get their buy-in.
However, in those rare cases, often associated with very visible business initiatives, where the business is being demanding, and that demand will interfere with the ability to make a system agile and ensure its stability, you may have to undertake the difficult task of selling the value of architecture to business.
JP Morgenthal is a leading independent IT architecture consultant in the Washington, DC region who focuses on enterprise architecture, SOA, BPM and cloud computing. He is an authority on XML, Java, SOA, EAI and EII. He has written three books on integration, the most recent Enterprise Information Integration: A Pragmatic Approach.