Maximizing and managing big data with SOA middleware
A comprehensive collection of articles, videos and more, hand-picked by our editors
"Big data" can be a new opportunity for organizations to gain business value or it could be another technology dead end. In either case, it has big implications for how IT professionals do their jobs today. In particular, it may shift the focus of the enterprise architect. Such software development leaders must now ensure big data is made to serve real business needs.
The big data approach -- which often revolves around reams of structured and unstructured data from social media forums, Web server logs, and other sources -- calls for myriad skills in analytics, data science and business strategy. It does not exist in a vacuum.
Industry observers say today's successful enterprise architect will embrace a standards- and scalability-friendly technology environment, focus on business goals, and avoid building enormous architectures to feed and nurture the big data app. When they do, the architect will find something familiar and something new.
The big data paradigm is commodity-driven and tends toward open source software such as Hadoop and MongoDB, according to independent consultant Tavo De León. This is a change from the time when most companies invested heavily in specialized hardware. Enterprise architects were traditionally responsible for maintaining a technical ecosystem that was vendor-dependent and reliant on one technology. Not so anymore.
"What companies do now is they spread things out horizontally on the cheapest hardware possible," De León said. "So enterprise architects have had to expand themselves horizontally from a technical point of view."
On the business side of the data
Effective enterprise architects have to stretch themselves in other ways too, experts say. Big data is a tech-enabled business trend. As such, it upends traditional organizational boundaries. Today's enterprise architects are no longer concerned only with the technical; they must also understand the business rationale behind the technology, said Ron Tolido, senior vice president and CTO of applications at Capgemini. "Big data is one of those technologies that is much closer to the business side," he explained. "It's a whole new area that [enterprise architects] need to master."
Jeff SchulmanGartner group vice president and team manager
This can be a challenge because of the rapid rate of business change. In the past, enterprise architects could spend long periods of time perfecting a data project for a static set of queries. Leveraging big data, on the other hand, is often about obtaining near-real-time responses to highly variable information using queries that are constantly changing.
"Enterprise architects are used to a slower cycle," said Brian Hopkins, principal analyst at Forrester Research Inc. in Cambridge, Mass. "But businesses get it; they're reacting to digital disruption and rapidly changing market conditions. They're going to enterprise architects and they're saying, 'I need something in six months.'" Sometimes the project horizon is even closer.
To keep ahead, Hopkins advised that enterprise architects inch their way out of IT and into business. "Big data is fundamentally shifting the [enterprise architect's] role away from being an aligned-IT technology planner into that of business-technology strategic partner," he said. The enterprise architect's end goal should be "to plan a strategy that effectively leverages big data for more value."
Big data, not big data architecture
With big data at play, a company's bottom line has more to do with enterprise architecture than ever before, experts say. By finding more efficient ways to leverage back-end data assets in order to achieve business goals, enterprise architects can ensure that the business case remains clear throughout big data technology efforts.
This should guide the way enterprise architects think about big data architecture, said Tolido. "There's always a reason why business is enthusiastic about big data. Why is the business side interested in it? Once [you understand] that, it's not a matter of big data architecture," he said.
"The approach that the enterprise architect should take is not to create big architectures that are all-encompassing and covering the entire data landscape," Tolido advised. "Instead, they should understand what the real business needs are in the company."
Still, not all enterprise architects are armed with the big data skills and business knowledge to make this transition. "Learning big data is a considerable challenge," De León said. "I don't know that [enterprise architects] are getting the training."