IT spending growth is somewhat flat overall, but the area of "big data" technology is expected to grow dramatically, according to industry analyst group Gartner Inc. Big data is high-volume, high-velocity information that includes both structured and unstructured types of data.
Gartner projects worldwide IT spending will grow from $3.6 trillion in 2012 to $3.7 trillion in 2013, a slight increase. Still, by 2015, 4.4 million IT jobs globally could be created to support new big-data undertakings, according to Peter Sondergaard, senior vice president at Gartner and its global head of research.
Big data is the strongest information industry component among a host of forces changing the IT industry, Sondergaard told a crowd assembled at this week's Gartner ITxpo in Orlando, Fla. He proposed that organizations of the future will be distinguished by the quality of the predictive algorithms their data scientists can create. Today, filling new big-data-related jobs can be a challenge.
While the task of analyzing data is not new, modern big data tools like Cassandra, Hadoop and NoSQL, for example, are relatively unfamiliar. As a job category, "data scientist" is a fairly new kid on the block, but it could be finding definition.
At Gartner ITxpo, Ray Valdes, Gartner analyst and research vice president, suggested that data scientists are developing definite ways of looking at problems. Their basic process, he said, is to first, define a population; second, get detailed data; third, run classifiers on the data; and fourth, retune the system they have created. "There is a lot of power in small bits of data," he added.
But knowing where to find the right data and knowing the right algorithmic classifiers, among other things, is quite an art. It also can be tricky, because regulations cover how data can be used.
Valdes warned that many people now working with data are unaware of its worth -- and its perils. "The data is more valuable than you think. It is also more risky than you think," he told the ITxpo crowd.
Brain drain in the data domain
The data scientist skill set is challenging because it calls for knowledge of the domain within which the data resides. That's not likely to change, according to viewers.
"The concept of the data scientist is still evolving," said Ken Rabolt, chief data architect at The Nielson Co. "Getting into the data is mostly the best way to figure out what the questions are," he said, adding, "In a few years, there will be enough knowledge out there so there will be mentoring, but now it is still kind of a discovery mode."
The data scientist position will find increasing importance, agreed Eric Williams, former executive vice president and CIO at Catalina Marketing Corp., who also was on hand at ITxpo. Companies can't move too quickly, he maintained. "If a company doesn't have one today, it needs to be embracing the role of data scientist like … yesterday," he quipped. "Companies that don't realize that will find themselves last on the list very rapidly."
Williams was asked whether the school-trained data scientist is ready to succeed in an organization. "We found getting people out of school that have the knowledge on how to handle the information aspects is critical," he replied. "But if you don't have some of the business knowledge, you will fail."
A mix of talent is needed: Data scientists cannot just have the statistics knowledge, nor can they just have the business knowledge, Williams said.
Follow us on Twitter at @SearchSOA.