For enterprises across the globe, I'm hard-pressed to think of a more game-changing development than the advent of relational databases. One could write a very long book about how they have vastly improved organizations' ability to collect, store, process, and act on customer, employee, and product information—at least certain types of information. Yes, I'm talking about the structured kind.
Will the big data revolution lead to similar advancements?
This is almost a rhetorical question for me. I don't claim to be impartial here. In Too Big to Ignore, I make the case that big data is, well, too big to ignore. New forms and sources are streaming at us faster and in greater volumes than ever. At the same time, though, petabytes of unstructured data don't just make sense out of themselves, much less in those same relational databases If organizations are to truly harness its power, then they need to do more than just employ "technology." No, people need to be involved from the get-go.
All of this begs the question, What skills will be required to make sense of big data? To be sure, new times have always called for new skills. Those who wrote SQL statements may very well not be able to do the same against large, unwieldy datasets. (I qualify that statement because one can use this time-tested language in Hadoop environments.)
Forget new languages, distributed file systems, and other "technical" skills for the time being. Sure, these are important. In order to make sense out of big data, something bigger (pun intended) is required.
A Holistic Mind-Set
It's not hard to find simple answers when dealing with relational data, provided that the data is of sufficient quality. Basic counts, maximums, minimums, and the like can easily be discerned. Ditto for trends.
Dealing with big data, though, is a very different ball of wax. One may spend weeks or even months on answering thorny issues, only to be thwarted. For instance, tools that analyze text have never been more powerful, but make no mistake: Determining actual employee or customer sentiment is not nearly as simple as counting the number of times that either uses a pejorative word, something that Seth Grimes knows all too well.
Because outcomes and timeframes are often uncertain, "skills" like risk-tolerance, patience, long-term focus, and general curiosity have never been more important. That goes double for communications. Highly technical folks need to be able to express what they're doing in laymen's terms, something that they have not historically done particularly well. (It's a key emphasis of my forthcoming book, Message Not Received.) By the same token, business users don't need become data scientists. They do, however, need to increase their numeracy.
Simon Says: Small data is akin to algebra; big data is like calculus.
I've said many times that there's no big data magic switch. Think a marathon, not a sprint. Yes, new technologies are essential. Don't make the mistake of thinking that tech alone gets an organization home. Machines don't think for themselves (at least not yet). Foolish is the organization that minimizes the importance of the people element.
Feedback
What say you?
1 Comment
If "Small data is akin to algebra; big data is like calculus" what's the equivalent of basic arithmetic? How about an understanding of Many-to-Many, One-to-Many, One-to-One, 1st Normal Form, 2nd Normal Form, and 3rd Normal form?