Over the course the last eight years, I have had the privilege of writing for this site and its predecessors. Yes, I've been a member of this forum since the Dataflux days. In my last post on this site, I'd like to offer a few observations and cautious predictions on data-related matters.
Some of my favorite posts include thoughts on big data, GPDR, data lakes, analytics and my most recent ones on artificial intelligence. I have tried to offer reasonably intelligent commentary based upon my years of experience in the field and a decent amount of research. And let's not forget the attempted humor. I have sprinkled in my fair share of Seinfeld and other pop-culture references.
Accomplishments
I have little doubt that, over the last decade, many organizations have made considerable progress in areas such as streaming data, master data management, analytics, data visualization and data quality. Sure, companies such as Amazon, Facebook, Google and Netflix continue to lead. In large part due to their data-related efforts, they have built enormous technology and data moats around their businesses. The current regulatory environment aside is very much a function of this power.
Based upon my observations and conversations with people in the know, lesser-known organizations are generally treating their data as more of an asset than they did back in 2011 when I started writing for SAS. Better data management is leading to better analytics. Thank Michael Lewis' Moneyball for bringing analytics into the forefront.
Current and future challenges
There's no doubt that significant challenges remain across a number of different dimensions: analytics, data governance and data management are just a few of the areas rife with minefields. These burdens are particularly acute for large firms struggling to adopt new technologies, data sources and Agile mindsets in an increasingly chaotic world.
I was reminded of this fact during a recent Phoenix Meetup I attended for data enthusiasts. The keynote speaker was a finance director for a large multinational aerospace and consumer product company. (Call it YYZ here.) Over the course of an hour, he detailed how YYZ struggled to move away from Excel- and PowerPoint-based reporting to a more contemporary solution. (Why anyone would use PowerPoint as a reporting application is beyond me.)
Beyond that, at YYZ redundant data sources often complicated arriving at a single version of the truth. The company's frequent struggles with structured data made gleaning insights from its unstructured counterpart difficult, especially in the short term.
More often than not, the obstacles were people- and culture-driven. I'm talking about employees loathe to learn new applications and think beyond their provincial domains. In a sense, then, nothing in the data world has changed since I starting writing posts for this site.
Simon Says
Put differently, organizations need to overcome significant business, security, technological, privacy and regulatory challenges in the coming years. There is no one silver bullet in the data realm – nor in any other. To quote Walter White (albeit in a very different context), "We've got work to do."
Feedback
What say you?
Thank you for reading, sharing, and commenting on my posts over the past eight years. I have really enjoyed chiming in with my predictions, comments and rants.
