Today, I was in a conversation about using Hadoop (a big data platform) for master data management (MDM). I still find it amazing when we have the discussion of what systems feed another system. Many of our friends have spent years creating MDM for customer, product, etc. with success. I'm a true believer that MDM should feed us all (operational data store, data warehouse, etc.).
So answer this: Why would you consider changing this to use Hadoop/big data as the entry point for MDM? Here are some reasons:
- Because big data can handle unstructured data, and we can use it for staging our data. It could be faster and cheaper for us.
- Our big data platforms can create structured data out of unstructured input. That could save time...right?
So, when asked whether to use Hadoop/big data for MDM – or not – the good consultant's answer is "It depends."
I believe that requirements will drive the design when gathered properly. I also believe that discovery should take place somewhere, and Hadoop/big data may be my answer.
But MDM, no matter where you put the data, still requires the following:
- Data management disciplines (the three D’s):
- Data quality.
- Data integrity.
- Data governance.
- A framework that meets ever-changing enterprise needs.
- Business personnel involvement in the MDM project.
That said, as we move into a new era of faster and cheaper ways of doing business, we need to be considerate of new technology. I like to think about how that technology fits into my vision of the current enterprise, and where it might go in the future. While it may not be the current “silver bullet,” it may very well end up being the platform that we drive home.
So, when a new project arrives at your doorstep, stop and ask yourself – Where does it fit? How will it be maintained? Is it sustainable? Does it need to be sustainable?
Download an e-book about the intersection of big data, data governance and MDM.
