My favorite IDEAS takeaway


I have worked on projects that today would be referred to as master data management (as well as customer data integration) for almost twenty years, so I have had some confidence in, over the years, referring to myself as an “MDM expert.” Yet the focus of most of the projects that I had worked on involved record linkage for the sake of data cleansing and resolution, as a way to identify records in different data sets that referred to the same individuals and then merging sets of records together into a single record, effectively combining the data into a single result. More to the point: the essence of the activity was integrating records together.

That being said, I recall one of my favorite SAS DataFlux IDEAS conference moments back in 2006. It was in Las Vegas, when Cathy Willot from Amgen described the process her company had gone through to employ MDM techniques for regulatory compliance. As a pharmaceutical company, Amgen is obliged to comply with numerous rules and regulations, and Cathy’s talk described how MDM and CDI techniques were used to increase productivity and reduce process complexity. She also noted something I had not really ever grasped before: the use of these data integration techniques for compliance with rules directing segregation of data.

As I recall, the specific challenge involved understanding certain (compliance-oriented) business processes where data records specifically could not be linked together. Similar to the concept of a “Chinese wall” in the financial industry (that separates individuals or groups and restricts the flow of information), her company needed to know where records in two different data sets and applications referred to the same individuals and therefore needed to remain segregated.

This was the first time I truly grokked the idea that MDM techniques could be used for more than just data integration and consolidation, and could be employed as tools to support different types of business intent. This has influenced our approach to considering master data management programs and in our consulting engagements, specifically in helping our customers differentiate between the adoption of MDM based on what the analyst/press suggests (that is the dreaded “single source of truth”) and sentient use of available technologies to help solve specific business challenges.


About Author

David Loshin

President, Knowledge Integrity, Inc.

David Loshin, president of Knowledge Integrity, Inc., is a recognized thought leader and expert consultant in the areas of data quality, master data management and business intelligence. David is a prolific author regarding data management best practices, via the expert channel at and numerous books, white papers, and web seminars on a variety of data management best practices. His book, Business Intelligence: The Savvy Manager’s Guide (June 2003) has been hailed as a resource allowing readers to “gain an understanding of business intelligence, business management disciplines, data warehousing and how all of the pieces work together.” His book, Master Data Management, has been endorsed by data management industry leaders, and his valuable MDM insights can be reviewed at . David is also the author of The Practitioner’s Guide to Data Quality Improvement. He can be reached at

1 Comment

  1. Hi David,
    Thanks a lot for your information.

    Record linkage is a common request cross various HealthCare system, Is there any new features from DataFlux which focus on linakge/clean up tasks? I like to learn more from it.

    A typical Healthcare will include multiple systems, such as membership enrolment, patient clinical visit, lab, pharmacy etc. Manual entry is involved in almost every systems, human errors, mistake about key ID field, and missing information happens every day.
    A easier, more robusted linkage method will help to ease the data-dedupe, data-cleanup tasks, it will definitely benefit many other industries too.

Leave A Reply

Back to Top