Implications of coalescing data quality


Over my last two posts, I suggested that our expectations for data quality morph over the duration of business processes, and it is only at a point that the process has completed that we can demand that all statically-applied data quality rules be observed. However, over the duration of the process, there are situations in which the rules might be violated yet the dynamic nature of the data allows for records to temporarily remain in a state that ultimately might be deemed invalid.

This stands in stark contrast to some other practitioners who state that there are only two points that are important in the life of a piece of data, namely the “moment of use” (at which the quality is presumably determined) and the “moment of creation” (at which the data must be created correctly). There are going to be many “moments of use” at different points in the processes that touch any piece of data, and those different uses will reflect different expectations depending on the business context.

And if, as I have suggested, the quality of data is both temporal and contextual, then it may be impossible to demand that the data be “correct” at its moment of creation, since the meaning of “correct” changes over time and in relation to different uses. Rather, we have to look at the assertion of data quality over a continuum, and understand the paths through which a process may meander and correspondingly, the impacts of the various data touch points along the way. Each touch point must carry its own “view” of quality presuming the uses that follow, and in turn, must apply the assertions that can be impacted at each stage of processing. (For more on data quality in a big data world, read my white paper: "Understanding Big Data Quality for Maximum Information Usability.")

By aligning data quality validation and monitoring with the dynamic nature of the underlying data, we can evolve a means for specifying the post-conditions (i.e., what quality means after the touch point) and the preconditions (i.e., what is acceptable prior to the touch point). These pre-conditions and post-conditions can then be folded into the design of the application implementing the business process and be directly integrated into the code to ensure a level of acceptable quality at all subsequent touch points.



About Author

David Loshin

President, Knowledge Integrity, Inc.

David Loshin, president of Knowledge Integrity, Inc., is a recognized thought leader and expert consultant in the areas of data quality, master data management and business intelligence. David is a prolific author regarding data management best practices, via the expert channel at and numerous books, white papers, and web seminars on a variety of data management best practices. His book, Business Intelligence: The Savvy Manager’s Guide (June 2003) has been hailed as a resource allowing readers to “gain an understanding of business intelligence, business management disciplines, data warehousing and how all of the pieces work together.” His book, Master Data Management, has been endorsed by data management industry leaders, and his valuable MDM insights can be reviewed at . David is also the author of The Practitioner’s Guide to Data Quality Improvement. He can be reached at

Related Posts

Leave A Reply

Back to Top