Blend, cleanse and prepare data for analytics, reporting or data modernization efforts
.@philsimon on the relationship between metadata and unstructured data.
As this is the week of Christmas, many, myself included, have Christmas songs stuck in their head. One of these jolly jingles is Santa Claus Is Coming To Town, which includes the line: “He knows if you’ve been bad or good, so be good for goodness sake!” The lyric is a
With our recent client engagements in which the organization is implementing one or more master data management (MDM) projects, I have been advocating that a task to design a demonstration application be added to the early part of the project plan. Many early MDM implementers seem to have taken the
In an era long gone by (actually not so long ago) we all interacted with computers via terminals to a mainframe or minicomputer systems. Sometimes, you had to book a slot for when you could access and exploit computer resources. The user was subject to interrupted connections or very poor
I have a rule – any conversion or upgrade will require the creation of a decommission plan. A decommission plan should include the following: A list and definition of each database, table and column (source and target). A list and definition of each of the current programs in use (you
The physical data model should represent exactly the way the tables and columns are designed in the in the database management system. I recommend keeping storage, partitioning, indexing and other physical characteristics in the data model if at all possible. This will make upkeep and comparison with the development, test
We've explored data provenance and the importance of data lineage before on the Data Roundtable (see here). If you are working in a regulated sector such as banking, insurance or healthcare, it is especially important right now and one of the essential elements of data quality that they look for
.@philsimon on an important question.
For a long time, master data management (MDM) practitioners boasted about their ability to build a 360° view of customers by aggregating and proactively managing information coming from various business applications such as CRM systems, ERP applications, and other operational systems. But was it really a 360° view? What about
‘Tis the season. While the season means different things to different people, its most common theme is people buying things for people. Things that become presents when they are covered in wrapping paper. Two retailers have been running television commercial campaigns this season about how presents are wrapped. One campaign
Well, Analytic Hospitality Executives, the year has once again flown by, and here we sit just before the holidays looking back on 2014, and figuring out what it all means for 2015. I traveled even more than usual this year (if that’s even possible), spending a significant amount of time
I have a question --- do we need a logical data model for a conversion? Here are my thoughts. I believe the answer is yes if the conversion has any of the following characteristics: The target application is created in-house. This application will more than likely be enhanced in the
In the last post we looked at the use case for master data in which the consuming application expected a single unique representative record for each unique entity. This would be valuable in situations for batch accesses like SQL queries where aggregates are associated with one and only one entity record.
Data. It's everywhere. It can reside in many places through replication, accessibility needs or infrastructure costs. For reporting, that same data can be structurally changed (denormalized or aggregated) into additional reporting and analytic data repositories. Over time, new sources of enrichment of that data become available through traditional data sources
Getting universal buy in for Hadoop needn’t be an uphill struggle. In many cases, it only takes one pilot project to realize the benefits of low cost storage combined with powerful analytics. The Hadoop topic provoked passionate conversatoin at a recent roundtable discussion attended by over 25 people from a range
.@philsimon on the stickiness of data
In my previous post I explained that even if your organization does not have anyone with data steward as their official job title, data stewardship plays a crucial role in data governance and data quality. Let’s assume that this has inspired you to formally make data steward an official job title. How
Last time I suggested that there are some typical use cases for master data, and this week we will examine the desire for accessibility to a presumed “golden” record that represents “the single source of truth” for a specific entity. I put both of those terms in quotes because I
To perform a successful data conversion, you have to know a number of things. In this series, we have uncovered the following about our conversion: Scope of the conversion Infrastructure for the conversion Source of the conversion Target for the conversion Management for the conversion Testing and Quality Assurance for
Here on the Data Roundtable we've discussed many topics such as root-cause analysis, continual improvement and defect prevention. Every organization must focus on these disciplines to create long-term value from data quality improvement instead of some fleeting benefit. Nowhere is this more important than the need for an appropriate education strategy, both in
So much for a single version of the truth.
The bigness of your data is likely not its most important characteristic. In fact, it probably doesn’t even rank among the Top 3 most important data issues you have to deal with. Data quality, the integration of data silos, and handling and extracting value from unstructured data are still the most
I have probably touched on this topic many times before: accessing the data that has been loaded into a master data environment. In recent weeks some client experiences are really highlighting something that is increasingly apparent (and should be obvious) for master data management: the need to demonstrate that it
There are multiple types of data models, and some companies choose to NOT data model purchased software applications. I view this a bit differently. I think that any purchased application is part of our enterprise, thus it is part of our enterprise data model (or that concept is part of the
When you examine where most data quality defects arise from, you soon realise that your source applications are a prime culprit. You can argue that the sales team always enter incomplete address details, or the surgeons can't remember the correct patient type codes but in my experience the majority of
Data. Our industry really loves that word, making it seem like the whole world revolves around it. We certainly enjoy revolving a lot of words around it. We put words like master, big, and meta before it, and words like management, quality, and governance after it. This spins out disciplines
For many industries, big data analytics have opened numerous doors for more employees to be groundbreaking and to challenge the corporate status quo. Prior to big data technologies, risk taking behaviors were primarily reserved for provocative souls who stretched organizational boundaries to disrupt industries, such as airline revenue management. There were winners and losers
Don't be shy! Interviewing people BEFORE or AFTER a facilitated session just takes a bit of confidence, and good preparation. Building your confidence gets easier and easier the more you participate in interviews. The objective is to prepare and not waste anyone’s valuable time. I like to prepare notes based on
Many managers still perceive data quality projects to be a technical endeavour. Data being the domain of IT and therefore an initiative that can be mapped out on a traditional project plan with well-defined exit criteria and a clear statement of requirements. I used to believe this myth too. Coming
Hadoop recently turned eight years old, but it was only 3-4 years ago that Hadoop really started gaining traction. It had many of us “older” BI/DW folks scratching our heads wondering what Hadoop was up to and if our tried-and-true enterprise data warehouse (EDW) ecosystems were in jeopardy. You didn't
