Dovetail process flow and master data management

man working with data outside at nightIn my last post, I discussed the issue of temporal inconsistency for master data, when the records in the master repository are inconsistent with the source systems as a result of a time-based absence of synchronization. Periodic master data updates that pull data from systems without considering alignment with in-process business activities create the potential for this inconsistency. The way to address this is straightforward: don’t do your master data consolidation as a periodic process. Instead, push your identity resolution and master data management (MDM) into your business processes.

That is, of course, easier said that done. It typically requires two key activities that are not insignificant:

  • Align all enterprise process flows and all related master data touch points with the master record life cycle.
  • Renovate existing systems to use defined master data services as a way to manage entity data.

Read More »

Post a Comment

Streaming data analytics: A better way to fight fraud and cybercrime

confident woman fighting fraud and cybercrime with streaming dataThe numbers are daunting.

  • More than 40 million Americans have their identities stolen each year.
  • Credit card companies lose more than $200 billion annually due to fraud.
  • Cybercrime-related losses exceed $3 million per claim for large companies.

If you’re like me, those stats are enough to give pause. To fuel the concern, 24/7 news channels often focus on technology as the culprit that enables modern day fraud and crime. It leaves some to reason that each new technological advance might not be something to cheer. Yet while technology for wrong gets the headlines, we rarely hear about the things proactive companies are doing with technology to protect us.

The banks we do business with, the retailers where we shop and the credit card companies that finance many of our purchases are all working behind the scenes to cut or eliminate those daunting numbers. They do it for the benefit of their organizations – and for us, their customers.

Read More »

Post a Comment

What's the relationship between big data and MDM?

At lunch with a nurse friend of mine recently, I happened to drop the term big data. No sooner had the words left my mouth when I asked her the question, "Have you ever heard of big data?"

woman with tablet, represents big data and MDMAs I expected, she responded in the negative. I proceeded to give her a short, jargon-free definition of the term rife with examples of the social media sites she frequents. Of course there's more to it, but the collection of photos, tweets, blog posts, Facebook likes, LinkedIn articles, YouTube videos and the like add up to a boatload of information.

I was thinking about this in the context of this month's theme: master data management (MDM). In particular, I stewed over the question "What – if any – is the relationship between big data and MDM?"

Although several of my books touch upon MDM, I don't consider myself a true expert on the matter. (David Loshin could dance circles around me here.) Still, I know a fair amount.

Read More »

Post a Comment

Temporal consistency for master data

copule discussing credit application, represents MDMMaster data management (MDM) provides methods for unifying data about important entities (such as “customer” or “product”) that are managed within independent systems. In most cases, there is some kind of customer data integration requirement for downstream reporting, and analysis for some specific business objective – such as customer profiling for product cross-sell marketing.

I've worked with many customers who were struggling to complete the development (and consequently production) of their master data repositories. Based on this experience, I've started to see an implementation artifact that exposes some of the key issues with the most common approaches to MDM deployment. In many environments, the deployment model effectively sweeps data from operational environments on a periodic basis, applies the identity resolution and matching algorithms, and links records representing unique entities together. These records are subjected to some kind of “survivorship” process that plucks assorted values to create the master record.

Read More »

Post a Comment

The quality of data in motion

streaming data and man checking smart watchI've spent much of my career managing the quality of data after it was moved from its sources to a central location, such as an enterprise data warehouse. Nowadays not only do we have a lot more data – but a lot of it is in motion. One of the biggest data movers is the Internet of Things (IoT), which is comprised of machines with embedded software, sensors and connectivity enabling them to collect and exchange data. IoT produces a mixture of machine-generated and human-generated data in motion. When we have traditionally waited until data stopped moving before we assessed its quality, how much quality should we demand from data in motion?

Read More »

Post a Comment

Considerations and tips for big data integration

I like many things about writing for the Data Roundtable. Near the top of my list is the fact that I can actually see the number of page views for each of my articles. Make no mistake: this isn't always the case. For instance, I scribe for The Huffington Post but can't tell you my page-view counts. I couldn't tell you why the site doesn't provide this valuable information to its writers, but that's neither here nor there.

Getting back to the Data Roundtable, one of my most popular posts concerns big data integration. With more than 6,000 page views as of this writing, I know it has reached more people than most of my other blog posts. As for why, I'm only guessing, but I suspect that more and more readers are thinking about how to combine their traditional data sources with newer ones.

Against this backdrop, here are some considerations for how to integrate an increasing array of data sources. Read More »

Post a Comment

Where should data quality happen?

Men working in coffee shopIn my previous post I discussed the practice of putting data quality processes as close to data sources as possible. Historically this meant data quality happened during data integration in preparation for loading quality data into an enterprise data warehouse (EDW) or a master data management (MDM) hub. Nowadays, however, there’s a lot of source data available to the enterprise and a significant amount of it doesn’t pass through an EDW or MDM process before it’s used. This begs the question: Where should data quality happen?

Read More »

Post a Comment

IoT: How could your company benefit from real data accuracy?

factory workers checking data on laptopWe had just completed a four-week data quality assessment of an inside plant installation.

It wasn't looking good. There were huge gaps in the data, particularly when we cross-referenced systems together.

In theory, each system was meant to hold identical information of the plant equipment. But when we consolidated the data, there were so many issues it was hard to know where to begin. There were:

  • Different naming conventions for buildings.
  • Different naming conventions for installed equipment and parts.
  • Empty bays in one system that were fully populated in another system.
  • Equipment that was active in one system but decommissioned in another.

The list was seemingly endless, so I thought to myself: "If the systems that are meant to represent the physical equipment are out of sync with each other, just how bad are they out of sync with reality?" Read More »

Post a Comment

How will the IoT affect your data management strategy?

It's hard to find a trend with more attendant hype these days than the Internet of Things (IoT). Read a few of the more than 55 million articles written on the subject and it's easy to believe that it's going to revolutionize the world tomorrow.

person checking a smart watch I wouldn't go nearly that far. But I also disagree with the many skeptics who, in the extreme, think that the IoT is a complete waste of time. I don't own a crystal ball, but I suspect that sensors, wearables and "smart" devices will proliferate eventually, especially once the powers that be agree upon common standards.

Rather than debate whether the cynics or prosthelytizers are ultimately right, I'd like address the impact of the IoT on organizational data management.

Before continuing, a disclaimer is in order. The very title of this post assumes that your organization has developed – and follows – a data management strategy. (For more on this, see my series from a few months back.) Read More »

Post a Comment

Characteristics of IoT data quality

measuring devices on factory floorIn my last post we started to look at two different Internet of Things (IoT) paradigms. The first only involved streaming automatically generated data from machines (such as sensor data). The second combined human-generated and machine-generated data, such as social media updates that are automatically augmented with geo-tag data by a mobile device.

In both of these cases, much of the data is automatically created – so what does it mean to talk about data quality? The answer requires two tasks: a reconsideration of the dimensions of data quality, and a focus on end-user data usability.

Read More »

Post a Comment