Struggling with data governance alignment? Look to history.

If your organization is large enough, it probably has multiple data-related initiatives going on at any given time. Perhaps a new data warehouse is planned, an ERP upgrade is imminent or a data quality project is underway.

Whatever the initiative, it may raise questions around data governance – closely followed by discussions about the need to "align" with the business. Aligning data governance to business value is where many initiatives falter, because it is not always easy to demonstrate tangible value. Read More »

Post a Comment

Big data, big governance

Traditional data governance is all about establishing a boundary around a specific data domain. This translates to establishing authority to define key business terms within that domain; establishing business-driven decision making processes for changing the business terminology and the rules that apply to them; defining content standards (e.g., metadata and data quality rules); and outlining an ongoing process for measuring and monitoring.

The recent data explosion highlights the point that data governance is critical to organizations' success. In fact, the need for a mature data governance framework is accepted more than ever. But despite this acknowledgement, established methods for governing data have not been challenged or altered.

Read More »

Post a Comment

Is effective data governance possible in an era of big data?

"A man's gotta to know his limitations."
—Clint Eastwood as "Dirty" Harry Callahan, Magnum Force

Let's go back in time to 2005, well before the arrival of what we now call Big Data.

A decade ago, YouTube didn't even exist. Facebook was still limited to college students. No one talked about cloud computing.

Seems like a long time ago, right?

Read More »

Post a Comment

ESP can determine if big data is eventful

Many recent posts on this blog have discussed various aspects of event stream processing (ESP) where data is continuously analyzed while it’s still in motion, within what are referred to as event streams. This differs from traditional data analytics where data is not analyzed until after it has stopped moving and has been stored.

Read More »

Post a Comment

Event stream processing – Tips 2 and 3: Understand the life cycle of the data, collection and consumption

Determining the life cycle of event stream data requires us to first understand our business and how fast it changes. If event data is analyzed, it makes sense that the results of that analysis would feed another process. For example, a customer relationship management (CRM) system or campaign management system like SalesForce.com. Here are some questions I would ask:

Read More »

Post a Comment

Can ESP bridge the data quality gap?

As consumers, the quality of our day is all too often governed by the outcome of computed events. My recent online shopping experience was a great example of how computed events can transpire to make (or break) a relaxing event.

We had ordered grocery delivery with a new service provider. Our existing provider gave amazing service – but at a higher cost – so we were keen to see how the competition fared.

The first order was a success. It arrived on time and at a considerable cost savings.

The second order was a disaster. It also highlights a data quality gap that I believe is a perfect scenario for event stream processing. Read More »

Post a Comment

Event stream processing – Tip 1: Don’t be overwhelmed

I believe most people become overwhelmed when considering the data that can be created during event processing. Number one, it is A LOT of data – and number two, the data needs real-time analysis. For the past few years, most of us have been analyzing data after we collected it, not during the event itself. Read More »

Post a Comment

Embedding event stream analytics

In my last two posts, I introduced some opportunities that arise from integrating event stream processing (ESP) within the nodes of a distributed network. We considered one type of deployment that includes the emergent Internet of Things (IoT) model in which there are numerous end nodes that monitor a set of sensors, perform some internal computations, and then generate data that gets pushed back into the network. Most scenarios assume these data streams are accumulated at a central server that analyzes the data and then subjects it to existing predictive and prescriptive analytical models. Then, the models generate notifications or trigger the desired, automated actions.

The conclusion we came to, though, is that forcing all the decisions to be made at the central server might be a somewhat heavier burden than is necessary. Because this approach requires a full round trip for communication (sensors to end node to network to central server, then back to network to end node to controllers, for example). Read More »

Post a Comment

Three things that need to get real – real-time, that is

In my previous post, I discussed the similarities, differences and overlap between event stream processing (ESP) and real-time processing (RTP). In this post, I want to highlight three things that need to get real. In other words, three things that should be enhanced with real-time capabilities, whether it’s ESP, RTP or both.

Read More »

Post a Comment