Is effective data governance possible in an era of big data?

"A man's gotta to know his limitations."
—Clint Eastwood as "Dirty" Harry Callahan, Magnum Force

Let's go back in time to 2005, well before the arrival of what we now call Big Data.

A decade ago, YouTube didn't even exist. Facebook was still limited to college students. No one talked about cloud computing.

Seems like a long time ago, right?

Read More »

Post a Comment

ESP can determine if big data is eventful

Many recent posts on this blog have discussed various aspects of event stream processing (ESP) where data is continuously analyzed while it’s still in motion, within what are referred to as event streams. This differs from traditional data analytics where data is not analyzed until after it has stopped moving and has been stored.

Read More »

Post a Comment

Event stream processing – Tips 2 and 3: Understand the life cycle of the data, collection and consumption

Determining the life cycle of event stream data requires us to first understand our business and how fast it changes. If event data is analyzed, it makes sense that the results of that analysis would feed another process. For example, a customer relationship management (CRM) system or campaign management system like SalesForce.com. Here are some questions I would ask:

Read More »

Post a Comment

Can ESP bridge the data quality gap?

As consumers, the quality of our day is all too often governed by the outcome of computed events. My recent online shopping experience was a great example of how computed events can transpire to make (or break) a relaxing event.

We had ordered grocery delivery with a new service provider. Our existing provider gave amazing service – but at a higher cost – so we were keen to see how the competition fared.

The first order was a success. It arrived on time and at a considerable cost savings.

The second order was a disaster. It also highlights a data quality gap that I believe is a perfect scenario for event stream processing. Read More »

Post a Comment

Event stream processing – Tip 1: Don’t be overwhelmed

I believe most people become overwhelmed when considering the data that can be created during event processing. Number one, it is A LOT of data – and number two, the data needs real-time analysis. For the past few years, most of us have been analyzing data after we collected it, not during the event itself. Read More »

Post a Comment

Embedding event stream analytics

In my last two posts, I introduced some opportunities that arise from integrating event stream processing (ESP) within the nodes of a distributed network. We considered one type of deployment that includes the emergent Internet of Things (IoT) model in which there are numerous end nodes that monitor a set of sensors, perform some internal computations, and then generate data that gets pushed back into the network. Most scenarios assume these data streams are accumulated at a central server that analyzes the data and then subjects it to existing predictive and prescriptive analytical models. Then, the models generate notifications or trigger the desired, automated actions.

The conclusion we came to, though, is that forcing all the decisions to be made at the central server might be a somewhat heavier burden than is necessary. Because this approach requires a full round trip for communication (sensors to end node to network to central server, then back to network to end node to controllers, for example). Read More »

Post a Comment

Three things that need to get real – real-time, that is

In my previous post, I discussed the similarities, differences and overlap between event stream processing (ESP) and real-time processing (RTP). In this post, I want to highlight three things that need to get real. In other words, three things that should be enhanced with real-time capabilities, whether it’s ESP, RTP or both.

Read More »

Post a Comment

Pushing event analytics to the edge

In my last post, we examined the growing importance of event stream processing to predictive and prescriptive analytics. In the example we discussed, we looked at how all the event streams from point-of-sale systems from multiple retail locations are absorbed at a centralized point for analysis. Yet the beneficiaries of those analytic results are not limited to central administrators. Yes, it is true that real-time event data can influence enterprise-wide forecasting and planning. But in essence, the more immediate opportunities for value occur at the warehouses, the logistics managers and the retail sites – not at the hub, per se, but rather the edges. Read More »

Post a Comment

Let us be smarter with the Internet of Things

As we enter the era of “everything connected,” we cannot forget that gathering data is not enough. We need to process that data to gain new knowledge and build our competitive advantage. The Internet of Things is not just a consumer thing – it also makes our businesses more intelligent.

Whenever we approach the idea of competing on analytics and building unconventional business strategies, we come up with one simple outcome – we need to be smarter in whatever we do. Looking at business models over time, they all get more complicated, more fuzzy. But what is constant is that every decision is based on past experiences and is driven by data and analytics.

Being smarter can have many faces. Let’s take a look at just three of them. Read More »

Post a Comment

Why data visualization matters

We've all met people a bit too enamored with reporting tchotchkes. I'm talking about folks who don't know the meaning of the word overkill. They don't understand that one can answer relatively simple questions (based on static data) without the aid of visuals. Examples include:

  • How have sales changed in the last quarter?
  • How much do our customers owe us?
  • How much do we owe our vendors?
  • How many employees did we hire last week?
  • What are our current inventory levels?

Read More »

Post a Comment