As consumers, the quality of our day is all too often governed by the outcome of computed events. My recent online shopping experience was a great example of how computed events can transpire to make (or break) a relaxing event.

We had ordered grocery delivery with a new service provider. Our existing provider gave amazing service – but at a higher cost – so we were keen to see how the competition fared.

The first order was a success. It arrived on time and at a considerable cost savings.

The second order was a disaster. It also highlights a data quality gap that I believe is a perfect scenario for event stream processing.

Our allotted delivery time came and went, as did the kids' favorite Friday supper. After hastily improvising with leftovers, we called the supermarket chain to find out what had happened.

The customer service rep believed that the order was on the way and that the store would call us. They didn't.

We called customer support two hours later; they said the store would call us. They didn't.

The next day, we called again and were told that our credit card payment didn't go through. The failure had apparently occurred 12 hours before the anticipated delivery time and had caused the store to stop loading the van with groceries.

We've now been billed the full amount for a delivery that never took place. We also have to take it on ourselves to request a refund from the credit card company. The grocery firm says they can't do anything from their side, even though they fouled up.

So, this was not exactly five-star service. And this creates a problem because, as consumers, we're increasingly shifting our shopping online. Events like these dictate our loyalty and shopping preferences. My wife has categorically stated she will never use that provider again based on the level of service offered. As this shows, lower cost may get your business to a certain point of popularity, but good customer experience clearly takes you a whole lot further.

While the lack of groceries was frustrating, the follow-up service was the final blow to the relationship. When no one can tell you what is happening because their data is so badly fragmented and out of sync, consumers start to shop around.

Where did the supermarket chain go wrong?

You could argue that this situation was caused by a data quality issue. I know that as data quality practitioners we tend to do this a little too eagerly, but I now ponder:

• Did the customer service team not get timely information, so they didn't know that the store would not deliver the groceries?
• Was there inaccurate billing information or validation at the point of the credit card transaction?

In a scenario like this, part of the problem is due to the batch approach to data management. Assessing data that comes from a batch environment invariably leads to an obvious setback when using a conventional data quality approach, because you're never looking at a real-time situation. You're often looking at end-of-day accounts or order analysis.

Could this be where event stream processing starts to gain ground as a data quality monitoring solution? If the grocery provider had been able to examine the different streams of data, could they have observed patterns of failure?

Then there was the failed banking transaction. Were there other similar events at the same time?

What about repeated call logs when I spoke to the service center to get their analysis of my account and delivery order? Are those logged into an event stream somewhere? Finally, what about our repeated website visits to check our order? Could these have been analyzed in real time?

It's not a stretch to believe that event stream processing could enable us to take data quality management in a new direction, actively spotting patterns in event streams that are the result of data quality defects.

By taking a closer look at the real-time streams of data from existing systems, could your company offer a more proactive, customer-focused level of service that analyzes patterns of behavior for likely events of interest? Not all of these streams will be related to data quality – but I believe that event stream data extends the data quality debate to an entirely new direction.