I'll be the first to admit that the term complex event processing is fairly new to me. Much like MDM three years ago, my career trajectory just hadn't led me to the term. I turned to the web for this post.
Complex event processing "combines data from multiple sources to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible."
Alright, so what does this really mean? To me, the key part of the definition is "infer events or patterns." Making important inferences in all likelihood hinges upon two things. First, I'd think that organizations need extremely accurate and complete structured data. Second, and perhaps not as intuitively, like many things these days, CEP can be more effectively utilized with reliable and related unstructured data.
Wall Street Speaks: Give Us Your Data...All of Your Data
Not unexpectedly, trading firms were quick to jump on the CEP train. Fortuitously a few weeks ago, I finished reading The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It. It's excellent book about some pretty smart cookies who racked their brains devising complicated, highly analytical models of the stock market. While CEP wasn't mentioned in the book, I'll bet that the quants were using similar means to make billions of dollars
If you're trying to beat the market, data matters – any data, even the unstructured type. Again, Wall Street has been quick to enhance their models with as much data as it can. Forget simple data on stock price, time and date of trade, quantity and the like. These are table stakes. Unstructured data has for a while been captured and built into these predictive models.
Simon Says: Remember Three Things about CEP
First, CEP isn't likely to be terribly effective if an organization's structured data is a mess. That much is obvious. Second, an organization that cannot properly define an event is in trouble. For instance, I've worked at organizations whose employees defined a sale very differently. Big problem.
Finally, it seems to me that structured data by itself can only do so much. Today, traditional structured data represents a mere fraction of all available data – and that fraction is shrinking. Why not try to benefit from the vast amounts of "other" data out there, even if there's a high noise-signal ratio?
Feedback
What say you?
