How can data privacy and protection help drive better analytics?

Balance. This is the challenge facing any organisation wishing to exploit their customer data in the digital age. On one side we have the potential for a massive explosion of customer data. We can collect real-time social media data, machine data, behavioural data and of course our traditional master and […]

Post a Comment

The growing importance of big data quality

Our world is now so awash in data that many organizations have an embarrassment of riches when it comes to available data to support operational, tactical and strategic activities of the enterprise. Such a data-rich environment is highly susceptible to poor-quality data. This is especially true when swimming in data lakes – […]

Post a Comment

Why analytical models are better with better data

Most enterprises employ multiple analytical models in their business intelligence applications and decision-making processes. These analytical models include descriptive analytics that help the organization understand what has happened and what is happening now, predictive analytics that determine the probability of what will happen next, and prescriptive analytics that focus on […]

Post a Comment

Why big data and streaming go hand in hand

As I've previously written, data analytics historically analyzed data after it stopped moving and was stored, often in a data warehouse. But in the era of big data, data needs to be continuously analyzed while it’s still in motion – that is, while it’s streaming. This allows for capturing the real-time value of data […]

Post a Comment

Finding the balance between short- and long-term big data storage

Back before storage became so affordable, cost was the primary factor in determining what data an IT department would store. As George Dyson (author and historian of technology) says, “Big data is what happened when the cost of storing information became less than the cost of making the decision to […]

Post a Comment

Beyond the boundaries of structured data: Part two

How many times have you gone onto a website, put a few things in a shopping cart, and then exited the Internet? I do it all the time. Sometimes when I log on to that site during my next visit, those same items are still in my cart – ready for purchase. I find […]

Post a Comment

Data integration modernization: Key recommendations

Modernization. It’s a hot topic for organizations in all types of industries that are looking for ways to streamline hardware and software footprints while gaining control and insights from the data deluge. In the data integration space, this means we have to look beyond a traditional ETL approach to one […]

Post a Comment

Big data modeling: An iterative approach - Part 1

In my prior two posts, I explored some of the issues associated with data integration for big data and particularly, the conceptual data lake in which source data sets are accumulated and stored, awaiting access from interested data consumers. One of the distinctive features of this approach is the transition […]

Post a Comment

ESP can determine if big data is eventful

Many recent posts on this blog have discussed various aspects of event stream processing (ESP) where data is continuously analyzed while it’s still in motion, within what are referred to as event streams. This differs from traditional data analytics where data is not analyzed until after it has stopped moving and has […]

Post a Comment

Event stream processing – Tip 1: Don’t be overwhelmed

I believe most people become overwhelmed when considering the data that can be created during event processing. Number one, it is A LOT of data – and number two, the data needs real-time analysis. For the past few years, most of us have been analyzing data after we collected it, […]

Post a Comment