Unless you’ve been living under a rock, you've surely noticed the increasing numbers of headlines about big data, Hadoop, internet of things (IoT) and, of course, data streaming. For many companies, this next generation of data management is clearly marked "to play with later." That's because adopting the next wave
At its core, data compliance is built on simple foundations. Dylan Jones closes this series by explaining the remaining components of the "4F framework."
As you work toward data compliance, Dylan Jones says keep it simple – start with the 4F’s: Function, Flow, Form, Foster. Part 1 looks at the first two.
The financial sector has always been subjected to regulatory compliance laws and directives. Consumers, lawmakers and politicians would expect no less. But it's fair to say that the financial sector has witnessed a "hockey stick" trend regarding new regulations in recent years.
What are the most useful skills a data quality leader can possess? As an editor of an online data quality magazine, I naturally get asked this type of question regularly at events and meetups. My answer may surprise some who are expecting a data-centric response. I firmly believe that sales and
During a data quality assessment, one of my clients discovered that a large chunk of data that ultimately fed into their business analytics engine was sourced externally. After examining the contracts surrounding this data, I found that 100% of it failed to possess service-level agreements (SLAs) for the quality of
Balance. This is the challenge facing any organisation wishing to exploit their customer data in the digital age. On one side we have the potential for a massive explosion of customer data. We can collect real-time social media data, machine data, behavioural data and of course our traditional master and
Many people have the perception that data governance is all about policies and mandates, committees and paperwork, without any real "rubber on the road" impact. I want to dispel this viewpoint by sharing a simple example of how one company implemented data governance to enforce something practical that delivered long-term
Data governance must encompass management of the full life cycle of a data policy – its definition, approval, implementation and the means of ensuring its observance - David Loshin, Data Policies and Data Governance I was checking out my Google stats on Data Quality Pro recently and observed that "How
We've witnessed a significant rise in data governance adoption in recent years. Careers, technology, education, frameworks, practitioners – there's growth in all aspects of the discipline. Regulatory compliance across many sectors is a typical driver for data governance. But I also believe one of the main reasons is the realisation by
Hadoop has driven an enormous amount of data analytics activity lately. And this poses a problem for many practitioners coming from the traditional relational database management system (RDBMS) world. Hadoop is well known for having lots of variety in the structure of data it stores and processes. But it's fair to
The demand for data preparation solutions is at an all-time high, and it's primarily driven by the demand for self-service analytics. Ten years ago, if you were a business leader that wanted to get more in-depth information on a particular KPI, you would typically issue a reporting request to IT
In my last post, I talked about how to observe the impact of modernisation through a data quality lens. I asked you to consider the quality of your legacy data and what that means on the "shiny new toy" you intend to buy in the future. In this post, I
At some point, your business or IT leaders will decide – enough is enough; we can't live with the performance, functionality or cost of the current application landscape. Perhaps your financial services employer wants to offer mobile services, but building modern apps via the old mainframe architecture is impractical and a replacement
Fellow Roundtable writer David Loshin has commented in the past that: "MDM is popular because it is presented as a cure-all solution to all data problems in the organization." Many people see master data management (MDM) as the silver bullet to all of their business and data woes. But in
Most companies are battling with master data challenges whether they realise it or not. When you're consolidating financials from multiple billing systems, you're doing MDM. When you're migrating legacy systems to a new target environment, you're doing MDM. When you're trying to perform root-cause analysis across multiple systems for a
We had just completed a four-week data quality assessment of an inside plant installation. It wasn't looking good. There were huge gaps in the data, particularly when we cross-referenced systems together. In theory, each system was meant to hold identical information of the plant equipment. But when we consolidated the
I'm frequently asked: "What causes poor data quality?" There are, of course, many culprits: Lack of a data culture. Poor management attitude. Insufficient training. Incorrect reward structure. But there is one reason that is common to all organizations – poor data architecture.
I recently presented a webinar (via the IAIDQ) on the topic of 7 Habits of Effective Data Quality Leaders. To prepare, I looked back at the many interviews of leading data quality practitioners I had undertaken over the years. A common trait among all these interviews stood out – they
In my last post, we touched on the importance of data migration in an overall data strategy. The reason I wanted to do this is because so many organizations see the migration of data as a technical challenge that can be outsourced and largely ignored by their internal teams. I contend
"I skate to where the puck is going to be, not where it has been." - Wayne Gretzky I love this quote from Wayne Gretzky. It sums up how most organizations approach data strategy. Data strategy typically starts with a strategic plan laid down by the board. The CEO will
Confusion is one of the big challenges companies experience when defining the data governance function – particularly among the technical community. I recently came across a profile on LinkedIn for a senior data governance practitioner at an insurance firm. His profile typified this challenge. He cited his duties as: Responsible for the collection
Working on a data migration project gives you a unique opportunity to learn where your organization has fallen short in its data management strategy. It's when you start to explore your legacy data landscape that you get a feel for how big a silo challenge your company has. It wasn't
No one knows for sure who coined the term Big Data. Despite etymological studies, we are still no closer to attributing provenance to any one person, or indeed any one period. Some say the term was coined in the '80s, others believe the '90s – and many are convinced the term originated
In the UK, technology trends move a little slower than for our US counterparts. It was about 5 years ago when I first met a data leader at a conference on this side of the pond who was actively engaging in large scale big data projects. This wasn’t a presenter
Many people who plan data governance initiatives ignore the need for a business case. "We've already had approval for the project; why do we need a business case when we've got the budget signed off?" The perception is that because they have a strong commitment, there is no need to get
If your organization is large enough, it probably has multiple data-related initiatives going on at any given time. Perhaps a new data warehouse is planned, an ERP upgrade is imminent or a data quality project is underway. Whatever the initiative, it may raise questions around data governance – closely followed by discussions about the
As consumers, the quality of our day is all too often governed by the outcome of computed events. My recent online shopping experience was a great example of how computed events can transpire to make (or break) a relaxing event. We had ordered grocery delivery with a new service provider. Our existing provider
One area that often gets overlooked when building out a new data analytics solution is the importance of ensuring accurate and robust data definitions. This is one of those issues that is difficult to detect because unlike a data quality defect, there are no alarms or reports to indicate a