Data quality issues don’t go away just because you have more data. Big data is sometimes considered exempt from the requirement to be integrated, cleansed and standardized. Unfortunately, chances are that the more data you have, the worse its quality will become.
There are many ways to do data integration. Those include: Extract, transform and load (ETL) – which moves and transforms data (with some redundancy) from a source to a target. While ETL can be implemented (somewhat) in real time, it is usually executed at intervals (15 minutes, 30 minutes, 1
Jim Harris addresses some of the most common questions and challenges big data poses for data quality.
.@philsimon on bridging the IT-business divide once and for all.
As a youngster in the 70s and 80s, Star Trek inspired my imagination and fostered a great love for science, technology and reading. (See the embedded Star Trek infographic for some interesting factoids – did you know that there were 28 crew member deaths by those wearing red shirts?) Captain Kirk and the
It’s been an amazing journey with Hadoop. As we discussed in an earlier blog, Hadoop is informing the basis of a comprehensive data enterprise platform that can power an ecosystem of analytic applications to uncover rich insights on large sets of data. With YARN (Yet Another Resource Negotiator) as its
.@philsimon on the new role of IT.
.@philsimon on the new challenges of an old problem.
In the UK, technology trends move a little slower than for our US counterparts. It was about 5 years ago when I first met a data leader at a conference on this side of the pond who was actively engaging in large scale big data projects. This wasn’t a presenter
Data integration, on any project, can be very complex – and it requires a tremendous amount of detail. The person I would pick for my data integration team would have the following skills and characteristics: Has an enterprise perspective of data integration, data quality and extraction, transformation and load (ETL): Understands
