Get the scoop on data lake vs cloud
Jim Harris takes a deep dive into data lakes and how they relate to the cloud.
Tag: Data Lake
3 tips for building a better data lake
Guest blogger Khari Villela shares tips to help you skip common pitfalls of building a data lake.
Reference data and data governance
David Loshin suggests that reference data could be the foundation of future governance structures.
Are data lakes stuck in the black and white world of the big data infomercial?
Guest blogger Khari Villela says data lakes are not a cure-all – they're just one part of a comprehensive, strategic architecture.
Data lake management for analytics
Jim Harris warns against allowing your data lake to become a poorly managed and ungoverned data dumping ground.
Data lake governance – Do you need it?
Kim Kaluba gives examples of the benefits of data governance for data lakes.
Data on demand or data as a service?
David Loshin describes how data as a service supports fast deployment, easy accessibility and data reuse.
How to use a data catalog to get more value from your data lake
David Loshin explains how to set up a data catalog that will help you get more value from a data lake.
How do I know what data is available?
Phil Simon shares his thoughts on this simple yet often-overlooked question.
On the way to data lakes and Hadoop – Part 2
In part 2, Jim Harris explains more about why you should address data quality and governance issues on the way to data lakes and Hadoop.
On the way to data lakes and Hadoop – Part 1
Jim Harris advocates addressing data quality and governance issues on the way to data lakes and Hadoop.
Managing data where it lives
Historically, before data was managed it was moved to a central location. For a long time that central location was the staging area for an enterprise data warehouse (EDW). While EDWs and their staging areas are still in use – especially for structured, transactional and internally generated data – big
Big data allows us to rethink data strategy
Start with the end in mind -- wise words that apply to everything, and in the world of big data it means we have to change the way we look at managing the data we have. There was a time when we managed data quality, and the main goal was
Data preparation and data wrangling, Part 1 (yippee, bring your lasso)
I'm a very fortunate woman. I have the privilege of working with some of the brightest people in the industry. But when it comes to data, everyone takes sides. Do you “govern” the use of all data, or do you let the analysts do what they want with the data to
Crowdsourcing data assets in the data lake
A long time ago, I worked for a company that had positioned itself as basically a third-party “data trust” to perform collaborative analytics. The business proposition was to engage different types of organizations whose customer bases overlapped, ingest their data sets, and perform a number of analyses using the accumulated
A modernized approach to data lake management
In my last post, I started to look at the use of Hadoop in general and the data lake concept in particular as part of a plan for modernizing the data environment. There are surely benefits to the data lake, especially when it's deployed using a low-cost, scalable hardware platform.
Leveraging the data lake concept for data modernization
More and more organizations are considering the use of maturing scalable computing environments like Hadoop as part of their enterprise data management, processing and analytics infrastructure. But there's a significant difference between the evaluation phase of technology adoption and its subsequent production phase. This seems apparent in terms of how organizations are
Hadoop und SAS – Frischwasser für Data Lakes
Ein Data Lake ist ein Konzept zur Speicherung von Daten in einem Hadoop-Cluster. Es entstehen heutzutage an vielen Stellen Daten, welche aus Kostengründen nicht ins klassische Data Warehouse fließen. Doch könnten mit diesen Daten zusätzliche Assets generiert werden, vorausgesetzt man speichert sie an einem Ort und hat dann eine analytische
Fields and dreams of data warehousing
“Field of dreams warehouse”– a historic phrase I used in the early days of data warehouse development. It describes the frenzy of activity that took place to create enterprise data infrastructure, before the business rationale for the data use was even understood. Those were the early days. In some ways
Big data modeling: An iterative approach - Part 1
In my prior two posts, I explored some of the issues associated with data integration for big data and particularly, the conceptual data lake in which source data sets are accumulated and stored, awaiting access from interested data consumers. One of the distinctive features of this approach is the transition
Data integration considerations for the data lake: Standardization and transformation
In my last post, I noted that the flexibility provided by the concept of the schema-on-read paradigm that is typical of a data lake had to be tempered with the use of a metadata repository so that anyone wanting to use that data could figure out what was really in
Data integration considerations for the data lake: The need for metadata
A few of our clients are exploring the use of a data lake as both a landing pad and a repository for collection of enterprise data sets. However, after probing a little bit about what they expected to do with this data lake, I found that the simple use of
Business needs and performance expectations: Data management for analytics
In the last few days, I have heard the term “data lake” bandied about in various client conversations. As with all buzz-term simplifications, the concept of a “data lake” seems appealing, particularly when it is implied to mean “a framework enabling general data accessibility for enterprise information assets.” And of
EMC and SAS redefine big data analytics with the data lake
Adoption of Hadoop, a low-cost open source platform used for processing and storing massive amounts of data, has exploded by almost 60 percent in the last two years alone according to Gartner. One primary use case for Hadoop is as a data lake – a vast store of raw, minimally processed data. But, in many ways, because
Swimming in a lake of confusion: Does the Hadoop data lake make sense?
All hail the data lake, destroyer of enterprise data warehouses and the solution to all our enterprise data access problems! Ok – well, maybe not. In part four of this series I want to talk about the confusion in the market I am seeing around the data lake phrase, including
Three ways to use a Hadoop data platform without throwing out your data warehouse
Working out where Hadoop might fit alongside, or where it might replace components, of existing IT architectures is a question on the minds of every organization that is being drawn towards the promises of Hadoop. That is the main focus of this blog along with discussions of some of the reasons they