Joyce Norris-Montanari explains why it's so important to pick the right tools to manage your big data.
Joyce Norris-Montanari poses the question: Is Hadoop/big data technology actually ready for MDM?
To show how they're compliant with regulatory mandates, organizations first need an enterprise data strategy. Joyce Norris-Montanari discusses the issues.
In the previous three blogs in this series, we talked about what metadata can be available from source systems, transformation and movement, and operational usage. For this final blog in the series, I want to discuss the analytical usage of metadata. Let’s set up the scenario. Let's imagine I'm a
As I discussed in the first two blogs of this series, metadata is useful in a variety of ways. Its importance starts at the source system, and continues through the data movement and transformation processes and into operations. Operational metadata, in particular, gives us information about the execution and completion
In the first blog of this four-part series, we discussed traditional data management and how we can apply these principles to our big data platforms. We also discussed how metadata can help bridge the gap of understanding the data as we move to newer technologies. Part 2 will focus on
Traditional data management includes all the disciplines required to manage data resources. More specifically, data management usually includes: Architectures that encompass data, process and infrastructure. Policies and governance surrounding data privacy, data quality and data usage. Procedures that manage a data life cycle from creation of the data to sunset
In Part 1 of this series, we defined data governance as a framework – something an organization can implement in small pieces. Data management encompasses the disciplines included in the data governance framework. They include the following: Data quality and data profiling. Metadata (business, technical and operational). Data security. Data movement within the enterprise.
Lately, the definitions of data governance and data management look very much alike. In this two-part series, we'll define data governance and data management. And we'll see that there's a big difference in the two.
In Part 1 of this two-part series, I defined data preparation and data wrangling, then raised some questions about requirements gathering in a governed environment (i.e., ODS and/or data warehouse). Now – all of us very-managed people are looking at the horizon, and we see the data lake. How do
I'm a very fortunate woman. I have the privilege of working with some of the brightest people in the industry. But when it comes to data, everyone takes sides. Do you “govern” the use of all data, or do you let the analysts do what they want with the data to
In my last post, I talked about how data still needs to be cleaned up – and data strategy still needs to be re-evaluated – as we start to work with nontraditional databases and other new technologies. There are lots of ways to use these new platforms (like Hadoop). For example, many
If your enterprise is working with Hadoop, MongoDB or other nontraditional databases, then you need to evaluate your data strategy. A data strategy must adapt to current data trends based on business requirements. So am I still the clean-up woman? The answer is YES! I still work on the quality of the data.
Today, I was in a conversation about using Hadoop (a big data platform) for master data management (MDM). I still find it amazing when we have the discussion of what systems feed another system. Many of our friends have spent years creating MDM for customer, product, etc. with success. I'm a
How many companies are using Hadoop as part of their master data management initiative? Come on, raise your hands! Well, maybe a better question is this: How many companies are using Hadoop for enterprise data? From what I have seen, Hadoop is coming along quite nicely. However, it may not be the
As I explained in Part 1 of this series, spelling my name wrong does bother me! However, life changes quickly at health insurance, healthcare and pharmaceutical companies. That said, taking unintegrated or cleansed data and propagating it to Hadoop may only help one issue. That would be the issue of getting the data
Does it upset you when you log onto your healthcare insurance portal and find that they spelled your name wrong, have your dependents listed incorrectly or your address is not correct? Well, it's definitely not a warm fuzzy feeling for me! After working for many years in the healthcare, pharmaceutical and
In the past, we've always protected our data to create an integrated environment for reporting and analytics. And we tried to protect people from themselves when using and accessing data, which sometimes could have been considered a bottleneck in the process. We instituted guidelines and procedures around: Certification of the data
As I explained in Part 1 of this series, creating a strategy for the data in an organization is not a straightforward task. Two of the most important issues you'll want to address in your data strategy are data quality and big data. Data quality There can be no data that is
Creating a strategy for the data in an organization is not a straightforward task. Not only does our business change – our software solutions also change before we can ever get done with a data strategy. So, I choose to understand that a strategy has a vision, and my vision may change
While setting up meetings with business consumers developing a data warehouse environment, I was involved in some very interesting conversations. Following are some of the assumptions that were made during these conversations, as well as a few observations. To get a well-rounded view of this topic, read my earlier post that focuses on the IT perspective.
The other day I was in a meeting with a client and there was an argument about who owns the data. Those arguing were IT people. In this scenario, the assumption was that data from source systems would flow into and integrate with a data warehouse. I found the discussion very interesting. Here are some of the
How many times have you gone onto a website, put a few things in a shopping cart, and then exited the Internet? I do it all the time. Sometimes when I log on to that site during my next visit, those same items are still in my cart – ready for purchase. I find
Most people have logged on to a social media site, maybe to look up an old friend, acquaintance or family member. Some people play games, or post funny pictures or other information they want to share with everyone. Do you ever ask yourself what happens with this information? What if your business wanted to purchase this information and
Twenty-five years ago (when I was 12 years old), we realized that data, across the corporation, was not integrated. Nor did our data let us predict the future by looking at the past. So we started creating these stores of historical data soon to be called “data warehouse.” Here are
The other day I was chatting with an ETL developer and he said he always pushes queries into the database instead of dragging data across the network. I thought “Hmm, I remember talking about those topics when I was a DBA.” I'd like to share those thoughts with you now.
I feel like I'm singing a song called Data in the Sky – With Options! The cloud is forever in our minds these days as a lower cost option because it requires fewer resources to address our data needs. Cloud solutions are an increasing part of many organizations' budgets every year. Whether enterprise data is
I don’t know about you, but I'm asked every day where some type of data lives in our enterprise. I keep thinking that we have not done a good job of helping people learn to help themselves! A few things I have learned about corporate data assets are: The data
There are many ways to do data integration. Those include: Extract, transform and load (ETL) – which moves and transforms data (with some redundancy) from a source to a target. While ETL can be implemented (somewhat) in real time, it is usually executed at intervals (15 minutes, 30 minutes, 1
Data integration, on any project, can be very complex – and it requires a tremendous amount of detail. The person I would pick for my data integration team would have the following skills and characteristics: Has an enterprise perspective of data integration, data quality and extraction, transformation and load (ETL): Understands