In the last post, we talked about creating the requirements for the data analytics, and profiling the data prior to load. Now, let’s consider how to filter, format and deliver that data to the analytics application. Filter – the act of selecting the data of interest to be used in the
One area that often gets overlooked when building out a new data analytics solution is the importance of ensuring accurate and robust data definitions. This is one of those issues that is difficult to detect because unlike a data quality defect, there are no alarms or reports to indicate a
In the last few days, I have heard the term “data lake” bandied about in various client conversations. As with all buzz-term simplifications, the concept of a “data lake” seems appealing, particularly when it is implied to mean “a framework enabling general data accessibility for enterprise information assets.” And of
As part of two of our client engagements, we have been tasked with providing guidance on an analytics environment platform strategy. More concretely, the goal is to assess the systems that currently compose the “data warehouse environment” and determine what the considerations are for determining the optimal platforms to support
“Begin with the end in mind” - Habit #2 from Stephen Covey’s ‘Highly Effective People’. The Edge Foundation is based on the premise of: “To arrive at the edge of the world's knowledge, seek out the most complex and sophisticated minds, put them in a room together, and have them
You may feel like the world is moving faster than ever. If so, then you can take solace in two facts: You're not alone in feeling this way. You're right. It is. Celebrating the 25-year anniversary of the Web, The Economist ran a piece examining the increasingly rapid adoption of new technologies.
As utilities expand analytic capabilities into more areas of the business, the reality of the data management challenge becomes very real. Most have accepted the era of "big data." But what about the quality of that big data? Is it reliable? What about the governance? Have the processes changed since
It’s great to get in on something on the ground floor. That’s what happened at the inaugural Data4Decisions Conference and Exhibition held in Raleigh, NC, in March. It brought together business people, academics and students to explore how organizations use data management and analytics technology to enhance business processes and
In my last blog I detailed the four primary steps within the analytical lifecycle. The first and most time consuming step is data preparation. Many consider the term “Big Data” overhyped, and certainly overused. But there is no doubt that the explosion of new data is turning the insurance business
As a data scientist, I have the rare privilege of possessing the job title that Tom Davenport and others have dubbed the sexiest job in the 21st Century. As this popular job title catches on, I’ve even noticed a trend where customers make direct requests for help specifically from “the data
The other day, I was looking at an enterprise architecture diagram, and it actually showed a connection between the marketing database, the Hadoop server and the data warehouse. My response can be summed up in two ways. First, I was amazed! Second, I was very interested on how this customer uses
In today’s era of digital marketing, advertisers have access to innovative tools and platforms, which enable them to provide Internet users with more personalized ads. Furthermore, in exchange for such helpful and free service, people do not mind sharing a little bit of personal information as it helps them find
In the Cold War techno-thriller WarGames, a marine monitoring a nuclear missile silo deep under the Nevada desert sees a red warning light blink on his console. “Just flick it with your finger,” his colleague tells him. He does, and the bulb goes out. Problem solved. But what will their
.@philsimon looks under the hood of 'analytics.'
Downstream data have been electronically available on a weekly basis since the late 1980s. But most companies have been slow to adopt downstream data for planning and forecasting purposes. Let's look at why that is. Downstream data is data that originates downstream on the demand side of the value chain. Examples
As the point person for SAS joining the new Open Data Platform (ODP) initiative, I want to make it clear why SAS is involved with ODP, and why we think it’s important to our customers, and the Hadoop and big data ecosystem as a whole. SAS is not in it to
Hadoop is increasingly being adopted as the go-to platform for large-scale data analytics. However, it is still not necessarily clear that Hadoop is always the optimal choice for traditional data warehousing for reporting and analysis, especially in its “out of the box” configuration. That is because Hadoop itself is not
Data Management has been the foundational building block supporting major business analytics initiatives from day one. Not only is it highly relevant, it is absolutely critical to the success of all business analytics projects. Emerging big data platforms such as Hadoop and in-memory databases are disrupting traditional data architecture in
Business analytics is about dramatically improving the way an organization makes decisions, conducts business and successfully competes in the marketplace. At the heart of business analytics is data. Historically, the philosophy of many insurers has been on collecting data, data and more data. However, even with all this data, many
Back in the day when the prison system forced inmates to perform "hard labor", folks would say (of someone in prison): "He's busy making little ones out of big ones." This evokes the cliché image of inmates who are chained together, forced to swing a chisel to break large rocks
This isn't Kansas anymore. Oz has become a sprawling, smart metropolis filled with sensor data. How do we make sense of, clean, govern and glean value from this big data so we can get Dorothy home? The answer is SAS Data Management. With the latest portfolio updates, customers will be
I cannot speak from experience, but predominately an Insurance CEO has three primary objectives: Grow the business Reduce expenses Ensure compliance. Let’s individually consider each of these objectives in more detail. Grow the Business How does an insurance company grow from a $2bn to a $3bn organization? Essentially, insurance has
In an era long gone by (actually not so long ago) we all interacted with computers via terminals to a mainframe or minicomputer systems. Sometimes, you had to book a slot for when you could access and exploit computer resources. The user was subject to interrupted connections or very poor
Data. It's everywhere. It can reside in many places through replication, accessibility needs or infrastructure costs. For reporting, that same data can be structurally changed (denormalized or aggregated) into additional reporting and analytic data repositories. Over time, new sources of enrichment of that data become available through traditional data sources
Getting universal buy in for Hadoop needn’t be an uphill struggle. In many cases, it only takes one pilot project to realize the benefits of low cost storage combined with powerful analytics. The Hadoop topic provoked passionate conversatoin at a recent roundtable discussion attended by over 25 people from a range
Data. Our industry really loves that word, making it seem like the whole world revolves around it. We certainly enjoy revolving a lot of words around it. We put words like master, big, and meta before it, and words like management, quality, and governance after it. This spins out disciplines
For many industries, big data analytics have opened numerous doors for more employees to be groundbreaking and to challenge the corporate status quo. Prior to big data technologies, risk taking behaviors were primarily reserved for provocative souls who stretched organizational boundaries to disrupt industries, such as airline revenue management. There were winners and losers
With big data, data governance challenges escalate in many ways: The diversity of data sources means that there are minimal standards for data structure, definition, semantics and content. The lack of control over data production means that you can’t enforce data quality at the source as you can do with
Marketers are walking a tightrope today with data privacy issues: Data can simultaneously bring customers and brands together and further drive them apart. Recent data breaches, potential changes in data-privacy legislation and regulations loom large as customer expectations concerning marketing data continue to rise. As a result, today’s complex data
At a recent MESA - Hollywood IT Society Digital Marketing conference, Claudio Ludovisi, SVP, Operational Strategy NBCU Entertainment for Marketing & Digital took a live poll of about 200 or so media marketing executives in the audience and asked them to “Think about what it takes to achieve true multiplatform
