Why do we need Hadoop if we’re not doing big data?
Is Hadoop enterprise-ready?
Isn’t a data lake just the data warehouse revisited?
What are some of the pros and cons of a data lake?
We’ve already tackled the first three questions, and we’re now on question 4, so it’s time to talk about the data lake.
Question 4: Isn’t a data lake just the data warehouse revisited?
Some of us have been hearing more about the data lake, especially during the last six months. There are those that tell us the data lake is just a reincarnation of the data warehouse—in the spirit of “been there, done that.” Others have focused on how much better this “shiny, new” data lake is, while others are standing on the shoreline screaming, “Don’t go in! It’s not a lake—it’s a swamp!”
All kidding aside, the commonality I see between the two is that they are both data storage repositories. That’s it. But I’m getting ahead of myself. Let’s first define data lake to make sure we’re all on the same page. James Dixon, the founder and CTO of Pentaho, has been credited with coming up with the term. This is how he describes a data lake:
“If you think of a datamart as a store of bottled water – cleansed and packaged and structured for easy consumption – the data lake is a large body of water in a more natural state. The contents of the data lake stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples.”
As a customer intelligence adviser, my work exposes me to a wide range of organizations with various marketing challenges and available resources. Over time, some common themes have emerged, one of which is omni-channel marketing as a business imperative. Changes in the ways customers engage with brands across an explosion of channels have prompted the need for organizations to engage in omni-channel marketing.
Best practices are starting to emerge for mastering omni-channel marketing, and I've seen that they seem to fall into a five-step pattern, which I will lay out for you in this short blog series titled, Five steps to omni-channel marketing.
Step 3: From one-to-one marketing to event-driven marketing
It’s important to understand that marketing has changed due to the growth of connected devices. Customers are using more and more disruptive customer journeys to engage with you as an organization. These disruptive customer journeys are being influenced by different signals and events. When doing event driven marketing the goal is to process the triggers and use them for marketing campaigns. Read More »
As mass marketing becomes less common and effective, we get closer and closer to the ideal of the “segmentation of one,” which involves high degrees of personalization. In that environment, businesses must be able to market to customers at an individual level to remain competitive and relevant. However, without customer analytics technologies -- such as predictive modeling, data visualization, information management and segmentation -- marketing to this degree of detail can’t be done efficiently.
If you interrupt her, you'd better make it good.
Let’s discuss the why, how and what lies behind marketing to the segment of one.
Why would an organization want to market at this level? Doesn’t it seem a bit creepy or intrusive? The numbers show that the opposite is true. Numerous studies have proven that this type of interactive marketing is more effective, has much higher success rates, costs less to execute and generates more revenue than mass offers.
Mass marketing over traditional channels usually has a success rate of about 3 to 5 percent. Event triggered marketing -- which executes an offer based on a trigger or behavior being performed -- has a success rate of 10 to 20 percent. Right time marketing to the segment of one -- using context, data and analytics -- comes in at a 40 percent success rate.
When done correctly, segmentation of one allows for the delivery of an offer that is hard to refuse. It is so appropriate, relevant and timely that any brand that can deliver at this level of granularity, should, as they will have much higher response rates to the marketing campaigns and programs that they run. Read More »
It’s a fair question. Typically, marketers are more interested in the car (in this case, big data) than they are in the engine (Hadoop). But Hadoop is not just another faster, more cost-effective engine option. It’s a game changer in the world of data management—much like the Prius and Tesla have been in the world of gas-guzzling cars, trucks, and SUVs.
Do marketers need to understand how Hadoop works? Not at all. But what should interest them is if and how this popular big data technology can help them gain better and more informed insights about their customers. If (big) data can indeed help take the customer experience from a 3-star to a 5-star experience, then isn’t it worth understanding what all the Hadoopla is about?
This dovetails nicely into our 3rd question in this 5-part series. My answer will be short—and it may surprise you.
Question 3: Is Hadoop enterprise-ready?
I have two answers to this question:
For your organization: Maybe.
For all organizations: No.
It all depends on what and why you want to use Hadoop in your organization. If you simply want to use it as an additional (or alternative) storage repository and/or as a short-term data processor, then by all means, Apache Hadoop is ready for you. (My last post discusses six ways Apache Hadoop can be used.) Read More »
How often do you give much thought to your card swipe? It’s become so commonplace that I doubt you think of it at all – and that’s how it's supposed to be. Fast, easy and a complete non-event so you can be on your way in no time at all - often just a second before you’re prompted for your signature, or directly given your receipt.
Look at what I found in my wallet!
What’s amazing is to find out what happens in that second between the swipe and the receipt – funds are verified, the transaction is analyzed for fraud risk, an approval is rendered and recorded and a transaction fee is applied. And the whole transaction is a simultaneous customer experience for you, the merchant, the financial institution and the payment processor, such as Visa.
Visa has been a pioneer in cashless payment processing for over 50 years and has grown to become one of the world’s most recognized brands. And one of the drivers of their recognition has been their ability to provide a consistent and secure transaction millions of times throughout the world around the clock. One of the most important priorities has been to operate on that real-time basis and maintain the integrity of the transaction - every time.
At any given time and in any given transaction, Visa analyzes up to 500 unique variables in real time to assess the risk of that transaction. With analytics and by using vast data sets, including global fraud hotspots and transactional patterns, the company can more accurately assess whether your transaction is legitimate or if someone stole your credit card. And that’s all done in real-time! Visa estimates that analytics has the potential to prevent an incremental $2 Billion of fraudulent payment volume annually.
For more details on this fascinating story, tune in to the short video below featuring Visa executive Nathan Falkenborg describing the impact that analytics has on the card-swiping customers’ experience.
It’s a clichéd rom-com scene: The couple is sitting at a table in clearly awkward silence. Finally, one of them announces that the relationship is over and in doing so delivers the classic line “It’s not you, it’s me.” And what I always hear in those scenes is “It’s not me, it’s you. Or if it is me, it’s me blaming you.”
You could be this comfortable with your vendor.
So how could the couple avoid getting to that point?
While every relationship is different, failure usually can be avoided, and Marketing Operations Management (MOM) offers some valuable lessons. As the veteran of many MOM implementations, I have seen enough situations crop up that could escalate to failure to know that the causes are usually avoidable. And seldom will a customer tell the service provider, “It’s not you, it’s me." But I digress. Assigning fault seldom is productive, so when possible it's best to have all parties focus on the same commitment to see the implementation through to success.
I will leave the relationship advice to others, but I can offer you four key ways your organization can significantly increase your MOM implementation chances for success.
Know the problem you’re trying to solve I have been involved in many sales cycles and implementations where the customer wants the software but has no idea what they’ll use it for. This is akin in a relationship to hoping your partner is going to “fix” you, even if you aren’t aware of what your own issues are. Read More »
In just five or so short years, we have all seen social media as a channel grow up right before our eyes. It's no longer that infant that everyone was afraid to touch or be around - it's now that fun kid that people want to engage with. Some may say, well social media is older than just five - Facebook was formed in 2004 after all - but for practical purposes I would say that social wasn't considered by businesses as a marketing medium until the 2010 time frame.
With the growth of social media, we have also seen the change in thinking with regard to social media. Brands, because they are much more comfortable with the idea of using social media for marketing purposes, don't need the level of education that some primer content provides. Here at SAS, we published a primer paper back in 2010 with the Harvard Business Review (HBR) titled - "The New Conversation: Taking Social Media from Talk to Action." This paper has proven extremely popular, so below I've pulled out the main points for folks that may not be interested in reading the entire piece. Surprisingly, a lot of the content is still relevant in today's world, though all references below are in terms of 2010:
The power of social media is shown by 79% of the 2,100 companies who participated the HBR survey said they were currently using social media or preparing to launch social media initiatives at that time.
88% of companies surveyed admitted they had not used social media up to its full potential. Furthermore, they were not using social media effectively to not only listen to, but analyze consumer conversations and integrate those insights into their strategies. Two examples show how using social media insights was still not widely used:
25% of social media users said they could identify where their most valuable customers were talking about them, and
23% were using any form of social media analytic tools. This could be due to the perception by some at the time of social media as “dangerous” because of the exposure and lack of control of web conversations.
The survey asked companies who use social media what they perceived to be the primary benefits social media brought to their organization. Read More »
Know if this assortment works at other locations with analytics.
Retailing holds great lessons for marketers in all industries because as long as there have been customers, retailers have focused on the customer experience. And one of the biggest elements of the customer experience is assortment - literally addressing the customer's need for variety or choice. Think of a time when you weren't exactly sure of what you were looking for - didn't you go first to places you knew had a wide variety? That's assortment.
How do retailers manage assortment? When you layer on the challenge of multiple locations, you can see how the issue quickly expands. In order to deliver a consistent customer experience, many retailers maintain a standardized layout or “floor set” among their locations - either keeping the same assortment in all locations, or varying by location. Neither approach is always better than the other, but in either case the question is how to best vary the product offerings to deliver the best customer experience.
That concept is also known as localization, and it can be quite a challenge without analytics. Early in my career, I would pull category level sales by location and put it into a spreadsheet. Then I'd sort it and make a note of the top locations by category and use that for decision making. I now know how inadequate that was. To complete the picture for you, when an order for a specific category arrived in to the distribution center, I would refer to one of the hundred sticky notes attached to my computer monitor and manually make sure that the locations received the product.
Not only was my little process painstaking, it fell short in accuracy, it was clearly inefficient, and it definitely was not scalable. With that old approach, a top volume location would likely end up on the top of the list for every category, but a lower volume store may not even get a sticky note. Sticking strictly to my sticky notes would mean ignoring the need to create a great customer experience in every location, regardless of sales volume. Quite a problem, right? But it's not insurmountable. Read More »
In the movie Minority Report, while the leading actor walks through the mall and experiences personalized greetings all around him, there is a clear flash of how the future of marketing may look: a customer journey marked by relevant and personalized experiences.
Getting back to the reality of today, the typical customer journey represents various customer interactions with your brand over time across all the digital and offline channels. When they are done right, each of these touch points builds on the others to play an important role in bringing your customer closer to choosing your brand over others.
The customer journey: what it takes It’s the “when they are done right” part of that scenario that requires a great deal of effort, particularly with regard to:
Personalization: Increasing response rates with uniquely tailored content, delivered instantly.
Automation: leveraging content across channels and doing it consistently.
Optimization: using real-time analytics to drive the best possible outcomes for each customer and for your organization every time.
Innovation: Finding ways to rise above the noise by promising – and delivering - compelling user experiences.
"Our corporate data is growing at a rate of 27% each year and we expect that to increase. It’s just getting too expensive to extend and maintain our data warehouse.”
“Don’t talk to us about our ‘big’ data. We’re having enough trouble getting our ‘small’ data processed and analyzed in a timely manner. First things first.”
“We have to keep our data for 7 years for compliance reasons, but we’d love to store and analyze decades of data - without breaking the machine and the bank.”
Do any of these scenarios ring a bell? If so, Hadoop may be able to help. In this 5-part blog series, Big Data Cheat Sheet on Hadoop, we’re taking a look at five big data questions from the perspective of a marketer. This post answers the second question in the series to help marketers understand how these big data technologies are impacting (or can impact) the customer experience, and what you can do to take advantage of this data playground.
Question 2: Why do we need Hadoop if we’re not doing big data?
Contrary to popular belief, Hadoop is not just for big data. (For purposes of this discussion, big data simply refers to data that doesn't fit comfortably – or at all – into your existing relational systems.) Granted, Hadoop was originally developed to address the big data needs of web/media companies, but today, it's being used around the world to address a wider set of data needs, big and small, by practically every industry.
Welcome to Customer Analytics, a blog for anyone who is looking for ways to improve the business of marketing and communicating with customers.
We strive to prompt new thinking in the way you tackle customer-related business issues. And we hope to inspire the use of analytics for everything from multi-level marketing to social media campaigns. Follow us here and on Twitter at @SAS_CI, or check out the Twitter hashtag #sasci.
I’m John Balla, Editor of the Customer Analytics blog and Principal Marketing Specialist in Customer Intelligence. Read more about me here.