High performance does not have to equal high cost

Jim Davis
Jim Davis|December 10, 2012

What does it cost to tackle big data? Do you need to make a major investment to build a high-performance architecture? The short answer is no. For a longer explanation, I’m going to start with a question from a comment on one of my earlier posts.

On the post, What kind of big data problem do you have? a reader asks:

You mention that the four types in the quadrant each do different things and they each require different architectures. Could you please elaborate that a little? And what should be the best way for a company to move from lower right quadrant to upper left quadrant considering the costs? Would appreciate your insight.

The first part of the answer is that you should think about what it is you’re trying to accomplish with your decision making, and what challenges you are trying to overcome with your infrastructure. Ask yourself what is driving your move from one stage to the next:

  • Is it the fact that information’s not accessible?
  • Are you moving from analyzing past activity to predicting future activity?
  • Or are you just taking too long to process data and finding it hard to get results in a timely fashion?

If you are satisfied with your BI architecture but realize that your data is growing and you need quicker response times, then you should look at architectures to support quicker extraction of data. Believe it or not, upgrading your architecture to increase throughput and improve speed to decision making is no longer an expensive proposition.

The architectures are actually similar in the lower right, upper left and upper right quadrants. The lower left is a more traditional BI architecture: You have a database and a query and reporting tool that sits on top of it. You may get reasonable response times but as your data grows and you want to reduce response times, you’re moving to the lower right quadrant. To cut the response time, you have to start looking at significant changes in your architecture.

Today, when we talk about upgrading your architecture for these purposes, we’re not talking about expanding mainframe capital, building more databases or adding large UNIX servers. Instead, we’re talking about a blade architecture based on commodity hardware that is relatively inexpensive. With that change, increased performance by 1,000 times is not unusual.

If you want to start small but see tremendous gains in performance, there’s an affordable entry point in the lower right and upper left quadrants. That is the single-box approach based on a symmetric multiprocessing environment.  You can have 50 users on a $10-20,000 box. It’s not a matter of jumping in all the way and spending a tremendous amount of money.

To decide between those two quadrants, you should look again at what your business is trying to accomplish. Do you want to get a jump on the competition by predicting future aspects of the business quickly? That’s where big analytics and big data analytics can provide the most benefit. They can offer predictive results in seconds instead of hours. When you’re looking for these types of answers, it’s not just a hardware or software issue. You also have to look at whether or not the business is capable of supporting that activity. Modelers and data scientists are needed. You can do it yourself, or host it in the cloud and partner with a vendor to develop and maintain the models. It’s the same blade architecture in the cloud and you can work with the vendor to bear the burden of the operational activity as opposed to buying and maintaining the hardware and models yourself.

The bottom line is that today’s architectures are making it easy to start doing big analytics in a relatively inexpensive way. You can start small, prove value and then work on spreading the results quickly throughout the organization. From there, you can begin to demonstrate what’s possible and build the groundswell to see what’s capable more broadly with big data and big analytics throughout the organization.

Post a Comment

What is the real reward of a great work place?

Jim Davis
Jim Davis|November 14, 2012

My last few posts here have talked about analytics and the role of analytics in changing and improving the organization. Of course, none of that – from data to change – would be possible for a company that doesn’t appreciate its assets. We talk a lot about the importance of data as an asset, but employees are an important asset too.  At SAS, our CEO has been famously quoted to say, “95 percent of my assets drive out the gate every evening.”

As a result of this mindset, SAS has once again been honored as a great place to work. This year, we’re at the top of the "25 World's Best Multinational Workplaces" list from Great Place to Work®.

Why are these types of rewards important? Because they honor a part of the business that is easy to ignore but essential for success. Really, it’s a shame that more companies can’t realize the benefits of cultivating the relationship between the company and the employee. We all spend a lot of time talking about how employees should treat customers well, but what about how the company treats employees?

When Wall Street evaluates a company, is employee trust part of the equation? Maybe it should be. If you look at it solely as a line item, employee satisfaction programs can be expensive, and the bottom line benefits of treating employees well are hard to measure, but you have to factor in more than the costs. Happy employees contribute to customer loyalty, product quality and low turnover rates, for example.

But isn’t it hard to win one of these awards? No. It really isn’t, if you recognize that people are your most important asset. That sounds like a cliché but if you treat people like they are an important asset, then other dimensions will fall into place. Product quality increases, customer service improves, you become an employer of choice and attract more talent. If you respect the differences between people and treat them as an asset, these awards will appear. It’s not like you’re out looking for them. They happen.

That doesn’t mean this is an idea you can just pay lip service to. It has to be at the core of what your organization believes. The older I get, the more I realize that if you try to set a path to be something you’re not, it will be a struggle and you’ll never succeed. If you’re true to yourself, however, and follow the path that aligns with who you are, success will come naturally.

In the case of SAS, our CEO Jim Goodnight made a commitment to being a great place to work since the day the company was founded in 1976. He has always valued employees, and employees have always come first. It’s at the core of who we are at SAS. This workplace award is just one more indication of that core value, but the long-term benefits of  healthy employees, happy customers and quality products are even greater.

Post a Comment

Our desire to know

Jim Davis
Jim Davis|November 9, 2012

… there are rare occasions when history matches the complexity of the world with just the right technology, innovation and leadership …

Do you think analytics is reserved solely for big businesses and data scientists? Or do you see it as something that can change the world?

Do you realize that analytics extends to endangered species, disaster relief and life-saving medicines?

SAS has been the leader in analytics for 36 years. In that time, we’ve seen analytics evolve from purely an academic environment to become a mainstream technology in the financial services, retail and other industries around the world.

In this powerful video, you can experience the effects of analytics on your life and the lives of others around the world.

Post a Comment

Turn marketing into a creative, analytical process

Jim Davis
Jim Davis|November 2, 2012

Just a quick post this morning to draw your attention to a smart marketing article you don’t want to miss. It’s written by my colleague Adele Sweetwood, and (according to Twitter) it was one of the most popular articles on Marketing Profs this month.

In the article, Six Tips for Creating an Analytics-Driven Marketing Culture, Adele advocates for an analytical culture in marketing that does not compromise creativity. You may be thinking that creativity and analytics live on opposite ends of the spectrum, but that’s not true. There can be science in marketing without losing creativity.

When we discussed big data analytics here a few weeks ago, we talked about calculating high-performance marketing optimization jobs in seconds rather than hours. Marketing is a business process. And this is another good example of how analytics can play a part in every process. Speeding up that business process actually brings in more opportunities for creativity and collaboration around marketing. As a result, you have more time for creativity, and you can learn how to apply your innovative ideas where they will bring the most benefit to the organization.

While there’s no direct path or big bang approach to automated, digital or analytic marketing you can find great tips for getting started in Adele’s article. Check it out and let us know how far along you are in the process. How are you combining analytics and creativity to see results? What have you been able to accomplish in terms of bringing analytics to bear on your marketing operations?

Post a Comment

Q4 2012 Intelligence Quarterly: Public sector innovation

Mikael Hagstrom
Mikael Hagstrom|October 26, 2012

Has the public’s tolerance for austerity reached the tipping point of social unrest? Is having an increasingly large portion of public resources spent on financing debt rather than education and health anti-social? Has the public’s confidence and trust in public reforms and government officials eroded beyond repair?

You might feel that we are caught in the doldrums, like the Portuguese sailors of old, desperately trying to catch the winds of trade. The question is: How do we enter the trade winds and bring inclusive growth back to our societies?

Part of the answer lies in treating data as the new asset class that it is and facilitating a new, technology-driven transformation of society, one that will foster inclusive growth and offer a brighter future for people all ages. It starts with using technology and big analytics as a platform of innovation to foster trust, confidence and transparency while equipping all parts of society with the new abilities now available in the information age.

The latest issue of Intelligence Quarterly sets out to highlight early examples of how big analytics has been used as an enabler to not only escape the state of despondency we find ourselves in today, but to also build the platform we so badly need for inclusive growth. In its pages, you will find concrete examples of projects that have helped build confidence and trust by eradicating tax evasion, helping create meaningful jobs and improving the quality of health care, touching on just a few areas where improved social fairness, inclusion and growth are needed.

We all know that 1.5 to 2 percent GDP growth is not sustainable in a world with a rapidly growing and aging population. I invite you to see how we can shape that growth with big analytics. It is humbling to take part in social transformation, shaping a future where regulations can be designed and measured by their outcomes – a world where young talent can realize their dreams while being part of the solutions. Never has the opportunity to build a better future been greater.

The State of Europe

On a related topic, I recently participated in a policy summit, The State of Europe: Escaping the Doldrums, with other European leaders such as European Commission President José Manuel Barroso, Deputy Director Guntram Wolff and European Council President Herman Van Rompuy. The roundtable focused on the growth of Europe and how Europe can restore its role in the global economy. You can hear a brief summary of the meeting in a 3-minute video provided by Voice of America.

Post a Comment

What kind of big data problem do you have?

Jim Davis
Jim Davis|October 8, 2012

Does it seem like almost everything is a “big data” problem right now? And nearly every vendor is offering big data or big analytics solutions? Is big analytics more important than big data? And what is the difference? I've encountered this confusion in the market a lot over the last year as I’ve traveled the globe talking to business and government leaders about big data.

In the process of explaining the market to others, I've come up with a clearer way to understand the landscape. This explanation has helped a lot of businesses understand what type of analytic problems they actually have, and sometimes it helps them see that their problems are more of the big analytics variety instead of the standard issue of big data alone.

Sometimes, for example, you don’t have that much data but it’s still taking you five hours to run a marketing optimization job because of the number of possible offers. There really aren’t a lot of records but you have to do multiple passes on the data, running complex algorithms with each step. That’s a big analytics problem and not just a big data problem.

Let’s dig into those differences a bit further.

Our first step is to revisit the distinction we’ve made over the years between reactive and proactive analytics. Standard business reports, ad hoc reports, OLAP and even alerts and notifications based on analytics are in the reactive category. Now, reactive analytics can still be very useful. They’re required for a lot of finance and regulatory reporting, and they help business users perform ad hoc analysis every day, but they are ultimately informing you about the past.

Proactive analytics like optimization, predictive modeling, forecasting and statistical analysis, however, are forward looking. They allow you to identify trends, spot weaknesses or determine conditions for making decisions about the future. They include optimization of complex problems with many dependencies, predictive modeling, regression analysis and other advanced methods for proactive decision making.

FIGURE 1: Reactive and proactive analytics

The next thing we need to define is big data. Put simply, when you have exceeded the capacity of conventional database systems, you’re dealing with big data. Before that, it’s what I like to call “growing data." It is still a large amount of data but it hasn’t hit the limitations seen with big data.

Today, we can store lots and lots of data but processing times have become excessive because traditional storage environments are not conducive for proactive analytics. When you have reached a point where processing times become unacceptable, you may be dealing with big data sizes but you may also be dealing with big analytics.

To better understand the difference, let’s create a chart with reactive and proactive analytics on the Y axis and the size of the data on the X axis, like this:

Figure 2: Data size and analytic competence

 

Now we can see the four major types software solutions available in the analytics market today. They are:

Business Intelligence (BI). If you are dealing with a large amount of data and providing reporting capabilities for end users so they can gain access to information, summarize data and even drill down into that data themselves, you are dealing with business intelligence applications. These solutions provide a strong look at various performance aspects of the company that occurred in past. That is BI.  That is the lower left quadrant in Figure 2.

Big data BI. Now, when data gets bigger and you’re dealing with outside data sources or – as more companies are starting to see – you’re pulling in unstructured data, your problems are getting bigger. It’s taking users too long to get the information they need, or you’re having a hard time combining data sources fast enough to provide reports like you used to and you need technology that allows quick access to data – but you’re still providing reactive analytics. This is the most common big data scenario in the market right now, and most businesses are trying to solve this with SQL based solutions. That is big data BI. It is in the lower right quadrant of Figure 2.

Big analytics. As I mentioned before, it takes a different kind of analytics to support forward looking decisions. If you’re looking at customer preferences, markdown optimizations or fraud predictions, you need a different type of architecture. These problems typically involve growing data sizes and proactive analytics. Instead of the data size slowing you down, it’s the fact that you’re making multiple passes on data that may take hours and hours to get results, and you’re running advanced analytic calculations that take longer to process. Today, you need those answers in seconds or minutes. This is big analytics. It is located in the upper left quadrant of Figure 1.

Big data analytics. Now, what about organizations that have a whole lot of data and are dealing with proactive decision making? Here, we’re talking about hundreds of millions of SKUs across multiple retail stores. We’re looking at future sources of data too like telematics data in the auto industry, which can be useful for manufacturers and insurers. These are the types of problems most businesses really haven’t dealt with in past. And these aren’t small data problems. You don’t want to summarize that information. Manufacturers want to be predict safety problems before they impact customers and insurance companies want to adjust rate plans for the best drivers, for example. This is big data analytics. You’ll find it in the upper right corner of Figure 2.

My point here is not to say that one is better than the other, but they each do different things and they each require different architectures. As you look at what’s going on the market and in your business, understand the difference between each of these four areas and how the different problems can be solved.

Analytics continues to be a broad term in market but it’s worthwhile to look at the problems you are trying to solve and determine where you fall in this landscape. It will help determine what your next steps are in your big data journey.

I’ll be presenting these concepts in more detail later this week at The Premier Business Leadership Series. If you’re attending, stop by after the presentation and let me know if this is a useful breakdown for you. I’d love to hear your thoughts.

Post a Comment

Q3 2012 Intelligence Quarterly: Fraud is everyone's problem

Mikael Hagstrom
Mikael Hagstrom|August 10, 2012

Intelligence Quarterly - 2012 Q3 issueWe have yet to recover from the economic crisis. But the turmoil seen in the struggling global market could yet be dwarfed by the utter devastation a cyber-crisis could unleash. As a company, SAS is frequently asked to support compliance efforts, such as FATCA (Foreign Account Tax Compliance Act), but our work extends beyond this, encompassing areas like fraud detection and deterrence.

Fraud is everyone’s problem and cannot be ignored, especially in the growing area of digital crime. Organizations of all kinds and from all sectors are already seeing an increase in fraud, waste and abuse – a trend accelerated by today’s turbulent economy. It may come as no surprise that most fraud goes untried or, in many cases, undetected. Costs are then passed on to customers, constituents or owners in the form of higher fees, increased taxes or lower margins.

It doesn’t have to be this way. As you’ll discover in the latest issue of Intelligence Quarterly, many governments and businesses today are using data and analytics – particularly big data and high-performance analytics – to build fraud prevention technologies that save money, protect their corporate reputation and reduce process costs. With analytics, it is possible to shift from detect-and-defend mode to the holy grail of predicting fraud to prevent it, thereby eliminating fraud in your organization.

Are your fraud prevention efforts keeping up with the level that social standards call for?

Post a Comment

How to transform big data from an obstacle into an asset

Jim Davis
Jim Davis|July 22, 2012

Talking about "big data" in Asia

In your organization, is “big data” an obstacle or an asset? Or is it still just a buzzword you’re trying to understand? A small group of us from SAS have spent the last week traveling to Mumbai, Singapore, Hong Kong, Seoul and Tokyo talking with business leaders about the benefits of using big data to improve their organizations.

One thing that seems to be resonating with these global business leaders is the idea that “big data” can become an asset when you apply high-performance analytics to an entire business process, not just a single application.

Let me use marketing as an example.  I’ve been browsing through this series of big data articles from CMO.com, and a couple of them are really good, but I keep thinking about the fact that marketing is not an application. It is a process. If you want to take advantage of big data within marketing, you need to look at the whole business process:

  1. Explore data. Can you imagine actually looking at a billion rows of data? How can you drill down to know which variables are important or relevant? You need a visual exploration tool that allows you to look at big data using high-performance, in-memory capabilities to understand your data, discover new patterns and determine what data is relevant.
  2. Model data. Now you can feed the results from your visualizations in to models to segment customers or develop lists for specific campaigns. The results? You don’t have to plan your day around the processing time anymore. Campaign management becomes more of a collaboration between people as opposed to an all-day or all-night run before you can look at results and see if it’s good enough.
  3. Optimize the business process. This is where high-performance analytics becomes really important. It allows you sit around a table with people to collaborate on the results of processing big data. You can optimize hundreds of campaigns for millions of customers during a single meeting, instead of waiting overnight for the jobs to process.
  4. Share results. The last piece uses dashboards to make the information widely available on tablets, ipads or desktops, essentially sharing reports and making the results of your campaigns available to everyone in a format they choose. Big data doesn’t have to be confined to five or six people in the back room. Results can be distributed on the platform they are already comfortable with, so they can play with data as well in a more structured format.

If you follow this four-step process, you’ll remove obstacles associated with big data, and you can begin to link previously disparate programs together as a single process. Before you know it, these changes will make big data an asset in your organization too.

Post a Comment

What is a data scientist? And do you really need one?

Jim Davis
Jim Davis|May 29, 2012

Regardless of the down economy, the amount of data being captured and stored didn’t slow down in the last few years. If anything, it has increased since 2008 when the recession hit. Even as business growth slowed and budgets were cut, most organizations continued to capture data.

The question is, what were you doing with your data? You might have used it to cut costs or be more efficient, but what else can you do with it now that we’re coming out of the recession and we have “big data” solutions available that offer faster processing speeds, more iterations of models and easy access to analyze your full arsenal of data?

Today, that data represents a significant asset that can be used for more opportunistic ventures. You can use it to prevent fraud, to get to know your customers better, or as an asset to develop new lines of business.

There are a couple of obvious prerequisites:

  1. Do you have the data? Yes, in most cases, more so now than ever.
  2. Do you understand the difference between summary statistics and advanced analytics that allow you to move your businesses forward?
  3. Do you have the talent or the expertise within your organization to help translate between the analyst and the business?

Number one is a no-brainer and I’ve covered number two before (see previous post: Is big data overhyped?). I’d like to discuss number three here by talking about this new career title: the data scientist.

When I first heard the term data scientist 18 months ago, Frankenstein’s laboratory came to mind. As we continue to research the job descriptions of data scientists, I now realize it’s a legitimate role that is useful in a lot of organizations to help you get the most out of your data and to help bridge the gap between IT and business needs.

So, what does a data scientist look like? Not like Frankenstein. It is somebody who has a background in mathematics, statistics and computer science. Data scientists are not necessarily experts in any one of those fields but they can understand all three. They have to be very good at translating the business value of data to the business and helping analysts understand what they possess.

The communication piece is a missing link in a lot of organizations, and data scientists can really help take full advantage of data to help overcome that challenge.

One obvious question that a lot of people ask is: Where do these data scientists live in the organization? They’re not IT. They’re not analysts. They’re not programmers. They’re the thing that brings it together and helps organizations communicate about the stories and the answers available in the data.

We’ve seen a lot of businesses find success with data scientists situated inside a Center of Excellence (CoE). That is one viable possibility, and it offers other benefits to really streamline and unify your efforts around analytics.

After all, you go to the trouble of creating a help desk to provide your employees with support for questions about email, hardware and other technical problems, right? If data is really important to you and you want to increase the use of data to drive decisions in your organization, why don’t you have a help desk for what could be one of your most valuable assets going forward?

Of course, a true CoE is more than a help desk, and a data scientist is more than a call center trouble shooter – but you get my point. You can’t just bring in the tools to solve your business problems and expect them to do all of the work. You need to have the right people in the right positions asking the right questions and teaching others how to use analytics to solve your biggest problems.

Ask yourself: How are you staffing your business analytics projects? Do you see a future for a data scientist role in your organization? What benefits could a CoE provide for your analytics projects?

Post a Comment

Four ways to unlock the value of big data in a hyperconnected world

Mikael Hagstrom
Mikael Hagstrom|May 23, 2012

When faced with every possible challenge – from socio-economic to environmental – man has taken seemingly small steps into cyberspace, resulting in a giant leap for mankind: the creation of the new asset class, “big data.” With 90 percent of the world’s data created in the last two years, we are only just starting to learn about the opportunities and discover the potential it holds. You might ask yourselves, how is big data providing opportunities?

Individually, we are all limited in what we can know. Together, hyperconnectivity makes it possible to overcome individual limitations and mine different types of data to find insights, including:

  1. Insights that will help us improve our health and quality of life.
  2. Insights that will help us use public resources more efficiently and effectively – to reduce the budget deficits without increasing taxes.
  3. Insights that will enable us to tap into the sleeping giant of Services to reveal new potential.
  4. And insights that will ultimately help us drive the skills and innovation needed to fuel inclusive growth in our global economies.

For more on each of these four topics, read the chapter I contributed to the World Economic Forum’s (WEF) Global Information Technology Report. My co-author Nina Gill and I set out to explore some of  the benefits of using high-performance analytics to improve society and provided multiple examples in each of the areas above. You can hear more about our Chapter 1.8 in the video below, or you can browse the full document at the WEF site.

Technology has come to our rescue in the past; in the future, High Performance Analytics is what will enable us to transform the world. If Big Data is the new oil, then High Performance Analytics is what will energize the new economy.

Post a Comment