Three ways to use a Hadoop data platform without throwing out your data warehouse

Working out where Hadoop might fit alongside, or where it might replace components, of existing IT architectures is a question on the minds of every organization that is being drawn towards the promises of Hadoop. That is the main focus of this blog along with discussions of some of the reasons they are drawn towards Hadoop.

For the past 12 months, I have spent a great deal of time speaking to organizations that were either thinking about adopting Hadoop as a data platform or were already well underway with their Hadoop journey. During this time, I have heard two core arguments as to why people want to adopt Hadoop: cost and agility. Perhaps unsurprisingly if you talk to those that have implemented, they often tout the same two things as the benefits of having adopted Hadoop. Lets dig into both.

Cheaper in many ways

The most frequent argument I hear is that using Hadoop as a data platform is economically efficient. I am consistently told that the cost of the software and support, per node or CPU, is significantly cheaper than almost every relational database management system (RDBMS) on the market by those who have adopted it. In addition, they tell me that the commodity hardware used by Hadoop is always a lot cheaper than the hardware organizations are advised to use in support of an RDBMS or a specialty appliance. One reason for the cost reduction is that redundancy is built into the system with Hadoop, so you do not need to worry too much about things such as redundant power supplies or disks, as a failed node in Hadoop is not a big deal.

In addition, many Hadoop users cite the fact that you can just upgrade disk and memory on nodes pretty much independently of the software licenses and other formalities as a major benefit. Likewise, I'm told, if an organization wants more CPUs in their Hadoop cluster, to aid with increased processing capability, it would not result in a rather significant and unrequired additional RDBMS license cost so incremental upgrades do not break the IT budget and are pretty predictable.

In adopting Hadoop as a data platform, organizations are hoping to slow down the burgeoning RDBMS growth, which is now starting to be a significant cost for many organizations, or to entertain the idea of taking the RDBMS out of much of the picture as data volumes grow except where it is really needed. Hadoop is seen as the route to cost effectively implement a data platform that is capable of handling the current explosion in data volumes and the continued acceleration of data that is expected with the Internet of Things.

Read More »

Post a Comment

Where oh where can I find data scientists?

475967869After attending the SAS Day at Texas A&M University on Oct. 2, I came away with a new perspective on some of the different educational offerings to help fill the analytic talent gap (which according to studies and research  continues to grow). In essence, there is a yin and yang of where to find your data scientists. While your company may need to hire new employees, you may also find that you have employees right under your nose who want to grow their careers while benefiting the company at the same time.

Universities from across the world are developing new programs and degrees to help meet this demand on both sides of the equation.  While many programs like North Carolina State University's (NC State) Master of Science in Analytics, provide graduate level education in a full-time 10-month program, Texas A&M's program offers a part-time, five semester Master of Science in Analytics open to working professionals.

Programs like NC State’s help develop new data scientists to enter the workforce, while Texas A&M's program is geared toward helping companies develop employees already within their workforce who possess a foundation of skills that can be further developed to fill these data scientist roles. And there you have it – the yin and the yang of where to find your data scientists!

Both types of programs are essential to help companies invest in the analytic talent they need to remain competitive, whether that means hiring new employees or investing in your existing workforce.  This is why SAS is actively involved in providing support for programs like these at many universities, as well as providing teachers and students with free access to SAS Analytics via our Analytics U program. With these kinds of investments across the board in analytics education, I am optimistic the skills gap will continue to narrow.

Post a Comment

Coming soon: The Internet of Cars

Today’s self-parking cars are a marvel, but that’s nothing compared to what’s just around the corner: Autonomous vehicles that can taxi you efficiently around a city, onboard navigation systems that warn of bad drivers or traffic jams nearby and offer safer, quicker alternative routes; trucks that haul commerce safely and quickly across the country, avoiding traffic delays and optimizing part replacement needs.

AndreasMai-Cisco

Andreas Mai, Director of Smart Connected Vehicles for Cisco

That’s the Internet of Cars, and it’s what Andreas Mai, Director of Smart Connected Vehicles for Cisco, will be talking about at The Premier Business Leadership Series in Las Vegas, Oct. 21-23.

Mai’s particular passion is the art of the possible and exploring how personal transportation can and will change as the Internet of Things becomes a reality. The implications for individuals, society, government and business are profound. And since Cisco is at the forefront of the connected car movement, Mai has had the opportunity to be in discussions at the highest levels of government and business.

“One concept we’ve discussed is the convergence of personal and public transportation,” says Mai. “The Internet of Cars will bring automakers, government, telecommunications and other companies together to share information and work on optimizing the public infrastructure. They’ll all need to partner to deliver new and better services.

Read More »

Post a Comment

STEM at the museum

On Wednesday, October 15, Ed and I will be spending the morning at the North Carolina Museum of Natural Sciences. It’s a great museum, and I always love going there, but that day holds an especially cool event. It’s called the STEM Career Showcase for Students with Disabilities. (If you don’t know what STEM is, it stands for Science, Technology, Engineering, Math).

We have a fun morning planned. We’ll have middle-school and high-school students with disabilities joining us. They’ll hear a group of amazing speakers, watch a game show of college students, and interact with a panel of STEM professionals with disabilities who are there to share how they achieved professional success.

On-site registration is already closed, but you can participate from anywhere in the world. We’ll be livestreaming the entire event. So listen in and feel free to submit questions! You might even see me; I’ll be the gorgeous dog at Ed’s side.

Read More »

Post a Comment

Statistics in the era of big data and the data scientist

468837845Depending on whether you are a half-full or a half-empty kind of person, the "big data" revolution is either a tremendous windfall for the career of a statistician, or the makings of a real existential crisis. As with most things, it’s probably a bit of both.

On the one hand, the Harvard Business Review calls data science the sexiest job of the 21st century. Since at least some statisticians would seem very qualified to fill in that role, AND those well-paying jobs, statisticians look ready to cash in on a very rewarding career indeed.

Yet on the other hand, statisticians are taught from the very beginning, FROM THE FIRST FIVE MINUTES OF STATS 101, about the values of a rigorous experimental design, of making sure there is a representative sample, and above all else, to never…ever jump to any premature conclusions. In the "big analytics" age of doing all the analyses on all of the data, statisticians often find this basic premise challenged.

Read More »

Post a Comment

Why organizational maturity matters

PillarsI wrote “Business Transformation” to guide leaders through a journey to transform their organizations. I included methodologies and examples gathered throughout my 29-year consulting career to assist them.

Every executive and leader focuses on how to use resources to produce value. Of course, value can be defined in many terms including profitability, market share and other objectives.

In my work with organizations in many industries, I've seen numerous cases where marginal business value is produced when an organization focuses on a particular business function or process.

But I've never seen greater value than when organizations broaden their focus beyond a specific business function to a holistic and enterprise-wide view and approach.

Read More »

Post a Comment

Exploring athletic data down to the individual stroke

Katherine Chinn addresses members of the GB Rowing Team squad

Katherine Chinn addresses members of the GB Rowing Team squad

Since my opening blog post about our new partnership with British Rowing and the GB Rowing Team, I’ve had a deep dive into their data, and we’re now close to our first key milestone – having all the data about a rower in a single location, so we can then start analysing that information and discover key insights to help make the boat go faster.

The data is often collected under extreme pressure in multiple locations (up mountains, beside lakes and away on training camps), so it’s been difficult to develop a standardised approach. The challenge has therefore been to link all of that information together – i.e. to link data in one spreadsheet to data in another spreadsheet about the same rower. However, we’ve now managed to do this in relation to most historical data, and a new consistent approach to recording data going forward is now in place.

Straight away it means coaches will be able to see everything they need to about a particular rower, rather than having to wait days for the information to come in from various sources. Biomechanical data from out on the water, for example, is now combined with all the indoor rowing data and other information such as blood lactate levels. Even sleep patterns are recorded to assess recovery between training sessions.

Read More »

Post a Comment

Is the SAS Embedded Process a secret sauce for Hadoop?

SAS has been developing "secret sauce" technology for more than 38 years.  Whether it has to do with being platform independent, processing in-database, running across a grid, or analyzing data in-memory like our SAS LASR Analytic Server or our High Performance Analytics offerings, secret sauce makes everything taste or, in this case, work more efficiently. And, as a result, the SAS secret sauce helps customers be more successful and profitable.

For example, the SAS Embedded Process helped one customer make 1 to 2 million dollars more a month. How on earth could technology make that big of a difference? Because this technology optimized a critical business process that targeted a subset of customers who are most likely to pay their debts. Before, this required a manual, 300-step process over 6 months to create, validate and deploy a predictive model into production. Now, this process has been streamlined into a 10-step, automated, 2-week process that works on all the organization's customers (instead of a small subset), making their collection efforts more effective and efficient.

Read More »

Post a Comment

Hadoop: the game-changer in banking

185263688At most banks, data is stored in separate databases and data warehouses. Customer data is stored in marketing databases, fraud analyses are done on transactional data, and risk data is stored in risk data warehouses. Oftentimes even liquidity, credit, market, and operational risk data is stored separately as well.

Bringing together regular, structured data from multiple data sources is the biggest challenge for companies in any industry. But leaving valuable data outside of the decision making cycle due to the limitations of existing data warehouses is not an option going forward.

What if…

Imagine what banking life would be like if you could access all the banks’ data and ask any question you could think of – and get the answers, fast. Wouldn’t that make risk reporting, fighting fraud, and taking the best action for your customers so much easier? Not to mention management decisions: which new products are we going to launch, what does our business need, which way should we go? It’s actually not that far away.

Read More »

Post a Comment

Every day is Customer Experience Day at SAS

It’s no secret that SAS is a constant among Best Places to Work lists all over the world.  But, what many may not realize is the connection between creating a company employees want to work for, and creating a company customers want to work with.

Read More »

Post a Comment