Beyond traditional clustering and predictive models lies social network analysis. It can help describe customers’ behaviors in new ways, but what exactly is it and how can businesses use it? To find out more, I interviewed Carlos Andre Reis Pinheiro. He’s been working in social network analysis around the world
In my last two posts, we concluded two things. First, because of the need for broadcasting data across the internal network to enable the complete execution of a JOIN query in Hadoop, there is a potential for performance degradation for JOINs on top of files distributed using HDFS. Second, there are
“When it comes to the Internet of Things, the future clearly belongs to the Things”. I made this brash statement in a previous post (“Cloud encounters of the Fifth Kind”) referring to machine-to-machine (M2M) being the fastest growing component of non-human traffic on the Web. I say “brash” because that
In my previous post, I talked about how a bank realized that data quality was central to some very basic elements of its initiatives, such as know your customer (KYC), customer on-boarding and others. In this blog, let’s explore what this organization did to foster an environment of data quality
I often blog about the usefulness of vectorization in the SAS/IML language. A one-sentence summary of vectorization is "execute a small number of statements that each analyze a lot of data." In general, for matrix languages (SAS/IML, MATLAB, R, ...) vectorization is more efficient than the alternative, which is to
For Parent’s Weekend this year, I needed to choose a restaurant for dinner in my son’s college town. Our extended family was attending the college football game and spending the weekend with our son. Before making my decision, I searched the internet for all the restaurants located within a reasonable
♦We learned this week that SAS is ranked #4 on Fortune's 100 Best Companies to Work For in 2015. This makes six straight years ranking in the top four (including twice at #1). ♦The March/April 2015 issue of Analytics Magazine includes a SAS company profile by my colleague Kathy Lange. As
As an executive coach, I've worked with thousands of managers and business leaders whose personal language sabotaged their effectiveness at driving change, not to mention their day-to-day team management. For your Inner Leader to shine through, you need to master your personal language--your way of communicating your company’s goals and
You may be intrigued to know how the average person compares to a gold medal winning Olympic athlete when it comes to things like height, body mass, resting heart rate, arm span, body fat etc. Or, perhaps more frightening, how you measure up? I know this will resonate with my
Public educators have increasingly been told to produce the “workforce of the future.” States are striving for alignment between what students learn and the jobs that ultimately will be available to them. This alignment is critical for students so they have the right skills and knowledge to excel at college
In my previous post, I showed how to approximate a cumulative density function (CDF) by evaluating only the probability density function. The technique uses the trapezoidal rule of integration to approximate the CDF from the PDF. For common probability distributions, you can use the CDF function in Base SAS to
Does your forecast look like a radio? No? Then don't treat it like one. A radio's tuning knob serves a valid purpose. It lets you make fine adjustments, improving reception of the incoming signal, resulting in a clearer and more enjoyable listening experience. But just because you can make fine adjustments to
The old adage is that “Data is the lifeblood of the insurance industry.” However, for many insurance companies, data is like the red-headed stepchild. No one is willing to take care or have responsibility for it. In the past, insurance companies have created data governance programs, but these have often
When I saw Robert Kosoro's cool ZIPScribble map, I knew I had to create a SAS version - and of course I had to add a few enhancements along the way.... I was perusing some of the examples on dadaviz.com, and Kosoro's ZIPScribble map caught my attention. It wasn't a particularly useful
One of the common traps I see data quality analysts falling into is measuring data quality in a uniform way across the entire data landscape. For example, you may have a transactional dataset that has hundreds of records with missing values or badly entered formats. In contrast, you may have
You might be surprised at how many movies and TV shows are made in North Carolina - especially within the last few years. This blog provides a SAS graph that will make the list of films even easier to read! A recent story by the Tar Heel Traveler, and an exhibit
In The Princess Bride, one of my favorite movies, our hero Westley – in an attempt to save his love, Buttercup – has to navigate the Fire Swamp. There, Westley and Buttercup encounter fire spouts, quicksand and the dreaded rodents of unusual size (RUS's). Each time he has a response to the
Evaluating a cumulative distribution function (CDF) can be an expensive operation. Each time you evaluate the CDF for a continuous probability distribution, the software has to perform a numerical integration. (Recall that the CDF at a point x is the integral under the probability density function (PDF) where x is
Warranties have a long - and some might say - interesting past. But the future is even brighter. New technologies and data sources are transforming our understanding of field quality, enabling deeper insights into product performance and customer preferences. These breakthroughs are accelerating the quest to reduce defects and satisfy customers.
I don’t know about you, but I get pretty determined to prove them wrong when people tell me that I cannot do something. I am not talking about fantastical things such as flying through the heart of the sun and out the other side without getting burned. Nor, am I
In my last post, I pointed out that an uninformed approach to running queries on top of data stored in Hadoop HDFS may lead to unexpected performance degradation for reporting and analysis. The key issue had to do with JOINs in which all the records in one data set needed
Helping students to reason statistically is challenging enough without also having to provide in-class software instruction. “Practical Data Analysis with JMP, Second Edition” walks students through the process of analysis with JMP at their own speed at home, allowing faculty to devote class time to crucial or subtle statistical concepts
Unter dem Motto "Big Data - Einsatzerfahrungen, Entscheidungsprozesse, Effekte" trafen sich gestern 580 Big Data Verantwortliche und Interessierte aus allen Branchen in Hanau, um Erfahrungen auszutauschen und Impulse für Big Data Initiativen in dem eigenen Unternehmen mitzunehmen. Ich möchte meine Erkenntnisse in diesem Blog weitergeben und diese insbesondere bezogen auf
Financial institutions are mired with large pools of historic data across multiple line of businesses and systems. However, much of the recent data is being produced externally and is isolated from the decision making and operational banking processes. The limitations of existing banking systems combined with inward-looking and confined data practices
Small data is akin to algebra; big data is like calculus.
Last week I received a message from SAS Technical Support saying that a customer's IML program was running slowly. Could I look at it to see whether it could be improved? What I discovered is a good reminder about the importance of vectorizing user-defined modules. The program in this blog
From the pressures of a highly competitive marketplace to changing economic conditions, to the evolution of the distribution network - the challenges facing the hospitality are many and varied. In this video, SAS asked a panel of experts to share their views on the issues that will challenge the hospitality
I recently wrote about how to overlay multiple curves on a single graph by reshaping wide data (with many variables) into long data (with a grouping variable). The implementation used PROC TRANSPOSE, which is a procedure in Base SAS. When you program in the SAS/IML language, you might encounter data
BI ist Auslöser für heftige Diskussionen. War es immer, wird es noch lange sein. Aktuell stehen zwei Lager im Ring. Die beiden debattieren aber nicht darüber, ob das Glas Wasser halb voll oder halb leer ist. Nein, sie reden darüber, wie schwer die jeweilige Variante ist. Und das hängt,
The electoral battlespace for the upcoming general election in the United Kingdom is starting to take shape. Campaigners are busily debating the political landscape. They want to own the high ground that dominates areas that matter most to voters – the NHS and the economy. With an ageing population and
