The first step on the path to Hadoop nirvana?

On May 7, 2015, Conservatives defied the polls and won the UK general election, handing Labour and Liberal Democrats a surprising defeat. Now, with Chancellor George Osborne presenting his budget in a few days' time (July 8), the new Conservative government is tackling how it will deliver on its campaign promises. That means governmental computers have been crunching away to help the Treasury conduct its Spending Review 2015, showing how different departments have fared with their revenue and policy targets.

BusinessBenefitsLast year's budget included Chancellor Osborne's announcement that end-of-year paper tax returns will be scrapped in favour of real-time online accounts by 2020. Deemed "a revolutionary simplification of tax collection" by the Chancellor, it was confirmed that the government plans to switch to "digital tax accounts."

That means that by 2020, instead of the annual rush to file tax returns, individuals and small businesses will be able to submit their affairs continuously via computer, tablet or smartphone – at any time throughout the year. Read More »

Post a Comment

Education meets big data – statewide longitudinal data systems

Yes, its true. Education has big data and always has. One perfect example? Statewide longitudinal data systems. Every state has one that tracks information on students from preschool until they enter the workforce (so some states call them P-20W). Think about all the schools and all the students across your state, now add 20 years of enrollment and test data, plus a whole lot more. It's lots of big data, right? Let me tell you more about these data systems in this and future blog posts.

A statewide longitudinal data system (SLDS) tracks student data from preschool through college and into the workforce across the state. While many states have had databases for years, there are some distinguishing characteristics of SLDSs that are important to note:

  • An SLDS tracks and maintains student- and staff-level data across the entire state (not simply district-wide or county-wide).
  • An SLDS links data across entities and over time, providing a robust and complete academic history for each student, as well as aggregated data about subgroups of students.
  • An SLDS makes these data available to researchers and other educational agencies for analysis and reporting.
  • An SLDS provides current data to stakeholders in a secure manner and with unique identifiers that ensure student privacy.

Read More »

Post a Comment

Are you experienced?

electric guitar close upAlmost fifty years ago, a band emerged from London that changed the landscape of rock music and set a new standard for musicianship. The band was The Jimi Hendrix Experience, and their front man redefined what it meant to play a guitar. Hendrix took an inanimate object – the same electric guitar that had been at the core of popular music for over a decade – and created a sound and style that made people wonder, “How does he do it?”

Fast forward to today’s world, and almost all of us have the opportunity to do something equally as transformational for your organization. We all make decisions – strategic, tactical, and operational. And most of us struggle at times to ensure that those decisions are based on sound information and reflect all sides of the issue.

Enter the New Analytics Experience.

Read More »

Post a Comment

M-payments could take money laundering to new levels

In my previous post, I talked to John Cassara, a former U.S. Intelligence Officer and Treasury Special Agent, about growing concerns over trade-based money laundering. I followed up with him about the growing threat of mobile payments and how mobile phones can potentially be used to launder illicit funds globally. Here are the highlights from our conversation:

You've been a consultant for SAS for the last four years, and I find it interesting that almost from the time we met you've been talking about m-payments.  First of all, describe what mobile payments or m-payments are.
John:  Certainly. Global commerce is witnessing a variety of new-high tech money and value transfer systems. The Financial Action Task Force calls them “new payment methods” or NPMs. They're also sometimes called “e-money” or “digital cash.” Examples include internet payment services, cyber currencies, stored value cards, prepaid calling and credit cards, digital precious metals, and mobile payments or m-payments. While m-payments is an umbrella term, the commonly accepted usage refers to the use of cell phones to credit, send, receive and transfer money and digital value.

Read More »

Post a Comment

See what you never expected with data visualization

A strong quote from John Tukey explains the essence of data visualization:

“The greatest value of a picture is when it forces us to notice what we never expected to see.”

Tukey was a famous American mathematician who truly understood data – its structure, patterns and what to look for. Because of that, he was able to come up with some great innovations, like the box plot. His powerful one-liner is a perfect introduction to this topic, because it points out the value of seeing things that we never expected to see.

With the large amounts of data generated every day, it’s impossible to keep up by looking at numbers only. Applying simple visualization techniques helps us to “hear” what the data is telling us. This is because our brain exists in two parts. The left side is logical, the mathematician; the right side is creative, the artist.

Read More »

Post a Comment

Get value from the V's of big data by using the D's of analytics

I've been playing around with this equation to show how big data derives value using analytics. From my perspective, it looks like this: Big data + analytics = Hadoop + SAS = Value.

To expand the equation, I started thinking about the V's of big data and the D's of analytics.

New-Analytics-ExperienceYou've probably heard about the three V's of big data – volume, variety, and velocity – for years. More recently, the V's have expanded to include others like veracity and value. All of these V's simply describe the attributes of big data, which is fine for understanding what makes up big data.

But, what are the D's of analytics? This requires you to go beyond just defining big data and making sense of it. For this, you need the three D's of analytics: data, discovery and deployment.

You may have seen the three D's earlier this week in a post about the New Analytics Experience. But lets look at each one in more detail. Read More »

Post a Comment

Four levels of Hadoop adoption maturity

So you’ve been monitoring or are already on the journey with Hadoop -- and you’re wondering: Where are we on the adoption curve compared to the market in general?

Based on my interactions with numerous companies, I want to share what I think that curve looks like so that you can orientate your organization and decide if you’re leading the way or lagging behind. Neither is inherently bad, but you do need to be conscious of where you are and why.

Read More »

Post a Comment

Looking "beneath the dots" of geospatial data

If you use a GPS system or a mapping app on your phone, you're familiar with geospatial intelligence. But did you know this same technology can be used to map the spread of diseases, to understand the spread of wildfires, to visualize environmental issues - and much more?

The proliferation of geospatial intelligence data has become ubiquitous in our lives and and geospatial capabilities are changing the world as we know it. Nowhere is that more evident than at GEOINT 2015, where my SAS colleagues and I are attending and exhibiting this week. Hosted and produced by the United States Geospatial Intelligence Foundation (USGIF), the annual GEOINT Symposium is the nation’s largest gathering of intelligence professionals and the preeminent intelligence event of the year.

There is eye candy everywhere. Ginormous displays of high resolution maps compete for attention throughout the exhibit hall. You don’t see this much technological glitz outside Hollywood. Geospatial technology is by nature so visual and the innovations so magnificent that attendees have a hard time not being wowed.

There are lots of “A-has!” when data is superimposed on top of maps with geolocation coordinates and served up for the audience to see. But, think about how much richer the analysis could be if underlying multiple data sources, including open source information, could be brought together so analysts could understand what it means at a deeper level. Not only could you get answers to complex  questions, but predictive and prescriptive analytics could be used to better guide decision making.

Open public health data could help pinpoint looming health crises

Open public health data could help pinpoint looming health crises

Read More »

Post a Comment

The New Analytics Experience: Build a culture that embraces big data challenges

Big data. Streaming data. Complex data. We’ve all heard the reasons why organizations feel like they’re facing an insurmountable data challenge.

Now, it’s time to do something about it.

For the past few years, SAS has helped some of the world’s leading companies make sense of an avalanche of data. We’ve seen healthcare and life sciences organizations pool patient data to find trends to help beat cancer. Utility companies now use weather data to pinpoint areas where demand may exceed capacity. Banks analyze activity to find signs that a hacker is trying to gain access to sensitive data.

This new wave of data is causing organizations to make the most dramatic technological changes since the Internet revolution of the 1990s. Big data technologies like Hadoop are allowing you to capture and process data like never before. And a new class of data-savvy professionals, often called data scientists, need the freedom to experiment with data to find new answers to old problems.

Here’s the kicker. You have to manage all of this without disrupting your business.

Welcome to the New Analytics Experience.

Read More »

Post a Comment

Ten quotes to inspire your next big data project

Are you embarking on a big data project or implementing a new analytics program and need some inspiration?

I'm not the kind of person who buys posters of inspirational quotes to hang on my office walls. But I have been known to scribble a favorite quote on my office white board. And to share quotes that speak to me on my Facebook wall.

If you find inspiration in good quotes too, see if any of the following resonate with you. We've collected these quotes below from keynote speakers at recent SAS conferences, where attendees heard presentations from inventors, astronauts, CEOs and data scientists across every industry.

My hope is that, even if you could not attend these events, you find inspiration in these statements to use data for good, to look at your data differently or to start a new project with your data.

Share them, tweet them and pin them. Or pull out your dry erase marker and write them on your white board. And if you have other favorites, share them with us in the comments - or on Twitter. Maybe we'll design something around one of your favorite quotes for our next post about inspirational quotes.

Read More »

Post a Comment