Can social media data give governments advanced notice of critical economic, societal changes?


"At a time when our need for policy agility has never been greater, our traditional 20th century tools for tracking international development cannot keep up. Too often, by the time we have evidence of what is happening at the household level, the harm has already been done … And the irony is that we are actually swimming in an ocean of real-time information.”

With that statement in a November 2011 address, United Nations Secretary General Ban Ki-moon declared the need for more sophisticated methods to understand, in near real-time, what is happening in the world. United Nations Global Pulse was created to research ways analytics can make that possible.

The world has now reached the point where billions of digital comments are posted daily on public web forums, blogs, and social media sites like Facebook and Twitter. These public comments express sentiments about employment, transportation, cost of living, and anything else that’s on people’s minds. If we collect and analyze all of these public comments with respect to current events (financial crisis, natural disasters, sporting events), clear and common sentiments begin to emerge.

For example, a recent study on unemployment conducted by SAS and the UN Global Pulse (@unglobalpulse) revealed that increased chatter about postponing vacations, increasing use of public transportation and downgrading one’s automobile could, indeed, predict an unemployment spike in the US and Ireland. While these relationships are not surprising, the analytics quantified the amount of time these conversations usually precede an event (i.e. chatter about delaying travel typically preceded unemployment spikes by five months in the US).

SAS' work with UN Global Pulse was the topic of today’s press release. In addition, Robert Kirkpatrick (@rgkirkpatrick), Director of UN Global Pulse, will join SAS CMO Jim Davis on stage April 24th at the 2012 SAS Global Forum Executive Conference to discuss the value of analytics in the context of Global Pulse’s work.

This infographic depicts some of the leading and lagging indicators of unemployment spikes. Please feel free to share with others!
Click to enlarge

Could social media analysis replace the need for official statistics? Not in my opinion. Social media analysis is an excellent complement to official statistics, often bringing more insights. For example, official statistics may tell us that there is a decline in the number of children enrolled in school in a developing region, and the public chatter may tell us why children are being pulled out of schools (to work in the local markets for food, to protect the village from civil unrest, etc.) And, as shown in the project with UN Global Pulse, the analysis can also determine the types of chatter that usually precede or follow an event captured by official statistics.

I believe this is just the tip of the iceberg of what we can do with social media and other non-traditional big data sources. For example, what if we took anonymized credit card data on spending habits or anonymized mobile phone data to see how people are moving before and after a crisis event? Similar to how social media analysis enriches official statistics, these other data sources would also provide additional detail and perspectives to tell a clearer story. Or how about also using satellite data to tell us how quickly things are growing throughout a country experiencing drought conditions, as a way to help understand migration patterns? The possibilities are endless. However, it all comes back to the UN Secretary General’s belief, that big data is useless without analytics.


About Author

I-sah Hsieh

Global Account Manager

I-Sah Hsieh is the Global Manager, International Development at SAS where he helps development organizations apply analytics to increase the pace of growth and prosperity in established and developing nations. I-Sah holds an engineering degree from Cornell University and has 15 years' experience in the software industry, across highly regulated industries like healthcare, defense, and telecommunications.

Leave A Reply

Back to Top