Every day we hear more about the impact of big data: 900 million users posting status updates and uploading images to Facebook, 140 million Twitter users generating 400 million tweets per day, smart grids optimizing electricity utilization, and sensors optimizing traffic flow on highways or factory floors. But, how do we manage this onslaught of data? How do we determine which information is relevant? And, how do we ensure that our information is trusted?
Last week SAS sponsored and I attended the Economist “Big Data” Information conference which focused on how Big Data, social media, and smart sensors are transforming the world and attempting to answer these questions. SAS is making huge waves in this space with our high-performance analytics (HPA), visual analytics and big data information management offerings. HPA, for example, leverages grid arrays of big data, in-memory, and in-database technologies to optimize your business decisions (like predicting churn or likelihood of loan default) by speeding computations that would normally take hours or weeks into taking only minutes or seconds.
Conference attendees and speakers were made up of a veritable litany of technology rockstars and many thought leaders who are putting their own “dents in the universe”, to quote Steve Jobs. It would be challenging to distill the amount of brainpower and ideas in the room into even 10 blog entries, but some of the more noteworthy paraphrased quotes, questions or concepts included:
- "When the world changes, correlation goes away… Causals are what endure." - Geoffrey Nunberg, UCAL Berkeley
- "Measuring more makes you more certain about the world." - Jeff Hammerbacher, Cloudera
- Every day, thousands of transactions between doctors and patients, insurance companies and members are being made with healthcare data. - Jon Bush, CEO, Athena Health
- "The Internet is a nervous system in the process of being born and we are all part of it." - Juliette Powel, Author of VibrantData.org quoting John Perry Barlow
- For every doubling of the size of a city, there is a 15 percent increase in every economic indicator (capital, crime, patents, wages) and a 15 percent savings on infrastructure costs. All companies die, but cities don't die. Unbounded growth requires accelerating cycles of innovation or else the system will collapse. - Geoffrey West, Theoretical Physicist, Santa Fe Institute
- "Gamifying" photoshop badges for work (ie., "Mayor with Most Layers") similar to Foursquare. - Kevin Lynch, CTO, Adobe
- Nowcasting is predicting the present . "401k" as search term is a predictor of a bad economy. - Hal Varian, Chief Economist, Google
- Invests in companies that own the data generator not just the data. - Max Levchin, Kaggle.com
- Building out Android apps to assist troops in theatre incorporating movement maps and IED blast sensors to save lives. - Mari Maeda, DARPA
- "Learning is remembering what you are interested in." - Richard Saul Wurman, Creator, TED
And here's a bonus of two more insights from Wurman:
- There are only 5 ways to organize information: location, alphabetic, random, categorical, and chronological.
- A good question is better than a brilliant answer.
I found the conference enlightening and intellectually thrilling. One co-founder of a big data startup said that he thought "it was a moral imperative" to leverage this ever increasing volume of information to help change the world. How's that for making a dent in the universe?
Learn more from "big data" experts in this special 32-page report on high-performance analytics.