“There’s no such thing as information overload - there is only filter failure”.  ~ Internet scholar Clay Shirky

Information overload is not just a recent phenomenon, it entered into human experience in the middle of the 15^th century with Gutenberg and his printing press, and we’ve been devising ways to cope ever since.  Today, more books are printed in a month than can be read in a lifetime.

And that’s just books – every day we create approximately 3 exabytes of data (that’s 3 million terabytes for those of you keeping score using last year’s counting system).  Every second, 3 million emails are sent, 50,000 Tweets are tweeted, and 2 hours of cat videos are uploaded. Every.  Single.  Second.  (in the time it took to read this far, you just missed 3,000 cat videos)  If you’ve only got 100 unread emails you’re still an amateur.

So how do we cope?  We do what nature does – we filter.

On your drive into work today, how many mail boxes did you pass?  Can you remember even one of them?  How many cars did you encounter, and what were their make, model, color and license number?  What color is the carpet in your office hallway?  Right now, what do you hear?  Alright, close your eyes and sit still for fifteen seconds, now what do you hear?  How could that garbage truck have been so loud and yet you hadn’t noticed it?

Our brains filter for the significant, the novelties, and what we pay attention to.  A police siren, a deer bounding across the road, a mailbox painted like a giraffe, our baby’s cry, that dark green, 2009 Dodge Ram pickup truck with the “As a matter of fact, I do own the road” bumper sticker that cut you off.  Discarding forever the insignificant bulk of the rest of the raw sensory input.

The early printers had to deal with information overload themselves by forcing publishers to act as an upstream filter for quality, and to take on the risk of overproduction – printers learned early on that they couldn’t just print a thousand copies of everything that came their way or they’d end up sitting on more unsellable than sellable inventory.

But in today’s world-wide-world of internet sports, where the distinction between information producers and consumers has been blurred – every device owner is a potential content creator, a potential blogger like me – the filters have moved far downstream, even to the point of falling into our laps.

Filters in the digital world work via selection, and the best of the intelligent filters work in several different ways.  The most common approach is for a smart someone or something to sort through all the information and select for you only what is important by some general criterion or algorithm.  We call these “editors” – CNN works this way.

A slightly more sophisticated filter would be a machine-learning system that gets to know your specific personal interests and priorities, getting smarter and better at it with more experience over time, standing in for you as your ‘agent’ – see my previous post, “Analytics for Agency”.

"Analytics is the filter for our big data-internet-information age."  I can think of no better plain language example or definition of what analytics is and does than that summary statement.

And given the circumstances, with what was exabytes soon to be measured in yottabytes (a terabyte of terabytes), there are no other options.  Analytics is going to become an integral part of our personal, social, commercial and political lives as a matter of survival, teasing out the significant from the noise, the patterns from the petabytes.  Behind the scenes, yes, but that’s how all good filters work, whether in your brain or your news feed.

Without analytics, for all intents and purposes with that much data, all of it will become nothing but noise, data without information, the truly significant indiscernible from the trivial.  The complaint, however, is not with the data or with information overload, it’s with the filters, the broken filters.  Wherever you find a broken filter there also you will find an application for analytics, pointing you towards the oases of information amidst the desert of data.