I worked at the General Dynamics (now Lockheed-Martin) F-16 jet fighter plant in Fort Worth, Texas, during the mid-1980’s, where they subsequently manufactured the F-22 and now the F-35. My tenure there spanned the era of the $400 hammer and $700 toilet seat scandals in the military procurement world. While we didn’t have any of those specific issues to deal with, management wanted to make certain that no similar skeletons hung in our closet, so we were directed to inspect and assess EVERY SINGLE VENDOR INVOICE during the prior twelve months.

We operated on a cost-plus basis with the government, where vendor and direct costs were assigned to various cost pools, such as production, R&D or maintenance, upon which a pool-specific, pre-negotiated overhead rate would be applied, and which included GD’s profit margin.  So while it was unlikely that any particular invoice would surface as a public embarrassment, the government had the right to audit any and all of the supporting documentation for the cost pools at any time, and we simply wanted to assure that we would come out squeaky clean should that occur.  We had the authority to reject obvious incorrect charges ourselves, such as reimbursed personal expenses that should have been excluded from the pool, whereas other more questionable charges were to be brought forward for management review.  I happened to come across one of those.

My memorable moment came when I unearthed an invoice for “twenty-four extra-large bunny suits, ears and tails removed”, from a local costume shop.  The department incurring this charge was the Flight Line, that row of hangers across the street and adjacent to the runway at Carswell AFB where the engines were installed in the otherwise completed aircraft and where the test pilots would run them through their paces before turning them over to the Air Force for active duty.

The question on my mind, and everyone elses back inside the main admin building was: What exactly are they doing over there on the Flight Line?

The Flight Line was not one of my normal departments, but the task fell to me to phone up the department manager and find out what was going on.  He was exceedingly polite and actually seemed excited that someone was paying attention.  “Can you come out here between 2:00 and 3:00pm this afternoon?  We’ll have two aircraft ready for their next flight tests, I’ll show you around, you’ll get to see them take-off close-up, and I’ll show you those bunny suits and what we do with them.”

Turns out the bunny suits were completely legitimate, which perhaps should have been obvious from the one clue – ears and tails removed.  During engine installation, adjustment and testing, the engine maintenance engineers have to crawl up into the engine bay.  The bunny suits were worn over their normal clothes, and with the outfit zipped up to the neck, served two purposes.  First, the soft, furry fabric would not scratch any of the maze of fuel or hydraulic piping and fittings (have you ever seen the inside of one of these engine compartments?), and it would prevent things like watches, shirt cuffs, buttons, buckles, belts, belt loops, shoelaces and the like from inadvertently snagging on something, bending or damaging it, often in a manner undetectable until years later during a high-G turn during combat.  And secondly, it prevented small objects like pens and coins from falling into the compartment and perhaps later working their way into the moving parts of the engine, causing catastrophic damage.

What this goes to show is that often your outliers can be as interesting as your “normal” data.  Often the outliers are discarded, replaced or ignored in an effort to get ever more precise in the measurement of that mean, median or next quarter’s forecast.  This of course is a commendable endeavor, but don’t let it distract you from the equally valuable information that lies on the other side of this analytical coin.  Tell me something new, tell me something I don’t know – these are the kinds of insights that can come from an analysis of the outliers (which are only "outliers" with respect to some other analytically determined norm), something SAS Visual Analytics is exceptionally well-designed to explore and address.

A few examples might help:

• Gray Swans.  I mentioned in an earlier post, “A Plethora of Black Swans”, that black-swan-like events are common enough to perhaps warrant the creation of a class of ‘gray swans’ – “predictable events with known parameters that evolve into something with more significant, and quantified, outcomes”.  Look for the correlations among your gray swan events, and the attributes that might be signaling something worse to come.
• Gray Swans (con't).  We tend to be both over-optimistic in our assessment of the future, and poor judges of risk in general.  In another earlier post, “A Favorable Product Mix”, I stated: “Your pessimism is not pessimistic enough, and your worst case is not even close to being worthy of the title. Your planning and budgeting is based on a most favorable product mix that has almost no chance of actually coming to pass. There is more variability in your business than you realize; your intuitive, gut-level instinct as to how bad things can get in the normal course of business is just not as trustworthy as the data itself”.  An analysis of your outliers can help rectify that – it can put a spotlight on not just what a worst case might look like, but what its contributing factors might be.
• Bluebirds (since I seem to be on a bird theme). Not all outliers are negatives - on average half of them should be on the plus side.   Innovation, new product ideas, new combinations of products, new and unique product uses, and even new business models, might be lurking there amid those unseemly discards.  It has been joked that only in a math problem can you buy 60 cantaloupes and no one asks what is wrong with you.  But why did that customer buy 60 cantaloupes, or light bulbs, or birthday cards?  What are they doing with them?  Is this a new trend we’re catching on the upswing (what Gartner calls a “weak signal” in a Pattern-Based Strategy). Can we bundle that together with a service and capture or invent a new market?  Innovation was really what the twenty-four bunny suits were all about.
• Process Management (the birds have flown). Outliers are going to show up in any analysis of your operating processes, the most worrisome being things like rejects, dropped calls, misclassified results, etc …  And while we are of course always on the lookout for ways to improve our efficiency, every now and then a real stunner emerges from the data.  Why did the reject rate spike from 0.02% to 3.6%, and why did it remain there for over an hour?  Why didn’t someone notice?  Why didn’t someone shut down the line?  Do we have a bigger problem here than just equipment failure – do we have a culture that is afraid to bring these events to management’s attention?

This is just a sampling off the top of my head of what benefits might be hidden in your outlier data;  I’m certain you can come up with a quick 3 or 4 that apply to your industry, market or circumstances.  And as for those bunny suits on the Flight Line, as legit as they were, management decided not to submit them to the government as  part of the cost pool for reimbursement – too risky and too difficult to explain if the media ever got a hold of it.  But for some small costume maker in Fort Worth, it was a most lucrative outlier indeed.