sascom voices - The sascom magazine blog

Gartner BI Summit

It’s been a long few weeks. A trip to SAS Campus for a DI Summit to discuss, among other things, the future of Data Integration, followed by a trip to Maryland for the Gartner BI Summit in which we are a platinum sponsor, to be followed by a trip back to Maryland for SAS Global Forum.

Basically, that breaks down to a trip to talk about what DI capabilities will be required/important in the future, a trip to talk to folks that are ahead or behind us in building the vision of future DI, and a trip to talk to our community to see what they think about everything. A lot of travel and a lot of talking (and listening...), but hopefully our products that come out in one year, 3 years or 5 years (not to mention the products coming out right now…) will be visionary and inspire passion.

Who, of the vendors, have the most accurate vision of the future? Who’ll be able to deliver on that vision and who will be bought and disappear? Stay tuned…

Some very interesting discussions at the BI Summit... I had five one-on-ones with analysts covering DI, BI and Performance Management. My discussions were heavy on Cloud Computing, Metadata, DI for Business (as opposed to DI for IT) and “Interesting Inquiries”. Interesting take-aways that will make future blog discussions: balancing the vagaries of public information (data from the internet, public clouds, etc.) with the demand for, availability of and independent access to this data; the rise, fall and rise again of departmental systems and applications; and why business users choose “easy” over “powerful” for BI apps.

Mmmmm, I think I’ll comment on the middle one here: multiple independent department apps vs. enterprise apps.

TDWI: Day 2, Night 3

Second full day at TDWI and activities were in full swing. Though I was looking forward to attending Evan Levy’s class “Beyond the Data Warehouse”, briefings and interesting ad hoc discussions kept pulling me away. Even though it can be frustrating when I get double booked with activities (meetings, classes, meals…), TDWI continues to be a great place to network, share ideas or just plain learn new things.

Yesterday, I had three “official” briefings and a reception (no colored dots to give out, thank goodness…) and today I had three more briefings, a podcast and our game show themed hospitality event. Even though there were a few common questions like “How is this economy impacting SAS?”, there were many interesting discussions that included SAS’ BAF (Business Analytics Framework), the upcoming 9.2 release, the upcoming SAS Global Forum, Data Quality, the Unity Project, current industry and economic factors, and “Ken’s Vision of the Future”.

In one discussion with Jim Ericson of Information Management, I had a chance to discuss one of my areas of interest, which is around managing the quality of data outside of your control, specifically web data. In a roundtable discussion at a previous TDWI, I asked the question: “Given the increasing focus on data governance, what are you doing to validate and manage the web data your users are downloading for presentations, reports, analytics, etc.?” The answers at the time included jaw drops and the general feeling that this was an area they were not ready to take on. My thoughts at the time were around creating a DQ rating system that could vet websites according to a set of governance requirements, ultimately giving DQ grades (“A”, “B”, “C”, etc.). The point being that even though specific data might not be validated, the source website could be. A year later, in having this discussion with Jim Ericson, he raised the point that today it’s much more common for there to be folks frustrated with IT’s inability to deliver reports when they want them, so they create their own mash-ups on the fly, avoiding IT altogether. Of course, the same governance/validation issues arise in addition to the creation of information silos of “questionable” trustworthiness. The ability to allow resourcefulness and creativity must be tied to rewards for “big picture” thinking. You might be successful as a “rogue report developer” or you might not if it turns out the data you used was bad. If, however, you reward people for creating new and innovative reports AND also for validating the content to make it “enterprise ready,” you’re taking a step in the right direction. There is still the issue of silos, but we already know about solving the problem around silos.

Closing thoughts for today are around the many economy related discussions I’ve had over the past few days. In my briefings, many of the discussions would start off with: “Your competitors have been saying that this awful economy has been great for their business. How has it affected SAS?” I find this line of thinking very disappointing. Companies certainly look for gains in efficiency and better intelligence when times are tough, but when I begin to start thinking that way, I remember that our customers, and our competitors, are not only companies, but people. In many cases, they are people that I worked alongside of at one or more points in my career and that now, unfortunately, are people in many cases looking for work. Hopefully, through our offerings, as an industry, we can help support new initiatives and create new job opportunities. And maybe… through the use of better predictive business analytics… we can help lessen or prevent the next down economic period.

Good night from Las Vegas! - Ken

TDWI: Day 1, Night 2

First full day at TDWI in Las Vegas has come and gone. I arrived yesterday in time to share hosting responsibilities of the SAS table for the Partner Networking Reception with Mark Moorman. Together we greeted and chatted with about two hundred visitors to our table (as most expertly setup by Bill Davis, Molly Hazard and Ericka Wilcher…) and placed colored dot stickers on entry forms which, when filled with dots from each sponsoring vendor, enabled attendees to enter into the drawing for prizes. Despite our best efforts, we still walked away with colored dot stickers on our shoes, pants, shirts…

Overall attendance is down from last year by about 50% (just under 1000 last year to about 550 this year), which, given the state of the economy, is better than expected. I haven’t gotten an official number yet, but based on over 100 conversations I’ve had with attendees, I expect about 80% of them to be first timers to TDWI. Based on these numbers and my many conversations, here are my general findings:

1. Despite the economy, there are still many DI/Reporting/DW projects going on in support of better reporting and/or cost cutting
2. Many of the resources brought in to work on these projects have come from other areas in the company, and need general training in data warehousing, data quality, and reporting
3. There is still an emphasis on fixing existing problems, with fewer resources, and enhancing reports with easy to get untapped data sources.
4. Still very few forays into unstructured/semi-structured data – focus still on the cake, not the icing
5. Data Integration/Bad data are still the most common challenges mentioned

My Conclusion:

1. The need for more confident decision making continues to drive the need for better report information
2. IT will, for the foreseeable future, continue to try to do more with less
3. The importance of useful, trustworthy reports continues to drive initiatives, trumping new project funding

Bottom line: Most companies are still focusing on the fundamentals, but looking forward to adding competitive differentiators – aka better analytics – when they can.

Yesterday, I had a chance to sit in on a class taught by Claudia Imhoff and Colin White on Operational Business Intelligence. Pretty good interest in the topic, based on the number of class attendees, but there were plenty of questions. My impression was that attendee interest was based on their desire to extend existing knowledge/technology to operational side (“.. we can report on that too…”), without a fully baked strategy around operational BI. Class did include a few bad airline experience stories around luggage loss. The point being that airlines have enough information to provide better service than they do.

This reminded me of one of my similar bad airline experiences. In my case, I waited for a flight home after many delays and cancellations. I was eventually booked on the last flight home, which was also delayed waiting for a crew on a different incoming flight, even though the plane for my flight was at the gate. Finally, right after the incoming flight arrived, after multiple hours of waiting, my flight home was cancelled because the flight attendant “timed out”, which means that she had worked too many hours in a row without “down time.” My frustration was that the airline should have known many hours earlier that the timeout would occur after a specific number of hours, at a specific time that evening. They could have either cancelled much earlier instead of having us wait around for a surely to be cancelled flight or they could have brought in a different attendant that had enough down time to work my flight home without timing out. Just another case of having data but not using it, sigh…

Three briefings today followed by a reception for the Executive Summit attendees. I’ll write more on these tomorrow, as well as the three plus more briefings and our hospitality event scheduled for tomorrow afternoon/evening.

Good night from Las Vegas! - Ken

Lies, @^%$# lies and benchmarking

Hello fellow world travelers! And by world travelers I mean people on planet Earth tooling around the Sun at around 66,507 miles/hour. Throw in the fact that we are also rotating around at something akin to 330 miles/hour (speed at the equator), and you realize that we are, for the foreseeable future, riding on a high-speed giant Tilt-A-Whirl.

While you may (or may not…) think this information is interesting, the big numbers theme is important. You should be asking yourself: Is this relevant to me? How do I use this information? What the heck does this have to do with data integration?

Of course, to those of you who have asked similar analytical questions and received the universal, second-most-frustrating, two-word answer: “It depends…”, you have my empathy. So, enough with this analogy, let’s get to it.

Benchmarking. Amazing metrics get published and your first thought is probably “What shenanigans did they pull to get those unrepeatable numbers?” If, however, you are one of the very intelligent select few who are considering the acquisition of a similar product to said benchmarked product, you may instead be thinking “Is this relevant to me?” and “How do I use this information?”

Data integration vendors recognize the need for clear and concise documentation describing realistic benchmark configurations, data types, data volumes, transformations and multiple commonly used source and target systems. In short, vendors want their benchmarked systems to be similar to your environment, which is what you want, so the benchmark will be meaningful to you. Is it possible and practical? Yes, but… unless you are paying for a proof of concept (POC), it is unlikely you will get even a close match. So, again, why pay attention to benchmarks?

Two reasons: technical and political.

Data integration benchmarks, above all, are grounded in technology. They tell you what to expect in a specific environment, using specific hardware and software, running specific tasks and using specific data. If any of these benchmark attributes can be applied to your vision of the new system, then you have successfully acquired new data points to help you with your decision. The first step is to remember why you are looking to replace what you have. If it is because of existing technical limitations, even one of these data points – if it is relevant – is useful.

Sometimes, changes are required for other than technical reasons: You’ve just been acquired… You’re moving to corporate standard applications… Your budget’s been slashed… In these cases, having data points to help you make your case one way or the other can be invaluable. No one wants to be the manager that knowingly forced a change to a system that cannot meet your business requirements. Everyone wants to be the manager (aka the “Hero”) with the foresight to prevent catastrophic failure.

There is something very satisfying about looking at a benchmark, understanding how it does or doesn’t apply, and then moving on. Note that even a poorly performed benchmark can be a useful “bad” example. Remember the children’s story Horton Hears a Benchmark: “A data point's a data point no matter how small…”. If a vendor’s benchmarks don’t provide useful information, that tells you something right there about the vender.

One last set of caveats on this subject: Your mileage may vary… Your results may be different… Professional driver on closed road… 7.3% APR… One dentist didn’t prefer it… Here is one case where it takes good data to get good data!

Have a great week! - Ken

"A journey of a thousand miles begins with a single step."
~ Lao-tzu, The Way of Lao-tzu

There are many topics that I considered writing about for my first blog entry. I thought about technology… about the merits of holistically approaching business/IT initiatives… even about how data integration is one of the major missing links in the war on terror.

There’s plenty of time for topics like these, and I’m planning on writing an update here about every 10 days or so. This being my first entry, I want to set the tone for future blog posts (and give you some insight into its highly intellectual word put-er out-er) by sharing my favorite data integration joke (no, I’ve never told this joke at a party…). Apologies in advance and without further adieu…

Two strings walk into a bar. The first string says “Give me a beer.” The second string says “Same here, @#%$&*^!%$&^>)~*” The shocked bartender turns to the first string and says “Hey, what’s with him?” To which the first string replies “You’ll have to excuse my friend, he’s not null terminated.”

Maybe I’m showing my age, but I think that the originator of this highly eloquent parable, probably, didn’t appreciate the lesson it exemplifies (OK, the originator was probably a coder like I was who only saw 9 a.m. if it was at the end of a long day…).

The lesson is both simple and universal: Knowing what you said doesn’t mean you were understood. And that’s under best case conditions, including communication was in real time, you were using a common language, everyone was paying attention, there were no changes made to what you originally said (remember playing the game “telephone”?), and on and on... As this applies to interpersonal communications, so too, does it apply to data integration. After all, isn’t data integration really all about moving information?

To people who create data, govern data, move-massage-manage data or simply rely on data to be successful in their job, the point is that when data is incorrectly interpreted (Is the data bad or is it my incorrect understanding of what the data represents?) and you don’t catch it until after you’ve used it, unexpected things happen and they’re usually bad. You don’t want to get a comment about the data you used, like the comment Inigo Montoya makes to Vizzini the Sicilian on his use of the word “inconceivable”, from the movie The Princess Bride: “I do not think that means what you think it means…”.

I’m sure you all have your own war stories about failed “information transfer” as a result of unexpected problems with data quality, data governance, or data transformation. My blog posts here will be about connections, between speaker and listener, sender and receiver, employee and manager, IT and Business as well as between source and target. We live in a society where life as we know it depends on data integration – digital cell phones, email, healthcare, constituent groups of the Department of Homeland Security… And small details count.

For example, only one small edit changes the follow note to the prison warden from “Pardon impossible, to be sent to prison!” to “Pardon, impossible to be sent to prison!”.

Since my blog posts will focus on data integration (DI) and how it fits into “The Big Picture,” I will refer to these scribblings as the “DI-ablog”. It can only be a real DI-ablog if you let me know what you think.

Coming up next… Lies, @^%$# Lies, and Benchmarking…

Have a great week! - Ken