Two years ago, I found myself the proud, first-time owner of a garage. My wife and I quickly started to add new items to the garage – a battery-powered lawn mower, two beach cruisers and four Tommy Bahama beach chairs. They were stored with ease. What a fantastic world I'd been missing out on. But it wasn't long before we outstripped our existing garage storage system's (GSS) capacity by adding new items like a surfboard, soccer goals and kid bikes. I could squeeze the bikes in there, but the surfboard was much larger than anything else I had stored in this system. Storing and retrieving objects soon became complicated, frustrating and time-intensive.

Next, I found myself fascinated by Rubbermaid FastTrack, a GSS that consists of numerous three- to six-foot bands of steel that you bolt onto the studs in your garage walls. You then hang overpriced hooks, ball bins and bike holders from these steel bands.

As I was hanging these tracks in the 95-degree heat last summer, I had fever dreams of how this garage storage problem was similar to the challenges of data modernization. And how, similar to homeowners, organizations struggle to modernize their data architecture to accommodate new and varying types and sizes of data at unprecedented rates.

Why do we need data modernization?

Today, organizations struggle with increasingly exogenous sources of big data and a gap in the skills needed to manage that data. Business users demand self-service access to the data. IT struggles to meet the demands of the business. And many different data integration and big data technologies like Hadoop are brought in to allow for increased volumes or increasingly real-time or streaming access to the data.

Think of your data architecture. Is it flexible enough to accommodate on-boarding a new customer intelligence solution or supporting Apache Hadoop technology? Or do you have to move everything out of the way or build a new data warehouse (i.e., garage) to make room for new applications, data stores or users?

Modernizing data architectures means adopting new data integration tools and patterns to better accommodate the needs of increasingly diverse users and use cases. Three new data integration patterns include:

• Data virtualization to create agile, secure and blended views of your data.
• Self-service data preparation to give business users simplified, immediate access to data.
• Event stream processing to monitor real-time streams of data for important events.

These relatively new data integration patterns are evolving, and they coexist in today's modern data architectures with older bulk or ETL processing approaches.

Advantages of data modernization

The benefits of modernizing your data architecture are multifaceted. They include:

• Agility. Modernizing allows for better alignment of IT initiatives with the needs of the business
• Operational efficiency. By streamlining access and integration, business users can get to the data they need, when they want it and without waiting on IT. IT is then freed from iterating with business users to get the data in exactly the right format.
• Scalability. A big driver for modernization is the need to scale to meet greater demands from more users, databases, data sizes and data types.
• Better security. With a modern data architecture, governance policies can be enforced more dynamically so that views are generated and sensitive information is encrypted or anonymized based on your role.
• Better decisions. By modernizing, you can use analytics to make better-informed decisions and create competitive advantage

Technology is only half the battle

Next birthday, I was faced with extending our garage storage system to accommodate a new and much larger item. That's right, I received a 12-foot dual kayak as a present – and our existing FastTrack architecture simply could not handle it. The kayak had to live on the garage floor, limiting our agility and our ability to store and retrieve other items. The lesson here is that no matter how hard you work to create an agile, extensible infrastructure, something new always comes along. Ongoing data modernization allows you to develop the necessary mindset, culture and business processes to plow through roadblocks and treat all of your data as a strategic asset.

Read more about data integration and data warehouse modernization from TDWI: 
Modernizing Data Integration to Accommodate New Big Data and New Business Requirements