As someone who has paid attention to the revolutions in technology and data over the past twenty years, I like to think that I observe important parallels from time to time. Folks with a sense of history are more likely than some to think about things with a modicum of perspective. We realize that software, programming languages, services and infrastructure evolve over time. Yes, any new app, device or tool may be new now, but history typically provides a useful guide on where it is going.
Take Hadoop, for instance. No doubt it has changed considerably from its early days. When I think about its present and future, though, I can’t help but think about another powerful tech juggernaut: Amazon Web Services (AWS).
From humbling beginnings to the dominant cloud-computing service
When it launched in 2006, AWS was downright difficult to use. In The New Small, I interviewed Colin Hickey, one of AWS’s very first customers. Let’s just say that AWS wasn’t entirely user friendly on a number of levels. As Hickey told me, the pricing was downright confusing. “It was like knowing in advance how much electricity you were going to need at the beginning of the month,” he said.
Contrast that with AWS pricing today. It’s relatively straightforward; you pay for as much as you need. No more, no less.
Beyond simplifying its pricing, though, AWS today is much more robust and diverse than its 2006 counterpart. In fact, AWS is a catchall term that encompasses a number of different but related services including Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS), Amazon Redshift for data warehousing, and many more.
It’s clear that Amazon has bet big on AWS and its wagers have paid off handsomely. As CEO Jeff Bezos told his company’s shareholders:
The difference between baseball and business...is that baseball has a truncated outcome distribution. When you swing, no matter how well you connect with the ball, the most runs you can get is four. In business, every once in awhile, when you step up to the plate, you can score 1,000 runs. This long-tailed distribution of returns is why it’s important to be bold. Big winners pay for so many experiments.
By expanding AWS as much as it has, Amazon has created what every company covets: A virtuous cycle. Profits and new customers spur more development, which spurs more profits – and so on.
Amazon listens intently to developers about what they want added and routinely launches new and important features. Need a more secure private cloud? No problem. Amazon built one inked a $600-million deal with the Central Intelligence Agency (CIA) in 2014. Signing up is exceptionally easy, as a few of my ASU students can attest.
Simon Says
I won't pretend that AWS and Hadoop have evolved in lockstep. They haven't. In my next post, however, I'll list a few of the similarities between the two environments.
In the second part of this series, I’ll explain how Hadoop has evolved over the past five or so years.
Feedback
What say you?
