Sunday, August 23, 2015

What is disruption, anyway?

Our business is a funny business.  Although we all claim to be logical data driven engineers the reality is that we are an industry driven by emotion, not logic.  Sometimes I feel like I'm on the set of "The Apprentice" instead of working for a real company.

There is a constant drive for attention, to stand out in a very crowded field.  This is understandable, quietly building an awesome product is a great way to starve to death.  The side effect of this is that everyone tries to adopt the latest trend and make crazy claims that just aren't substantiated in the real world.

Two things that you hear all the time are that a product is "disruptive" or "innovative."  In fact, there has recently been a spat between some good friends of mine about how "innovative" they are.  It's funny to watch, but probably not doing anyone any good.  As an architect, "innovative" is a throw away word.  An architect is not really worried about innovation.  Either the technology is a fit for the problem at hand or it is not.  Old tech is awesome and safe.  If it works, then that's a great thing, move on to something that's broken.

On the other hand, disruption is something you really have to watch as an architect.  If you are participating in a market that's being disrupted or about to be disrupted, this must play into your planning in the long term if not in the near term.  Thus, when someone says that their product is "disruptive" this is a claim you really need to test.  True disruption is very rare but extremely dangerous.  Strangely, many companies wear this term as a badge of honor.  In reality, it is a risky move that I try to avoid if at all possible. 

If you look up the definition of the word  at, you get:
  • Business. a radical change in an industry, business strategy, etc., especially involving the introduction of a new product or service that creates a new market.
Thus, to be truly disruptive, you must create a new market.  Of course, defining a market is a tricky thing and I'm sure this is what economists argue about over beers.  I don't really know.  From our non-technical perspective, a market is simply a group of buyers and sellers who are competing.  Thus, you could view cars as a single market but in reality it is actually many smaller markets.  Nobody cross-shops a BMW 740iL with a Chevy Spark.  Well, I hope they don't.

Thus, we have things in our business that are truly disruptive like SDN and things that are just innovative like Flash.  That last statement is what Product Managers argue about over beers.  On this one, I'm certain because I've had that argument over beers more than once and I am a Product Manager.

Let's look at SDS and flash and talk about why they are or are not disruptive.  Starting with Flash (and by flash, I mean solid state storage as opposed to rotating media).  Within the enterprise storage space, there are a ton of people who will tell you that flash is a very disruptive technology.  As proof, they point to a host of new startups who are using flash based devices to carve out a new market.  Thus, since there is a new market for all flash arrays, this means that by definition flash is disruptive.  I think that there is something deeper that must be examined before we can declare a technology as disruptive.  There needs to be a massive change associated with this technology first.  To use the definition of the word, a "radical" change.  Is flash in the form of an all flash array truly radical?  In my view, it is not.  All flash arrays are better in many (if not most) ways than traditional magnetic media based arrays but they're not really that different from the outside.  They are still presenting disks, objects or files.  Faster to be sure, but not functionally different than what we've had before.

On the other hand, Software Defined Storage (or SDS) is a very fundamental change in the way systems are architected.  In effect, SDS moves the point of control away from the storage admin and towards the infrastructure consumer.  This accelerates the trend of collapsed operations teams and allows things like dynamic services composition for storage which simply isn't possible with traditional storage architectures.  The repercussions of this is that we can now offer our internal customers a highly differentiated service offering.  That is to say, we move from a legacy "one size fits all" architecture to a modern cloud style services economy where customers have choice in the services they consume.  This is a very "radical" change by anyone's standard.

So, why does this distinction matter to the architect?  It matters because this highly disruptive change will affect the way we design our systems and processes.  A fundamental shift in the way we manage and operate our systems, like cloud, will need to be factored in.  We have seen many enterprises attempt to simply port existing systems and processes to cloud architectures.  These efforts have been less than successful in most cases.  Yes, you CAN run the system, but you will not achieve the benefits that cloud brings without a significant amount of re-architecting. 

Luckily for us, really disruptive changes don't happen very often in the enterprise IT world.  Enterprise IT is slow to change and many of the fads that rip through the industry simply pass us by and never get adopted.  Looking back at my 20 plus years, I can only think of a few truly disruptive technologies to hit the Enterprise IT world.  The shift from centralized computing to distributed computing in the early nineties was extremely disruptive and many of the big enterprise companies of the day (Wang, Tandem, NCR, Amdahl, etc.) have either ceased to exist or faded to a pale shadow of their former selves.  Now, the cloud era is forcing a change back to centralized compute and the enterprise market is changing the game again.  Salesforce and Amazon, the #1 and #2 cloud companies by revenue were not in existence before the cloud and I'm sure further disruption will occur before this round is done.

Disruptions like cloud force us to re-consider our options.  When my old company bought a minicomputer from DEC in the late 80's, this seemed a very safe choice.  They were one of the must trusted names in enterprise computing at the time.  Today, they don't exist.  While the changes to our industry can be unexpected, a large company like DEC doesn't fail overnight.  When you take a look at your suppliers, their overall financial health is an important part of the picture.  If they are making you a deal that seems too good to be true, this may tell you something about their long term viability as a company.

It is the Architect's responsibility to review not only the technical viability of the proposed solution but also the business viability.  If you are building a major new system that depends on vendor support, you need to have a plan b to deal with a failed vendor.  People talk about avoiding "lock in" all the time, but very rarely do they achieve it.  Truly dual-sourced systems are very expensive because it implies that everything must interoperate.  It's usually cheaper to build a single source system but there is some business risk that must be mitigated on the back end.

If you are working with a "disruptive" company or technology, ask yourself this:  who are their competitors?  Is this technology likely to put them out of business?  How big is this company?  Is it likely they will be bought in the next few years?  Is this technology truly disruptive?  Or is it merely innovative?

No comments: