Virtualized Containers: Performance

It is always fun to watch pundits and industry analysts proclaim the death of this technology or that technology.  This is something that happens regularly enough that it's become a bit of a sport.

In my case, because I work at VMware, I often get told that containerization is the end of virtualization.  This is a very interesting contention that doesn't seem to be based on any sort of factual evidence.  The conversation usually starts out with me showing our new Photon Platform or vSphere Integrated Containers (vIC) product.  Fundamentally, both of these products allow you to run containers on ESXi.  vIC allows you to run Docker naively in vSphere and Photon Platform is a new platform for running containers and cloud native workloads.  Or to put it another way, they allow you to virtualize your containers.  After I show the demo, there's a polite pause.  Then comes "the question" that I always expect.  They ask "Why would I want to virtualize my containers?  Everyone knows that containers run best on bare metal."

The really fun part is that when you press, there really isn't any sort of basis for the assertion.  Mostly, the objection seems to come down to performance.  They want to run containers on bare metal Linux because "it's faster."  Again, we ask what that assertion is based on.  The really clever folks will Google it for you and usually come up with this paper from IBM.  While this paper is fascinating and very well researched and documented, there is one fatal flaw:  it's based on KVM.  What IBM is really saying is that bare metal containers are faster than KVM.  Well, OK.  I'll let the KVM folks answer that one.  I don't work on KVM, I work on ESXi.  The thing is, we know that ESXi is significantly faster than KVM for some workloads.  See this or this.  So, what does that mean?  It means that we don't actually know anything based on this report.

Thankfully, VMware has done their own research.  They have found that in smaller workloads, there is very little virtualization overhead for ESXi.  In an interesting development, they also found that some workloads like Redis, actually run FASTER when you virtualize them.

So, where does the truth lie?  As is usually the case, your mileage will vary.  If performance is a key issue for you, test your workload and draw your own conclusions.

Just don't tell me that "everyone knows" that containers run slower when virtualized.  That just ain't so.

CNA in the Enterprise?

Following up from my post about joining the Cloud Native team at VMware, I'd like to transition to the challenge that we've decided to take on:  Moving Cloud Native into the Enterprise.

Like IaaS and other cloud technologies, the idea of enterprises running cloud native applications is all the rage right now.  Every CIO and every IT architect I talk to has some plan or research project going on around Docker, Kubernetes, Mesos or some other aspect of the cloud native stack.  This is awesome for a product manager like me because it means everyone is interested in what I'm working on.  It's very easy to draw a crowd by showing off vSphere Integrated Containers or one of the other cloud native projects we're working on.

Unfortunately, the current state of the market is not very mature.  In fact, it very much reminds me of the awesome Dan Ariely quote about big data: 

"Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it..."

You could basically insert "Docker" or "Cloud Native" in the quote above for Big Data and you wouldn't be far wrong.  Similar to big data, there are some folks out there doing amazing work.  However, for most mere mortals, this is a completely theoretical discussion at this time.  Pretty much like every locker room conversation you had in middle school.

The reality is that making Cloud Native work in the enterprise is going to be VERY VERY hard.  There are some very significant barriers to entry here:

1. There are millions of lines of code running key business functions that will need to be completely replaced.  You cannot simply port existing code to cloud native, you need to re-write the application and re-architect it to get the benefits.
2. Your existing development process will have to be abandoned.  The traditional dev/test/stage/prod cycle you are using now will not cut the mustard.  One of the key values you get from Cloud Native is speed and agility.  That won't happen without changing out your processes.
3. Cloud native assumes a robust PaaS layer.  If you look closely, there is a massive "hand wave" in most architectures when it comes to persistence.  Nobody runs databases in containers.  This is because platforms like AWS already have a great PaaS layer.  Why build your own object store when you have S3?  Do you have a full PaaS stack inside your firewall?
4. Your current security and it governance policies will have to be re-written.  It is extremely unlikely that your current policy and governance structure was designed for the type of rapid iteration and agility implied by cloud native.  Do you have an Architectural Review Board?  Forget about that.  Centralized governance model?  Nope.
5. Deploying within your firewall assumes you already have a robust PaaS and IaaS platform.  Do you?  Our research shows fewer than 10% of enterprise IT shops have a fully deployed IaaS platform.
6. Deploying outside your firewall (AWS, GCE, Azure, etc.) assumes that you have policies and procedures for managing things like data protection, security, etc. in a public cloud context.  Do you?

Given the issues above, is it even practical to deploy or build cloud native applications in my enterprise?  Most definitely yes.  The reality is that cloud native solves so many problems with traditional application development that it's worth all the hassle of changing out your existing infrastructure, processes and policies.  However, you should begin this journey with open eyes.  Like every other technical project you've ever done, this one will have issues, complications and unforeseen consequences.

Assuming that you're not going to re-write your existing line of business apps for fun, my assumption is that your first cloud native application will be a net new application.  It's going to be much easier if you apply this toolkit to a new development effort than if you attempt to retrofit an existing application.  Therefore, you have the opportunity to create a new team with a new set of dev tools and processes.  Thus, you are really talking about of a net new team writing a new application.  Setting aside the policy and procedure issues from above (not because they're trivial, but because I can't generalize them) your biggest problem comes down to PaaS.

It turns out that running cloud native assumes you already have a cloud and all of the fun things that this implies.  If you already have a robust IaaS and PaaS environment running, you're good to go.

Unfortunately, we know that for most customers this is simply not true.  We know that you don't have this infrastructure running because we've spend the last year talking to literally hundreds of enterprise customers about this very topic.

The next thing that usually happens is that engineering teams go off and get an account on AWS, GCE, Azure or another cloud provider service.  This is awesome because it allows them to begin writing code almost immediately.  All the services that we've already talked about exist up there and they work great.  They code away like mad and everyone is happy.  Right?

Well, not really.  If you are a software company and product velocity is the only thing that matters, this may be a very acceptable state of affairs.  However, if other things matter to you like data sovereignty, multi-sourcing, ROI, etc. then you might not be OK with being locked into a single hoster.  This means that you don't want to consume PaaS services that are limited to a single cloud.

Thus, we get back to creating your own PaaS layer.

I believe that this will be the single greatest stumbling block for enterprises who seek to adopt cloud native.

This is one reason why our team is working on tools and services to make this simpler.  We need a way to deploy "just enough PaaS" quickly and easily inside the firewall and outside the firewall.  We need a consistent services layer that will run on any cloud.  We need a single management interface for all of those instances running across any cloud.

Hmmm.

Sounds like a good idea for a product, right?

Stay tuned.......

My Move to Cloud Native

In late July, I decided to move back to VMware.  There were many reasons for this change, but one of them was the opportunity to work in a group within VMware called "Cloud Native Applications" or CNA.  While, I've been a "cloud" guy for years; I've mostly worked on IaaS.  I was on the public IaaS team at Oracle Public Cloud but I worked primarily on the Private Cloud space at VMware and NetApp.  Thus, CNA is adjacent to what I was doing but very different in practice.  Naturally, I have been asked many times to explain exactly what CNA is and what our mission in life is.  In order to do that, I had to define exactly what we mean when we say cloud native and I thought that this would be a good topic for a blog post.

The very short answer is that when we say Cloud Native Applications, we mean a class of applications that were born in the cloud and natively use the infrastructure services of cloud.  However, that answer doesn't really give you a good picture of how fundamentally different CNA is from the rest of VMware. 

As a Product Manager who owned features within vSphere, I can tell you that vSphere is a great business for VMware and probably will be for the foreseeable future.  It is one of those rare products that was so good, it created an industry.  Like Windows or Linux, there is an entire ecosystem around vSphere and it's component technologies.  As you can imagine, there is a great deal of pressure as a member of the vSphere team not to mess that up.  When you don't take your time and things slip, people notice.  Like front page Wall Street Journal notice.  Thus, there are all kinds of systems and processes in place within vSphere to make sure that mistakes are not made and to address any issues that do come up.  This is a good thing and I think that we're doing a pretty good job keeping that product stable while introducing new features at a measured, steady pace.

However, there are other markets that are moving much more quickly.  In those cases, the type of process driven organization that produces a product like vSphere cannot keep pace.  Thus, you cannot have one team working on a very mature product like vSphere and also going after a brand new, fast moving market.  Thus, the creation of the CNA team at VMware.  Our mission is to serve the cloud native market and to introduce new products, platforms and services for this market.

Naturally, because this market is so new, there is some debate about what this means and how to go after it.  Our first challenge as a team was to identify why this market was different and how customers in this market make decisions about what technologies to use.  It has become abundantly clear to us that unlike vSphere, CNA is fundamentally a developer driven organization.  The move to cloud is largely driven by developers seeking to solve their development, deployment and operations problems by getting away from traditional architectures and infrastructures.

Thus, the fundamental truth about cloud native is that it revolves around an architectural paradigm.  The notion that we need to change the way we design, build and operate is what lies at the heart of what we mean when we say cloud native.  This paradigm shift then drives technologies to meet the new requirements of these applications.  Note the cause-effect loop here.  Different REQUIREMENTS are driving different TECHNOLOGIES, not the other way round.  This important distinction is lost on some in the industry and will ultimately lead them into bad decisions and products.  Thus, things like Docker are very interesting because of problems they solve, not as technology per se. 

So, how do we define this new architectural paradigm?  One popular way is the "12 factor app" as defined at 12factor.net.  While this site is super detailed and provides everything you'd want to know about creating your own cloud native application, it may be TOO detailed for some.  There are a couple of things we can summarize out of this and generalize for our use.

Fundamentally, Cloud Native is about four requirements or assumptions for building applications:

1. Cloud Native implies that applications are built with the assumption that infrastructure is inherently unreliable.  All availability, data replication and other availability features are moved away from infrastructure and into the application.
2. Cloud Native assumes that the infrastructure is "Software Defined" and thus fungible.  The assumption is that I can re-configure my infrastructure simply via an API call and thus control the underlying platform from within the application.  
3. Cloud Native assumes that applications need to scale to internet scale.  That is to say, Cloud Native applications should inherently "scale out" instead of "scale up" because you eventually run out of "up".
4. Cloud Native assumes that technology velocity is more important than ROI.  Yes, cost is always a factor in the real world, but there are always trade-offs.  If your goal is to be the lowest cost provider, you are unlikely to be on the leading edge.  Thus, when you optimize for speed, you may incur additional cost and that's OK.

Here is the really interesting thing:    ALL my customers say they want cloud native, but MOST of them are not willing to accept the assumptions and requirements above.

Similar to what happened a few years ago when IaaS started to penetrate the enterprise, there is a huge amount of "buzz" around cloud native and all things cloud within enterprise today.  And just like I did five years ago, I am talking to customers about why this ISN'T a good idea for them.  At least right now.

I cannot tell you how many companies I have worked with around IaaS only to see them fail utterly.  This is not to say that their cloud didn't work.  The technology works just fine, thank you very much.  The problem is that they did not change the way IT worked.  Thus, the "impedance mismatch" between what IT saw as the proper way to do things and the way their internal customers wanted to consume cloud services as an impossible barrier to overcome.

The first question you must ask yourself as an IT leader is:  "what business am I in today and what business do I want to be in tomorrow?"  I think the last item on the list above is probably the killer for most companies.  If your company views IT as a cost center which needs to be controlled, you can never adopt the same methodologies and architectures from companies like Twitter and Google there technology is a profit center.  Never.

On the other hand, if those requirements match your requirements, there are some really interesting things that can be done with the technology stack that has been developed by those companies.  There are lessons to be learned about the way to scale out huge web farms and run large businesses on entrusted platforms.  When those lessons are applied, there are some architectural patterns that stand out:

1. Containerized workloads.  This is not the only way to build cloud native applications, but it is the dominant trend and we think it’s one of the better ways.
2. Stateless workloads.  Not to say that state is never persisted anywhere, what we mean is that any individual host or VM should not hold state that can’t be recovered easily.   To say it another way, I don’t “fix” the container or the VM, I kill it and start another.
3. Language and framework agnostic.  The rise of REST and things like Zookeeper and Etcd have made it possible to integrate components that were not developed together and have no common technology platform.  Thus, it becomes "run what you brung" and language becomes a developer choice.
4. Continuous Integration.  If speed is your #1 driver, then traditional dev/test/stage/prod cycles are not going to work for you.  Naturally, that means you need to have a programmable infrastructure, but we stated that in our requirements section.
5. REST.  The reality is that everyone is moving to REST.  Things like SOAP and RPC are not fast enough and are too "leaky" to maintain the level of abstraction and the scale required for cloud native.

Thus, we see that these technologies are interesting because they make it simpler to achieve our stated objectives and meet the requirements we've stated above.  This also means that while these are currently the way we are doing things, there is no guarantee that this will not change.  Thus, there is no point in creating a "Docker" product group within VMware.  We also urge our customers to focus on requirements and let technology decisions flow from there.  Yes, we love and support Docker here at VMware's CNA group, no we don't think this is the ONLY way to go about things.  Our focus here is to help our customer develop, build and deploy cloud native applications, regardless of technology or platform.



 

What is disruption, anyway?

Our business is a funny business.  Although we all claim to be logical data driven engineers the reality is that we are an industry driven by emotion, not logic.  Sometimes I feel like I'm on the set of "The Apprentice" instead of working for a real company.

There is a constant drive for attention, to stand out in a very crowded field.  This is understandable, quietly building an awesome product is a great way to starve to death.  The side effect of this is that everyone tries to adopt the latest trend and make crazy claims that just aren't substantiated in the real world.

Two things that you hear all the time are that a product is "disruptive" or "innovative."  In fact, there has recently been a spat between some good friends of mine about how "innovative" they are.  It's funny to watch, but probably not doing anyone any good.  As an architect, "innovative" is a throw away word.  An architect is not really worried about innovation.  Either the technology is a fit for the problem at hand or it is not.  Old tech is awesome and safe.  If it works, then that's a great thing, move on to something that's broken.

On the other hand, disruption is something you really have to watch as an architect.  If you are participating in a market that's being disrupted or about to be disrupted, this must play into your planning in the long term if not in the near term.  Thus, when someone says that their product is "disruptive" this is a claim you really need to test.  True disruption is very rare but extremely dangerous.  Strangely, many companies wear this term as a badge of honor.  In reality, it is a risky move that I try to avoid if at all possible. 

If you look up the definition of the word  at dictionary.com, you get:

• Business. a radical change in an industry, business strategy, etc., especially involving the introduction of a new product or service that creates a new market.

Thus, to be truly disruptive, you must create a new market.  Of course, defining a market is a tricky thing and I'm sure this is what economists argue about over beers.  I don't really know.  From our non-technical perspective, a market is simply a group of buyers and sellers who are competing.  Thus, you could view cars as a single market but in reality it is actually many smaller markets.  Nobody cross-shops a BMW 740iL with a Chevy Spark.  Well, I hope they don't.

Thus, we have things in our business that are truly disruptive like SDN and things that are just innovative like Flash.  That last statement is what Product Managers argue about over beers.  On this one, I'm certain because I've had that argument over beers more than once and I am a Product Manager.

Let's look at SDS and flash and talk about why they are or are not disruptive.  Starting with Flash (and by flash, I mean solid state storage as opposed to rotating media).  Within the enterprise storage space, there are a ton of people who will tell you that flash is a very disruptive technology.  As proof, they point to a host of new startups who are using flash based devices to carve out a new market.  Thus, since there is a new market for all flash arrays, this means that by definition flash is disruptive.  I think that there is something deeper that must be examined before we can declare a technology as disruptive.  There needs to be a massive change associated with this technology first.  To use the definition of the word, a "radical" change.  Is flash in the form of an all flash array truly radical?  In my view, it is not.  All flash arrays are better in many (if not most) ways than traditional magnetic media based arrays but they're not really that different from the outside.  They are still presenting disks, objects or files.  Faster to be sure, but not functionally different than what we've had before.

On the other hand, Software Defined Storage (or SDS) is a very fundamental change in the way systems are architected.  In effect, SDS moves the point of control away from the storage admin and towards the infrastructure consumer.  This accelerates the trend of collapsed operations teams and allows things like dynamic services composition for storage which simply isn't possible with traditional storage architectures.  The repercussions of this is that we can now offer our internal customers a highly differentiated service offering.  That is to say, we move from a legacy "one size fits all" architecture to a modern cloud style services economy where customers have choice in the services they consume.  This is a very "radical" change by anyone's standard.

So, why does this distinction matter to the architect?  It matters because this highly disruptive change will affect the way we design our systems and processes.  A fundamental shift in the way we manage and operate our systems, like cloud, will need to be factored in.  We have seen many enterprises attempt to simply port existing systems and processes to cloud architectures.  These efforts have been less than successful in most cases.  Yes, you CAN run the system, but you will not achieve the benefits that cloud brings without a significant amount of re-architecting. 

Luckily for us, really disruptive changes don't happen very often in the enterprise IT world.  Enterprise IT is slow to change and many of the fads that rip through the industry simply pass us by and never get adopted.  Looking back at my 20 plus years, I can only think of a few truly disruptive technologies to hit the Enterprise IT world.  The shift from centralized computing to distributed computing in the early nineties was extremely disruptive and many of the big enterprise companies of the day (Wang, Tandem, NCR, Amdahl, etc.) have either ceased to exist or faded to a pale shadow of their former selves.  Now, the cloud era is forcing a change back to centralized compute and the enterprise market is changing the game again.  Salesforce and Amazon, the #1 and #2 cloud companies by revenue were not in existence before the cloud and I'm sure further disruption will occur before this round is done.

Disruptions like cloud force us to re-consider our options.  When my old company bought a minicomputer from DEC in the late 80's, this seemed a very safe choice.  They were one of the must trusted names in enterprise computing at the time.  Today, they don't exist.  While the changes to our industry can be unexpected, a large company like DEC doesn't fail overnight.  When you take a look at your suppliers, their overall financial health is an important part of the picture.  If they are making you a deal that seems too good to be true, this may tell you something about their long term viability as a company.

It is the Architect's responsibility to review not only the technical viability of the proposed solution but also the business viability.  If you are building a major new system that depends on vendor support, you need to have a plan b to deal with a failed vendor.  People talk about avoiding "lock in" all the time, but very rarely do they achieve it.  Truly dual-sourced systems are very expensive because it implies that everything must interoperate.  It's usually cheaper to build a single source system but there is some business risk that must be mitigated on the back end.

If you are working with a "disruptive" company or technology, ask yourself this:  who are their competitors?  Is this technology likely to put them out of business?  How big is this company?  Is it likely they will be bought in the next few years?  Is this technology truly disruptive?  Or is it merely innovative?

The Architect’s Place in Society

When you think about our modern world, you quickly see that almost everything you depend on has some sort of software component.  Even things that used to be simple, like the food we eat is now sitting on a very complex and technological base that always includes complex software systems.  Each of these systems was designed by some flavor of software architect.  Some were very good at their jobs, but inevitably, some were not.  If we think about other complex tasks that have such fundamental impact on our lives, there are much more stringent controls on how these systems are designed and operated.

In most cases, this is a byproduct of maturity.  Things like roads and medicine have been around much longer than things like software.  Thus, the state of the industry and the regulation surrounding them is also much more mature.  We can look back into history to see epic failures in all of these mature disciplines.  Each failure has added to the body of knowledge about what not to do.  This results in our current regulatory and industry best practice structures.  None of those things exist in our business.

Yes, we have our share of epic failures, but in most cases these failures do not lead to loss of life.  Thus, we are largely free to make mistakes and we are the only ones hurt from those mistakes.  Or so it appears.

What started out as domain for academics and then later for hobbyists has become very foundational to our modern world.  It is the internet which thrusts the architect into the center of almost everything we do.  Today, the impacts of Internet failure are largely commercial.  A major company gets taken down by hackers and this is annoying or even worrying but it is not catastrophic.  President Obama described the Sony attack “an act of cyber vandalism.”  Thus, it is lower in the pantheon of crimes one can commit.  However, there are other systems, other platforms that would be far more dangerous.  Things like building control software or air traffic control systems.  Water and power management systems.

Ironically, these systems are very slow to advance and seem almost comical in their primitive design to the child of the internet.  However, it is this lack of connectivity that makes them safe and secure today.  A prime example of this is the way US nuclear weapons silos work.  “Ancient” technologies like eight inch floppydisks with zero connectivity are the norm.  It turns out this is a very good thing.  Imagine someone like North Korea getting the root admin password for the computer that launches a US ICBM.  This is something that doesn’t bear thinking about and is not possible today.  Looking forward, we know that this cannot continue.  At some point, the architectures and platforms we are designing today to sell books and play games will underlie the power grid, the air traffic control system and everything else we do.  As these systems evolve and mature, our industry practice and regulatory structures must also mature and evolve.

Thinking about the future of our industry, we must think of ourselves as critical to the functioning of our society.  We should be holding ourselves to the same standards that doctors, lawyers and civil engineers hold themselves to.  While I’m sure there are plenty of examples of where these systems have failed, we also know that in the vast majority of cases, the system works.  Think of your own life experience.  If your child is very ill in the middle of the night, would you hesitate to bring her to the emergency room?  Would you wait until you knew that the “good” doctor you trust was available?  Certainly not.  You would immediately rush her to the nearest emergency room with the full expectation that they would know what to do.  And in 99.99% of cases, that is exactly correct.  The vast majority of doctors in this country are very skilled and know what they’re doing.  The system of board certifications, internships and residency produces a very high level of professionalism that results in a very high standard of care in this country and in most western societies.  There are variations and you could make arguments about what could be improved (certainly, that’s true in the USA) but overall, the system works.

Take this same example to software.  Let’s say that your daughter had a digital virus.  Let’s say that for some inexplicable reason, you needed a computer technician to save her life.  What would you do?  Would you go down to Best Buy and have the dude from “Geek Squad” have a go?  I definitely would not.  I know that those people have no idea what they’re doing (no offence to Geek Squad or Best Buy, but I bet they would not want my daughter’s life in their hands either).  I have some people I know and I trust but there is no way in hell I would allow some dude to work on her in a life threatening situation.  Granted, I am a computer professional so I know things that non-techie folks don’t, but I don’t think that our industry would work the same way if children’s lives depended on the outcome.   It just would not happen.

Naturally, this does not actually occur and this is why our standards are so loose.  What I’m asking you do to is to think forward when actual lives ARE on the line.  When this DOES actually matter.  Then what?  How will we respond as an industry?  This is the critical question that we must answer.

As architects, our job is not to simply examine the world as it is today.  Our job is to project into the future.  To seek this future truth that does not yet exist and cause it to exist.  This is the only way we can know if things are true.

I am currently working on a new book which will explore this issue and other issues around the professional development.  Feel free to join in on the comments below to share your thoughts about this issue.

 

The one eyed man

"In the land of the blind, the one eyed man is king."

I've always wondered about this saying.  It strikes me that this isn't the best analogy to use when talking about the unique value that each of us bring to the world.  If you think about it, a sighted person who is suddenly thrust into a community of only blind people would not be hailed as a conqueror.  No, they would be ostracized as a freak, perhaps feared, most certainly the subject of ridicule.  This is the human condition.  We are social animals and instinctively seek the safety of conformity.

Strangely, we live in a society that puts individualism at a premium.  And then we seek out and attempt to destroy anyone who's actually different.  What an interesting contradiction.

In the workplace, the high performers are often praised and set as examples above the rest of the team.  However, the team will usually seek to bring this person down and will resist a single member outshining the rest.  It's just a natural competitive instinct to resist being placed into a lesser category.  In reality, the most successful teams attempt to share the credit and bring the entire team up at the same time.  Naturally, this is not actually true.  In any group, there will be variations in performance and thus you will always have a #1 performer.  This inherent dishonesty in team dynamics has always puzzled me.  It's necessary, but seems like a dishonest way to go about things.

Thinking back to my childhood, I was raised in the '70s in San Francisco.  One thing that was consistent drummed into me was that we were all equals as humans.  All people are the same.  If you thought of someone as less than you, that was discrimination.  Today, I understand that this was simply a reaction to the horrible oppression of blacks and other minorities; at the time I was just a kid so I took the advice literally.  I really believed that we were all the same.  That men and women were the same.  That there was no difference between an American and a Japanese person.

Then I grew up.

I found out, much to my horror, that this is simply not true.  Men and women are NOT the same.  We have different bodies, we generate different hormones, we look at the world differently.  When I started traveling extensively in my '20's I realized that Americans were very different than Europeans and very different than Asians.  This is a very good thing.  Through diversity, we find strength.  However, diversity does not imply that we are all equally good at everything.  We are not the same, we are not equal, interchangeable identical parts in a giant machine.

Thus, the idea that we're all equal just doesn't work.  We need to have a level playing field, we need to have the same opportunities.  But we're not the same.  It is OK to discriminate between people, but we need to do it on merit, not the color of their skin.

This gets us to the point of the argument:  In business, democracy does not work.  We don't all get the same vote because we're not all equal.

Why the dichotomy?  Why use democracy for political life but not for business?  Because in business, there is a winner.  That changes everything.

When you have winners and losers, there is a required difference and ranking.  You must have a #1 by definition or there is no winner.

Yes, I'm aware that there are businesses out there where "making money" is not the claimed objective.  That's fine.  I'm not saying it's about making money all the time.  What I'm saying is that all businesses have a purpose.  They either achieve this purpose or they do not.  This is winning or losing.

The phrase that I've always liked is "Meritocracy."  The implication in a meritocracy is that people rise or fall based on merit.  There is no assumption of equality, but there is an assumption of fairness.  Not everyone wants to live in a meritocracy.  I assume that is because they don't like the competitive nature and don't want to be constantly stack ranked against others.  However, I would argue that they're simply fooling themselves.  Every company I've ever worked for has had some sort of ranking system.  It may be hidden and very covert, but it's there.  How else do you assign promotions and bonuses?  I don't want to work somewhere that awards promotions based on "time in grade."  Just being able to breathe regularly does not seem like the correct criteria for becoming a supervisor.

I think another key difference here is the ability to vote with my feet.  In government, you don't really have the option of leaving because you don't like the current administration.  In my line of work, I do have that option.  It would be different if my career choices were limited due to circumstance, but I'm lucky to work in an industry that's still growing.  Thus, when I have no choice in leaving I prefer equality because that gives me at least some guarantee in life.

Thus, "fairness" is not the same thing as "equality."  So, what do you want in your workplace?  Fairness?  Or Equality?  I know how I would vote.

What's the difference between SDS and Storage Virtualization??

One of the things that always happens when a new term comes up in our industry is confusion about what it really means.  This is after the phase when everyone claims that the term is actually not new and is something that they've been doing all along.  I've already blogged about that phase of SDS here.

Now we are in the (very healthy) phase where we all argue about what the term REALLY means.  This is really good progress!!

One thing that I'm seeing consistently on the web is references to things that look suspiciously like Storage Hypervisors being defined as Software Defined Storage.  Let's be clear:  Storage Hypervisors are a very good thing, but they are not the same as SDS.

SDS is a Storage Management Architectural Pattern that allows for the remote management and configuration of storage assets.  Storage Virtualization is a technique for creating "virtual" storage assets (LUNS, Shares, Objects) on top of heterogeneous storage platforms.

Let's say you had four arrays from four different manufacturers.  To make this simple, let's say they're all block devices that support iSCSI (yes, simplistic but stay with me here).  If I were to apply an SDS style architecture, I would simply create a control plane (like ViPR from EMC) to manage them.  Whenever I wanted to do anything (provision a LUN, unmask a LUN, turn on Dedupe, etc.) I would go to the SDS control plane and issue my requests.  I would be ignorant of the underlying implementation (it's an ABSTRACTED control plane) but what I get back is an IP address pointing to the iSCSI end point on the actual SAN that I already had.  Thus, SDS allows me to abstract the management of my SAN but does not fundamentally change my data path.  Got it?  OK...

Let's take the same example, and install a Storage Hypervisor.  In this case, I install an appliance (either virtual or physical) and that appliance then mounts LUNs against my existing four arrays.  See the immediate difference here?  In this case, the Storage Hypervisor is the consumer of the storage.  Then, the storage hypervisor does it what it does (formats with it's own file system, adds meta data, etc.) and then RE-PRESENTS this storage to the actual clients.  This can be in the form of new LUNS or shares or whatever.  AT THE SAME TIME, this storage hypervisor might have a very slick SDS style abstracted management layer.  However, these are totally separate things.

For an example of this, take a look at the V-Series from NetApp.  These are basically NetApp head units that mount LUNs from non NetApp arrays.  They then format the luns with WAFL and pretend they're disks.  This is a great example of a "storage hypervisor" but not really SDS since it does not offer an abstracted control plane.

Got it?  Awesome!!