The Architect’s Place in Society

When you think about our modern world, you quickly see that almost everything you depend on has some sort of software component.  Even things that used to be simple, like the food we eat is now sitting on a very complex and technological base that always includes complex software systems.  Each of these systems was designed by some flavor of software architect.  Some were very good at their jobs, but inevitably, some were not.  If we think about other complex tasks that have such fundamental impact on our lives, there are much more stringent controls on how these systems are designed and operated.

In most cases, this is a byproduct of maturity.  Things like roads and medicine have been around much longer than things like software.  Thus, the state of the industry and the regulation surrounding them is also much more mature.  We can look back into history to see epic failures in all of these mature disciplines.  Each failure has added to the body of knowledge about what not to do.  This results in our current regulatory and industry best practice structures.  None of those things exist in our business.

Yes, we have our share of epic failures, but in most cases these failures do not lead to loss of life.  Thus, we are largely free to make mistakes and we are the only ones hurt from those mistakes.  Or so it appears.

What started out as domain for academics and then later for hobbyists has become very foundational to our modern world.  It is the internet which thrusts the architect into the center of almost everything we do.  Today, the impacts of Internet failure are largely commercial.  A major company gets taken down by hackers and this is annoying or even worrying but it is not catastrophic.  President Obama described the Sony attack “an act of cyber vandalism.”  Thus, it is lower in the pantheon of crimes one can commit.  However, there are other systems, other platforms that would be far more dangerous.  Things like building control software or air traffic control systems.  Water and power management systems.

Ironically, these systems are very slow to advance and seem almost comical in their primitive design to the child of the internet.  However, it is this lack of connectivity that makes them safe and secure today.  A prime example of this is the way US nuclear weapons silos work.  “Ancient” technologies like eight inch floppydisks with zero connectivity are the norm.  It turns out this is a very good thing.  Imagine someone like North Korea getting the root admin password for the computer that launches a US ICBM.  This is something that doesn’t bear thinking about and is not possible today.  Looking forward, we know that this cannot continue.  At some point, the architectures and platforms we are designing today to sell books and play games will underlie the power grid, the air traffic control system and everything else we do.  As these systems evolve and mature, our industry practice and regulatory structures must also mature and evolve.

Thinking about the future of our industry, we must think of ourselves as critical to the functioning of our society.  We should be holding ourselves to the same standards that doctors, lawyers and civil engineers hold themselves to.  While I’m sure there are plenty of examples of where these systems have failed, we also know that in the vast majority of cases, the system works.  Think of your own life experience.  If your child is very ill in the middle of the night, would you hesitate to bring her to the emergency room?  Would you wait until you knew that the “good” doctor you trust was available?  Certainly not.  You would immediately rush her to the nearest emergency room with the full expectation that they would know what to do.  And in 99.99% of cases, that is exactly correct.  The vast majority of doctors in this country are very skilled and know what they’re doing.  The system of board certifications, internships and residency produces a very high level of professionalism that results in a very high standard of care in this country and in most western societies.  There are variations and you could make arguments about what could be improved (certainly, that’s true in the USA) but overall, the system works.

Take this same example to software.  Let’s say that your daughter had a digital virus.  Let’s say that for some inexplicable reason, you needed a computer technician to save her life.  What would you do?  Would you go down to Best Buy and have the dude from “Geek Squad” have a go?  I definitely would not.  I know that those people have no idea what they’re doing (no offence to Geek Squad or Best Buy, but I bet they would not want my daughter’s life in their hands either).  I have some people I know and I trust but there is no way in hell I would allow some dude to work on her in a life threatening situation.  Granted, I am a computer professional so I know things that non-techie folks don’t, but I don’t think that our industry would work the same way if children’s lives depended on the outcome.   It just would not happen.

Naturally, this does not actually occur and this is why our standards are so loose.  What I’m asking you do to is to think forward when actual lives ARE on the line.  When this DOES actually matter.  Then what?  How will we respond as an industry?  This is the critical question that we must answer.

As architects, our job is not to simply examine the world as it is today.  Our job is to project into the future.  To seek this future truth that does not yet exist and cause it to exist.  This is the only way we can know if things are true.

I am currently working on a new book which will explore this issue and other issues around the professional development.  Feel free to join in on the comments below to share your thoughts about this issue.

 

The one eyed man

"In the land of the blind, the one eyed man is king."

I've always wondered about this saying.  It strikes me that this isn't the best analogy to use when talking about the unique value that each of us bring to the world.  If you think about it, a sighted person who is suddenly thrust into a community of only blind people would not be hailed as a conqueror.  No, they would be ostracized as a freak, perhaps feared, most certainly the subject of ridicule.  This is the human condition.  We are social animals and instinctively seek the safety of conformity.

Strangely, we live in a society that puts individualism at a premium.  And then we seek out and attempt to destroy anyone who's actually different.  What an interesting contradiction.

In the workplace, the high performers are often praised and set as examples above the rest of the team.  However, the team will usually seek to bring this person down and will resist a single member outshining the rest.  It's just a natural competitive instinct to resist being placed into a lesser category.  In reality, the most successful teams attempt to share the credit and bring the entire team up at the same time.  Naturally, this is not actually true.  In any group, there will be variations in performance and thus you will always have a #1 performer.  This inherent dishonesty in team dynamics has always puzzled me.  It's necessary, but seems like a dishonest way to go about things.

Thinking back to my childhood, I was raised in the '70s in San Francisco.  One thing that was consistent drummed into me was that we were all equals as humans.  All people are the same.  If you thought of someone as less than you, that was discrimination.  Today, I understand that this was simply a reaction to the horrible oppression of blacks and other minorities; at the time I was just a kid so I took the advice literally.  I really believed that we were all the same.  That men and women were the same.  That there was no difference between an American and a Japanese person.

Then I grew up.

I found out, much to my horror, that this is simply not true.  Men and women are NOT the same.  We have different bodies, we generate different hormones, we look at the world differently.  When I started traveling extensively in my '20's I realized that Americans were very different than Europeans and very different than Asians.  This is a very good thing.  Through diversity, we find strength.  However, diversity does not imply that we are all equally good at everything.  We are not the same, we are not equal, interchangeable identical parts in a giant machine.

Thus, the idea that we're all equal just doesn't work.  We need to have a level playing field, we need to have the same opportunities.  But we're not the same.  It is OK to discriminate between people, but we need to do it on merit, not the color of their skin.

This gets us to the point of the argument:  In business, democracy does not work.  We don't all get the same vote because we're not all equal.

Why the dichotomy?  Why use democracy for political life but not for business?  Because in business, there is a winner.  That changes everything.

When you have winners and losers, there is a required difference and ranking.  You must have a #1 by definition or there is no winner.

Yes, I'm aware that there are businesses out there where "making money" is not the claimed objective.  That's fine.  I'm not saying it's about making money all the time.  What I'm saying is that all businesses have a purpose.  They either achieve this purpose or they do not.  This is winning or losing.

The phrase that I've always liked is "Meritocracy."  The implication in a meritocracy is that people rise or fall based on merit.  There is no assumption of equality, but there is an assumption of fairness.  Not everyone wants to live in a meritocracy.  I assume that is because they don't like the competitive nature and don't want to be constantly stack ranked against others.  However, I would argue that they're simply fooling themselves.  Every company I've ever worked for has had some sort of ranking system.  It may be hidden and very covert, but it's there.  How else do you assign promotions and bonuses?  I don't want to work somewhere that awards promotions based on "time in grade."  Just being able to breathe regularly does not seem like the correct criteria for becoming a supervisor.

I think another key difference here is the ability to vote with my feet.  In government, you don't really have the option of leaving because you don't like the current administration.  In my line of work, I do have that option.  It would be different if my career choices were limited due to circumstance, but I'm lucky to work in an industry that's still growing.  Thus, when I have no choice in leaving I prefer equality because that gives me at least some guarantee in life.

Thus, "fairness" is not the same thing as "equality."  So, what do you want in your workplace?  Fairness?  Or Equality?  I know how I would vote.

What's the difference between SDS and Storage Virtualization??

One of the things that always happens when a new term comes up in our industry is confusion about what it really means.  This is after the phase when everyone claims that the term is actually not new and is something that they've been doing all along.  I've already blogged about that phase of SDS here.

Now we are in the (very healthy) phase where we all argue about what the term REALLY means.  This is really good progress!!

One thing that I'm seeing consistently on the web is references to things that look suspiciously like Storage Hypervisors being defined as Software Defined Storage.  Let's be clear:  Storage Hypervisors are a very good thing, but they are not the same as SDS.

SDS is a Storage Management Architectural Pattern that allows for the remote management and configuration of storage assets.  Storage Virtualization is a technique for creating "virtual" storage assets (LUNS, Shares, Objects) on top of heterogeneous storage platforms.

Let's say you had four arrays from four different manufacturers.  To make this simple, let's say they're all block devices that support iSCSI (yes, simplistic but stay with me here).  If I were to apply an SDS style architecture, I would simply create a control plane (like ViPR from EMC) to manage them.  Whenever I wanted to do anything (provision a LUN, unmask a LUN, turn on Dedupe, etc.) I would go to the SDS control plane and issue my requests.  I would be ignorant of the underlying implementation (it's an ABSTRACTED control plane) but what I get back is an IP address pointing to the iSCSI end point on the actual SAN that I already had.  Thus, SDS allows me to abstract the management of my SAN but does not fundamentally change my data path.  Got it?  OK...

Let's take the same example, and install a Storage Hypervisor.  In this case, I install an appliance (either virtual or physical) and that appliance then mounts LUNs against my existing four arrays.  See the immediate difference here?  In this case, the Storage Hypervisor is the consumer of the storage.  Then, the storage hypervisor does it what it does (formats with it's own file system, adds meta data, etc.) and then RE-PRESENTS this storage to the actual clients.  This can be in the form of new LUNS or shares or whatever.  AT THE SAME TIME, this storage hypervisor might have a very slick SDS style abstracted management layer.  However, these are totally separate things.

For an example of this, take a look at the V-Series from NetApp.  These are basically NetApp head units that mount LUNs from non NetApp arrays.  They then format the luns with WAFL and pretend they're disks.  This is a great example of a "storage hypervisor" but not really SDS since it does not offer an abstracted control plane.

Got it?  Awesome!!

Top 5 Software Defined Storage Myths

Whoah.  Everyone has an opinion about Software Defined Storage (SDS) these days.  That's awesome!

Unfortunately, we have some really funny ideas running around about what SDS really is.  The only way to prevent this is to build up a standard definition and then compare alternates against this definition.  When they don’t hold up, you have to call out those willing to twist the term.  Here are some common SDS myths that we’ve seen running around the internet lately:

Myth 1:  SDS Isn’t Real

Our favorite is that claim that SDS is not a real thing and therefore I can call any product I want SDS.  This one is particularly awesome because it refutes itself instantly.  If there is no such thing as SDS, why on earth would you even use this term?  If it means nothing, then by definition your product or architecture is not SDS because SDS is nothing.  See?  Silly argument.  SDS is a real thing and we see customers asking for SDS features daily.  For vendors who claim that SDS isn’t real:  stay on the path you’re on.  More money for the rest of us.

Myth 2:  SDS is the Same as Storage Virtualization

This one is more interesting.   In this case, people are comparing two terms that are pretty loosely defined and claiming they are the same.  Tougher to refute because both terms a bit fuzzy so comparing them is twice as hard as just talking about one.  In looking deeper at this argument, we mostly see this being proposed by folks who already make Storage Virtualization technologies.  This is red light number one.  When someone takes a new term and applies it to something they’ve been doing for years, you should be suspicious.  Yes, it’s possible that they were way ahead.  Most likely, they’re just twisting the facts.  Second, it appears that most people who use the term “Storage Virtualization” apply it to the notion of providing a software aggregation layer on top of existing storage back ends.  While this is super interesting technology, it is certainly not new and it mixes data plane and control plane architecture.  One thing we know for sure is that SDS implies a segregated control plane.  This means that traditional Storage Virtualization products are not SDS.  They may embrace SDS style control planes but the notion of a virtualized data plane is not required for SDS so they’re not synonymous. 

Myth 3:  SDS means Software Delivered Storage

This one is really fun.  Basically, there are some vendors out there who have very cool software only storage products.  We won’t name them to protect the innocent, but ten seconds on Google and you can find them.  They are claiming that because they’re software only (no hardware as part of the bundle) that they’re SDS.  That’s pretty silly.  If you take a look at most of the major “legacy” storage players out there, what you will see is a software solution that’s bundled with hardware.  The vast majority of SAN/NAS products out there are software running on pretty standard Intel/AMD hardware.  They choose to bundle the hardware and software into a package to make it simpler do deploy/service and perhaps to gain some revenue on the hardware, but fundamentally all the value in the platform is software.  By breaking this delivery model and selling just software, does that mean I have an SDS solution?  Obviously not.  SDS is not a business model.  SDS is a Storage Management Architectural Model.  If software only vendors want to create an abstracted control plane, then that’s great and they can support SDS.  However, a traditional SAN can do the same thing if they choose to do so.  There is nothing inherent to SDS that requires you to have a software only business model.

Myth 4:  SDS means Object Storage

Ummm.  What?  We would not include this at all but there are some folks who seem to think this.  The notion that object stores are somehow related to SDS is pretty bizarre but it seems to be common.  Perhaps because many more modern storage systems are starting to be object based??  We don’t really know.  The reality is that object stores are not new.  Things like S3 and CDMI have been around for years and some of them have SDS attributes and some don’t.  Object stores are really cool, but they’re data plane constructs.  They are completely orthogonal to what SDS is trying to achieve.

Myth 5:  Traditional SANs Cannot Support SDS

Again, quite a silly notion.  As we have discussed, SDS is all about abstracting control surfaces from data plane and data services.  There is no inherent reason why you cannot do this on a traditional SAN or NAS platform.  What is true is that many of these platforms were designed before the concept of SDS was formalized but it’s not true that they cannot adapt to this new management paradigm.  It’s up to the SAN vendors to decide how/if they want to join the SDS world or not.  Since we are highly abstracted from the underlying implementation there is no reason why we cannot support traditional architectures along with newer software only or object store based systems.

Who's got more Myths?