Layers of Abstraction

As I mentioned in a previous blog post, my team and I have been working on GenAI lately.  You may have seen Outshift’s announcement about the AI Assistant in Panoptica:

https://docs.panoptica.app/docs/cisco-ai-assistant-for-panoptica

Building the AI Assistant for Panoptica was an interesting learning experience.  It helped us understand what the limits of current LLM-based agents are and allowed us to start work on what the next generation will be.

This has also caused us to start thinking about how applications will change over time due to LLMs.  There is a very interesting debate raging at the moment that ranges from “nobody will write code in five years” to “everyone will write code in five years” and pretty much every other variation.  It’s exciting because we are right on the cusp of a major paradigm shift and nobody really knows where this is going.  I’ve been around long enough that I’ve seen these paradigm shifts happen a few times in my career so I do have some expectations about how this will go.

The first thing that I think we should remember is that this trend towards higher and higher layers of abstraction is not new at all.  If we look back into the history of computer science, we see a steady progression into higher and higher levels of abstraction.

For extremely old computers like the Altair above, you had to manually enter the boot code via switches before it would start (yes, they had ROMs but you had to program them yourself).  Then boot loaders were introduced.  Eventually, shops would write their own operating systems.  Older mainframe shops didn’t buy their OS, they built it custom.  Later, operating systems became a standard commercial product.  Then came the GUI revolution.  Later, we moved to distributed computing and the Web.  Then came virtualization.  Containerization, microservices, PaaS and cloud followed.

At each stage of this evolution, higher order services were offered.  When I was in college, I learned to program and I had to talk to storage devices directly.  For example, you could write to disk.. Today, with something called S3, I can create an object, persist information to that object and then refer back to that object whenever I want from wherever I want.  Yes, there are still disks sitting somewhere in a data center, but I don’t really need to know how S3 works and I don’t really care.

Does that mean that I don’t write software any more?  Well, as a Product Manager, I don’t write software at all, but you know what I mean.  We still write software.  We just don’t write low-level stuff any more and we don’t use low-level constructs.  Of course, this evolution wasn’t even or orderly.  Does anybody remember 4GL?  Fourth Generation Languages were supposed to take over and make things like C irrelevant.  That didn’t happen.  I used one called R:Base for a while but it was just too limited to write serious applications.

Just like the rest of the stack, programming languages have evolved but the low-level stuff is mostly still there.  In school, I learned machine language, assembler, Ada and C.  Other than Ada, all those programming languages are still around even though most software teams would never touch assembler any more.  However, if you work on an operating system project like ESXi from VMware, you are still writing pretty low-level code.  You’re probably writing in C, but you’re still down at the hardware.  So, the lower levels don’t go away.

What happens is really more about leverage.  Instead of EVERYONE writing an operating system, we buy that operating system from someone who only does that.  Then we use it without worrying about the down-in-the-weeds details.  It is similar with cloud.  Yes, I probably COULD write my own object store or set up a Ceph server, but I choose not to most of the time because renting S3 buckets from Amazon is super convenient.  There is still a team in Amazon deploying servers, disks and all that stuff but relatively few people in the industry are dealing with that detail because most of us just rent it by the hour from AWS, Azure, GCP or another cloud provider.

What does this say about GenAI and the future of programming?

Well, it’s pretty clear that writing source code is going to get cheaper.  We know that LLM’s can already do this and are getting better every day.  Tools like CoPilot mean that a developer can produce way more code more quickly, and thus the code costs less to produce in terms of human hours invested.

Andy Jassy recently claimed that AWS saved 4,500 developer years of work due to GenAI based programming tools.  That is an insane number.  Even if the number is only half that, it means that their cost per line of code is WAY WAY less than it would have been if that work had been done manually.  You can read his quote here:

https://x.com/ajassy/status/1826608791741493281?lang=en

However, it’s extremely unlikely that traditional algorithmic programming is going anywhere anytime soon.

What is much more likely is “hybrid” systems where traditional code sits alongside GenAI-based agents and they work in concert to solve problems.  This means that even if you are not writing your code by hand, you are still writing and maintaining software.  All the labor and effort that goes into running a SaaS-based application is still there, just hidden behind a new layer of abstraction.

Thus, GenAI is likely to change HOW we do things but probably won’t change WHAT we are doing that much.  We will still have security issues, we will still need observability, we will still need to diagnose and fix bugs just like we do today.  The tool chain will change and we will probably be more efficient, but we’re still building and running software.  

So, the question for all of us is to figure out how to build and optimize this hybrid world we will likely wind up in.  How do we automate away the toil?  How do we move faster?  How do we operate more efficiently?  Not surprisingly, these are the same questions I was asking ten years ago when I first started working on cloud and SaaS-based products.