A prized principle of design is the hiding of unimportant details from the users of a product that enables them to work with it without getting overwhelmed by its possibly complex inner workings. This act of deciding that something is unimportant and hiding it behind a simpler facade is what abstraction is about. In this sense, abstraction, when it works, is considered A Good Thing™.

You could say that much, if not all, of product development in a modern economy relies on clever ways to abstract complex inner workings to provide a simple facade. Most car drivers know what happens when they push/pull various levers and turn various knobs, but are unlikely to know how a car actually works. As Joel Spolsky points out in his post on The law of leaky abstractions, the TCP protocol hides the details of the unreliability of IP - the underlying communication protocol - by building in mechanisms that compensate for it. When we order a pen from Amazon, we have little to no knowledge of how exactly it comes to us. When we search for something on the internet using Google, we’re unaware of all the activities that Google needs to (and in some cases arguably doesn’t need to) undertake in order to provide you with answers. We stay functional in part because we’re not required to know all the details of everything that we do.

We can now broaden the notion of “abstraction” as a principle to societal development at large. When we turn on a light switch, we do not need to know about which power plant the electricity is coming from. When we open a tap, we are not required to know where the water is coming from. These urban abstractions stand out especially when you ask a child that’s grown in an urban environment “what do you think happens to the crap when you flush the toilet?” If you’re lucky, you’ll get an imaginative answer, as I got from mine who said “it goes to where all the cow dung is”.

Abstraction is prized primarily because it enables us to do certain things that would otherwise be near impossible if we had to take care of every detail ourselves. As a society, we have been able to scale our activities to “higher” goals in part because our common needs are met by services that abstract details from us. When we send a parcel by courier, we can be sure it will reach the person we intend it for and move on to doing other things. A fundamental abstraction that makes these abstractions possible at the societal level is money.

These days, we’re seeing what happens when we scale our treasured abstractions to all of society. We discard our daily trash down a chute or in a tree corner trash bin, depending on where you live, and forget about what happens to it after. The fact that these all end up in landfills is now catching up to us. Centralizing so many people in cities who then need water to come out of their taps demands the damming of entire rivers for the supply. We’re now seeing the global consequences of irresponsible damming activities in droughts and large landscape changes. The fact that we don’t care how our cars burn fuel to take us where we want to go, when extended to our growing population, has resulted in increased CO2 emissions and is partially causing the climate change patterns were noticing today. Literally a few weeks ago, the state of Karnataka in southern India was ravaged by drought. Today it is flood due to monsoon.

Joel Spolsky famously stated that “all non-trivial abstractions, to some degree, are leaky”. By that, he meant that while an abstraction can be designed with best of intents to hide some unimportant detail, occasionally, that “unimportant” detail will rear its head and make its presence felt.

Abstractions at scale

One category of “non-trivial” abstractions is that done to exploit effects of scale, such as the society level architecture features highlighted earlier. Any leak in such a scaled abstraction tends to be amplified by the scale of application. Spolsky’s example of your pet snake chewing through your network cable is a local example of abstraction breakdown, but sharks biting an undersea fibre cable will make the abstraction leak possibly felt by millions. A flood can bring back toxic garbage that was intended to be forgotten as “unimportant” in landfills and waterbodies into the very streets and houses that produced them, as happened during the Chennai flood of 2015 (which was a manmade one) or the Kerala monsoon floods of 2018. Countries that took on trash processing towards potential use for recycling or for energy production can decide to stop accepting the trash. For these abstractions implemented at scale and which constitute important markers of “development”, a leak can freak out millions.

Given this perspective, the trend of “software eating the world” comes across as a disturbing one. The layers of software being provided as “apps” to general urban populace is adding to the isolation they’re already provided by the earlier generation of brick and mortar abstractions. When you order food from Swiggy using their app or call a cab on the Ola or Uber app, the interactions you have with your desired services almost ends with the app itself. Users don’t need to care about how exactly their asks are being fulfilled, and hence lose a little bit more touch with humanity. The people who provide these services are provided “ratings” based on which their ability to provide a good service is judged. They aren’t thought of as equal participants in society because they are not thought about, because they’re seldom spoken to. When booking a cab ride, we no longer need to interact with the cab driver because the driver already knows where to go and how to go there. When we order food on Swiggy, we aren’t required to know how much carbon footprint the transport incurred or how much waste the packaging will result in or over what period it will degrade. The abstraction that facilitates this degree of separation? It’s good ol' money. I’m afraid e-governance will face the same fate - citizens knowing progressively less about the mode of operation of government and the reasons behind its activities, because they latch on to the digital presentation as a statement about the reality.

The continued use of abstractions at scale also tends to postpone large scale effects to a time beyond which they may not be reversible any more, as we’re seeing with global climate change phenomena. What is worse is that the lay consumer of these abstractions does not have the necessary knowledge to reason about higher order effects of the system’s behavior. It is these very lay persons (you and me included) who get to vote in large numbers in democratic countries, which makes the issue of communicating the importance of awareness of how the abstractions they’re so used to will eventually leak in disastrous ways, a significant public relations and public educational undertaking. Timely action to set appropriate countermeasures in place becomes an uphill battle for the few who’re acutely aware of these consequences.

One approach to dealing with these at-scale societal abstractions is to deliberately “leak” them early as part of system design. For example, knowing how we dispose of our garbage could be a field trip exercise in schools with parents accompanying their children. If you’re going “yikes!” and thinking about potential health hazards, that’s sort of the point. Designing bottlenecks into the system can help too. For example, if there’s a rough “per person per day” trash figure that is manageable for the society mechanisms, then capping the trash disposal mechanism to, say, 50% of that will result in increasing the visibility of the garbage generation process. Even simple mechanisms like handling all the trash work during normal traffic hours instead of during the lull hours of the night can help leak the abstraction a bit and increase awareness. Schools teaching about the local geography, the local water bodies and sources, local wild life and how they participate in the environment, could all contribute to building higher order thinking individuals. It is of little consequence to the world if a kid today knows the capitals of 500 states, but of tremendous consequence if the kid knows that a particular area used to be a lake and we’re building residential apartments in the dried up lake. I’m being hopeful here, but that might just provoke them to ask what will happen when heavy rains come down to fill up the lakes. The deliberate leakage tactic is also perhaps applicable in the software engineering world - where we teach abstractions alongside the gritty details that can help build a mental model to help predict how these abstractions can fail.

Back at the societal level, it therefore seems important that those designing implementations of these scaled abstractions take on higher order concerns, somewhat in proportion to the scale itself. A government deciding to build a dam over a river to make water supply to a certain area controllable, must necessarily also consider environmental impact, run long term studies at least as simulations, undertake educational efforts to “leak” the abstraction back to the populace, document factors that will likely change irrespective of whether they are, currently, seen to have a bearing on the decision, and so on.

If there is one thing that we’ve learnt about bio systems, it is that they’re too complex for us to stop with a first order model. Dietrich Dörner’s book “The Logic of Failure” documents cognitive patterns which when deployed in administration result in societal destruction. Pesticides which have endocrine disruptors, for example, have been linked to early occurrence of puberty in girls. When a solution is introduced, the societal level process does need to think about possible unintended consequences and sponsor studies almost as part of that process. While it can be argued that this increases costs for everyone and delays adoption of practices, it is counter-arguable that that is kind of the point.

Furthermore, the cost argument discounts the possibility of creative solutions that account for past knowledge in sustainable societal architecture in the new situation. An example is the recent banning of plastic bags and take away boxes in Chennai, which has resulted in the resurgence of traditional biodegradable natural packaging means and the production of newer vegetable extract based bags. The existence of a body such as the FDA is more easily justified since the immediate impact of a proposed ingestible is communicable to The Lay Person™. Such regulations for the areas that produce other abstractions that we enjoy, such as disposable plastic shopping bags, are harder to justify though impactful well beyond just us. Marine life is getting wiped out due to plastics in oceans, for instance. It does come back to us as well, as microplastics are apparently everywhere in our water. It gets even harder as the time gap between the introduction and consequence gets longer. This is akin to how the engineering disciplines only rely on abstractions developed by more seasoned and experienced engineers. It takes a lot longer and more resources to develop these longer standing abstractions in the engineering disciplines than to use them. The development of these abstractions are accompanied by experience gained in applying the tech prior to the abstractions gaining stature as reliable fundamental tools.

Abstraction in the sciences

From a daily life perspective, abstractions tend to be about detail hiding, as discussed. In the sciences, including mathematics, the term abstraction is used in a related sense, but with some important distinctions. Abstractions in mathematics, for example, are still about “what I choose not to pay attention to”, but the goal of choosing an abstract view is to understand the principles of the structure. The result of that understanding is that if you do find the same structure in another context, we know, for sure, that the same understanding can be brought to bear in the new context. In this sense, abstractions in the sciences are highly desirable due to their reusability. Scientists and mathematicians who construct and work with these abstractions do so for these reusability benefits, not because they wish to save themselves the trouble of working out details. Scientists do understand that when you need to apply an abstraction in a concrete situation, you have to roll up your sleeves and get down and dirty with the details. The great thing is that we now know how the details will behave.

A good example of an abstraction in mathematics is the idea of a “vector space”, which is itself built up from the abstractions of “rings” and “groups”. Once you understand an aspect of a vector space such as the existence of basis vectors that span a space, you know for sure that any particular vector space you work with in a context will necessarily also have basis vectors and you’ll know how to find them. It doesn’t matter whether you’re working with vectors in conventional 3D space, or state vectors of quantum mechanical systems, or working with functions in transform domains. Similarly, understanding the structure of finite groups and the fantastic beasts that you can find in that area enables cryptographers to switch from prime number based groups to those based on elliptic curves with ease. In this sense, abstractions serve as a piece of technology for the mind.

Abstraction in engineering

Engineering is applied science. So it isn’t surprising that abstractions play a great role in engineering as well, given their role in science. Here, however, we start to see the “I don’t need to know” aspect begin to dominate with undesirable real world consequences.

Computing systems, for example, are built on several layers of abstraction. The lowest layer deals with the hardware directly, providing a uniform “interface” to the layer above, which in turn builds new functionality which it exposes to the layer above and so on - until you reach the level of the common graphical interfaces you normally interact with. Each layer provides some building material that the layer above can use to organize its functionality.

The layering common in computing systems is not without its cost though. For example, in recent times “microservices” are run in “containers” have many layers of nesting that are, in principle, pass through, until the actual hardware is touched. A program, for example, runs in a virtual machine like the Java Virtual Machine, which itself runs within a cordoned off area called a “container”, inside a “process” managed by an “operating system” that runs with a “virtual machine” which runs on a “hypervisor” that emulates hardware resources for each VM instance. For this reason, from time to time, engineers take the effort to throw away some of these layers as we see with the research on unikernels.

In more conventional engineering, such as building construction, the layers of abstraction don’t quite hide details, except perhaps in the design and architecture phase. The level at which they do hide details is in the assumptions that they start with - for example, that water supply for a building can be taken for granted as long as we get approval from the appropriate government body. Usually for cost reasons, many such projects therefore do not peel back on these societal scaled abstractions. The questions of how a building may impact local ground water or the ecology of the area due to the cell tower on top of the building don’t usually get any ears. In the worse cases, the engineers are forced to increase the habitable area thereby not even making provisions to harvest rain water and return it into the ground. At the engineering level, therefore, we’re back to square one.

Abstraction in medicine

Evolution - the great dance between variation and natural selection - is a ruthless optimizer. The organisms that evolutionary processes have produced, including us humans, are very resilient yet very complex structures. They consist of many parts that cooperate to maintain each organism, but the “break it down into parts” picture does not work well to understand the body in the long run. This is because the idea of a reducible “part” is an abstraction invented purely to reduce our mental load in thinking and reasoning about the system. The fallacy at play here is that because it is easier for us to reason about a system if we thought of it as consisting of parts that interact exclusively via a few mechanisms, it doesn’t mean that the system actually works that way. Modern medicine is full of blunders that have resulted from this fallacy.

A “part” as an abstraction is appealing to us because we can pretend to study this part in isolation, ignoring other parts, understand it and bring that understanding to bear on the whole organism that has this part. This works to some extent, but fails in the largest sense. Some decades ago, nutrition researchers analyzed mother’s milk and didn’t find anything that is not there is the manufactured baby food of the day. The idea that nutrition can be broken down into ingredients is partly at the root of that reasoning. The other part is the fallacy that “We haven’t found any differences, so there isn’t any.” If we’re truly data driven, our first instinct here should be to trust dear old ruthless evolution that’s stood the test of hundreds of thousands of years, and not the few days level studies conducted by scientists. We know today that the first milk that mothers give their babies interact in a complex way with their systems. Babies cannot have very large heads because they need to come out of their mother’s vagina. So the mother’s milk contains a specific kind of food for bacteria that has lodged in the baby’s gut that results in that particular kind of bacteria growing to dominate the gut. This bacteria, when it processes the specific sugar in the first milk of mothers, produces a by-product that triggers brain growth in the baby.(Refs 1, 2) Who knew?

If such a reductionist view doesn’t work, what about the “holistic medicines”? I’m not convinced that the holistic medicines have got it either, because their models of human or animal biology are far less precise than what we know as modern medicine. However, that very lack of precision perhaps gives enough space for our bodies, which have coping mechanisms built in, to leverage a crutch or two, heal the legs and start walking. This isn’t an attestation of any particular medicine system, but excluding long standing works from study just because the models and language don’t fit the abstractions we’ve come to accept doesn’t seem wise ‘cos there may just be something useful there within grasp that we might miss otherwise. If we don’t take that attitude, an abstraction pretty much turns into a religion.

Closing thoughts

Building abstractions is valuable to society, but the sweetness of abstractions needs to be tempered with the consequences that they bring and the various ways in which they can leak. So abstractions per se aren’t the root of all evil, but abstractions implemented at scale without the society being exposed to the underlying layers is evil and is gradually killing our planet. Let’s teach the art of abstraction, but let’s also teach the art of implementing abstractions at scale. Our survival on this planet will not only depend on what tech we can hide behind, but also how long our understanding reaches into the depths of how our planet supports us in having that tech in the first place.

(last edited: 2020-04-07)