What is Docker or containerization?

The Beginning

Imagine you are an average guy who is interested in coding and building something great that solves someone’s problems.

Imagine you are not really writing an interactive desktop program, but some kind of network related program — it is basically listening for input at some port and sending its output to some other port. Such programs are officially called service — because they service clients that have input. If this program has straightforward ideas, you may call it a microservice.

So, you have written your microservice but because it is listening to the network, there is no limit on what kind of input it may receive. Maybe some of them may cause your program to crash? Quite possible. Or maybe there is a power outage. Or maybe the hard disk crashed. Or maybe even there was a memory leak in some other program that exhausted all the available memory and there was none left for your service?

There is no way you can guarantee that this program will run 100% of the time, all the time. And see, you don’t really care either — what you really want is your clients to be serviced. You want to keep solving their problems. So what you do is you:

1. Add some more ram, faster bigger hard disk etc. This is called vertical scaling, mostly because the pile of cash to buy this stuff rapidly grows vertically.
2. Run this same microservice parallelly on another computer, and have another program that always redirects the input to either of the running microservices, i.e. load balancer. This is called horizontal scaling.

From the onset, horizontal scaling looks like not solving the root cause — what if both microservices crash? Run three? What if all three fail? This way we will never reach 100% availability.

In comparison, vertical scaling looks simple and direct. Your service ran out of ram? Buy more ram! Does it require an insane amount of ram? Fix the memory leak. You think it is already fixed? Buy more ram! Okay, we should probably do some calculation on our memory requirements before buying more ram, this stuff costs money you know.

But even then, we won’t achieve 100% availability if your program crashes for some other reason. For example, a DDOS attack. You see, vertical scaling is actually more of a whack-a-mole solution that has no end. At some point in time, you will run out of money and be forced to do some cost analysis.

So, people did that and eventually found out that it is cheaper, in the long run, to scale horizontally, because as you add more machines that are just good enough, the availability reaches close to 100% while the expenses remain constant.

Making sure it is easy

So now, you know you want to do horizontal scaling. But it takes a lot of time to make sure a new machine is exactly like the already running and well-tested one. It is possible the admin made some mistake in a configuration that doesn’t get exposed until some critical situation…

And it all becomes costly again. Unless — virtualization. This way you can create instances of a running os and deploy them as many times on the same machine and on as many machines as you want without worrying about machine changes.

Okay, but you have so far one (micro)-service. You may have more. Maybe some of them were written by someone else. Possibly in a different language, even maybe for a different operating system altogether. Each will have its own way to start and run. Each will have its own way to configure, or some may even require conflicting dependencies. All this will take a lot of time for someone to do, and that will also increase the cost.

So, just like before, when we see a problem we go back to our original driving force — we want to make sure our clients are always serviced. And as it turns out, clients don’t know or care about operating systems or dependencies… they only want a highly available service. So instead of virtualization of the operating system, what if we virtualized only the service instance we are starting? This is called containerization.

Now, instead of having an image of an operating system with our service installed on it, which is duplicated exactly as it is when cloned, we will also write down, in a file, how to run a service and what configuration changes are required and clone that too? Then we will have some program that can read this file and do the necessary changes. This is literally scripting, now isn’t it? Like, a Bash script, or a batch script, or Zsh script, or a Powershell script, or a Python script, or Perl script…… ugh. To really scale, we need standardization here.

What’s more, not all scripting languages are equally capable. For example, they don’t really help you maintain multiple dependencies. But, okay, containerization will fix that. But what if you want to run on a particular port that becomes only accessible when a particular device driver is installed? It seems we actually require a script that installs OS and programs on that OS, and not just runs on an OS. That way we will standardize the script too, as it will be OS agnostic.

One of the popular programs to run such a script is Docker.

The finale

So, basically, Docker is a proprietary program that can process a rather simple and high-level script, or what Docker calls configuration file in YAML format, and do stuff like install a virtualized os from an image (a process called installing guest os on a host os) and run specific commands on that server. And it does other things like redirecting the open port of a guest os to some other port on host os and stuff like that. On top of it, Docker provides cloud storage where it keeps images that you can download and upload, which it calls a registry.

As it turns out, downloading from docker, creating containers and making sure they are running correctly can also be automated to a large extent by another thing called Kubernetes. But that’s for another day…