Draw architecture diagrams like a boss!

I am always grateful when I see any architecture diagram once I join a new project. A picture is better than a thousand words. A well-designed diagram can quickly enlighten us about the architecture of the project. The more we see such diagrams in the company we work for, the better understanding w have of what is actually going on from a more high-level perspective.

Nevertheless, I do not see such diagrams as often as I would wish and I guess you have the same feeling (hopefully not!). There are a couple of reasons for that, I think the main ones are:

• it takes time to write a well-design and understandable architecture diagram
• once the architecture evolves, such a diagram has to be imported into dedicated software and updated
• the drawing software is often paid and many companies are reluctant to provide developers with them. Only “lucky” developers can get i.e. MS Visio. If you are a junior dev, forget.

Python diagrams for the rescue

Is it possible to address the above issues at once? It is! The solution is Diagrams – an open-source tool to generate beautiful diagrams from Python code. In comparison to the above points:

• generated diagrams are beautiful. The API is also pretty easy
• diagrams are written as a code so it is not only easy to update and regenerate them. They can be stored together with the code in a version control system so we can track how the architecture evolved over time
• it’s free!

The installation and quick start guide can be found on the GitHub or a Diagrams website, however, we only need to install the Diagrams and Graphviz. Python 3.6 is also required.

To install Diagrams:

# using pip (pip3)
$ pip install diagrams

# using pipenv
$ pipenv install diagrams

# using poetry
$ poetry add diagrams

For Graphviz in Linux:

# Ubuntu
sudo apt install graphviz

# or by yum:
sudo yum install graphviz

For Windows or Mac, check this site: Download | Graphviz.

Show me the code (… and diagrams)!

Let’s start with a simple example. For obvious reasons, I will post only examples from the official website, but they are really good! Diagrams support clouds (AWS, Azure, GCP, Alibaba, Oracle), Kubernetes, OpenStack as well as on-prem solutions.

Let’s start with the “hello-world-like” example. Create a diagram.py file with a few components:

from diagrams import Diagram
from diagrams.aws.compute import EC2
from diagrams.aws.database import RDS
from diagrams.aws.network import ELB

with Diagram("Web Service", show=False):
    ELB("lb") >> EC2("web") >> RDS("userdb")

… and generate the diagram:

$ python diagram.py

The result will look as follows. Pretty good, isn’t it?

Below there is an example of Clustered Web Services in AWS. Below code:

from diagrams import Cluster, Diagram
from diagrams.aws.compute import ECS
from diagrams.aws.database import ElastiCache, RDS
from diagrams.aws.network import ELB
from diagrams.aws.network import Route53

with Diagram("Clustered Web Services", show=False):
    dns = Route53("dns")
    lb = ELB("lb")

    with Cluster("Services"):
        svc_group = [ECS("web1"),
                     ECS("web2"),
                     ECS("web3")]

    with Cluster("DB Cluster"):
        db_primary = RDS("userdb")
        db_primary - [RDS("userdb ro")]

    memcached = ElastiCache("memcached")

    dns >> lb >> svc_group
    svc_group >> db_primary
    svc_group >> memcached

will generate this diagram:

The result is also good!

Let’s see a different provider. Below there is an example of a Message Collecting System on GCP:

from diagrams import Cluster, Diagram
from diagrams.gcp.analytics import BigQuery, Dataflow, PubSub
from diagrams.gcp.compute import AppEngine, Functions
from diagrams.gcp.database import BigTable
from diagrams.gcp.iot import IotCore
from diagrams.gcp.storage import GCS

with Diagram("Message Collecting", show=False):
    pubsub = PubSub("pubsub")

    with Cluster("Source of Data"):
        [IotCore("core1"),
         IotCore("core2"),
         IotCore("core3")] >> pubsub

    with Cluster("Targets"):
        with Cluster("Data Flow"):
            flow = Dataflow("data flow")

        with Cluster("Data Lake"):
            flow >> [BigQuery("bq"),
                     GCS("storage")]

        with Cluster("Event Driven"):
            with Cluster("Processing"):
                flow >> AppEngine("engine") >> BigTable("bigtable")

            with Cluster("Serverless"):
                flow >> Functions("func") >> AppEngine("appengine")

    pubsub >> flow

And the result:

Let’s see how what the library can do for us in terms of the on-prem architecture. Below is yet another beautiful example. Advanced Web Service with On-Premise:

from diagrams import Cluster, Diagram
from diagrams.onprem.analytics import Spark
from diagrams.onprem.compute import Server
from diagrams.onprem.database import PostgreSQL
from diagrams.onprem.inmemory import Redis
from diagrams.onprem.aggregator import Fluentd
from diagrams.onprem.monitoring import Grafana, Prometheus
from diagrams.onprem.network import Nginx
from diagrams.onprem.queue import Kafka

with Diagram("Advanced Web Service with On-Premise", show=False):
    ingress = Nginx("ingress")

    metrics = Prometheus("metric")
    metrics << Grafana("monitoring")

    with Cluster("Service Cluster"):
        grpcsvc = [
            Server("grpc1"),
            Server("grpc2"),
            Server("grpc3")]

    with Cluster("Sessions HA"):
        primary = Redis("session")
        primary - Redis("replica") << metrics
        grpcsvc >> primary

    with Cluster("Database HA"):
        primary = PostgreSQL("users")
        primary - PostgreSQL("replica") << metrics
        grpcsvc >> primary

    aggregator = Fluentd("logging")
    aggregator >> Kafka("stream") >> Spark("analytics")

    ingress >> grpcsvc >> aggregator

… and the result:

More examples like these can be found here: Examples · Diagrams (mingrammer.com).

Drawing diagrams like a boss?:)

Yes, I think so. I think even after looking at the above code examples we can understand how to more-or-less build diagrams that are maintainable yet beautiful in their simplicity. Just a few lines of code can generate a fantastic result. I think searching for dedicated components and placing them appropriately in a drawing tool would take me more time compared to learning the API of “Diagrams” and writing just a few lines of code. This can go even better with little practice.

If you have not used “Diagrams” before, I hope this post encouraged you to do so. Have fun!

Bonus – see the “Diagrams” in action

Some time ago, Paweł has made a great video explaining how to use “Diagrams” (in Polish). 6 mins of great content – enjoy!