I am always grateful when I see any architecture diagram once I join a new project. A picture is better than a thousand words. A well-designed diagram can quickly enlighten us about the architecture of the project. The more we see such diagrams in the company we work for, the better understanding w have of what is actually going on from a more high-level perspective.
Nevertheless, I do not see such diagrams as often as I would wish and I guess you have the same feeling (hopefully not!). There are a couple of reasons for that, I think the main ones are:
- it takes time to write a well-design and understandable architecture diagram
- once the architecture evolves, such a diagram has to be imported into dedicated software and updated
- the drawing software is often paid and many companies are reluctant to provide developers with them. Only “lucky” developers can get i.e. MS Visio. If you are a junior dev, forget.
Python diagrams for the rescue
Is it possible to address the above issues at once? It is! The solution is Diagrams – an open-source tool to generate beautiful diagrams from Python code. In comparison to the above points:
- generated diagrams are beautiful. The API is also pretty easy
- diagrams are written as a code so it is not only easy to update and regenerate them. They can be stored together with the code in a version control system so we can track how the architecture evolved over time
- it’s free!
To install Diagrams:
# using pip (pip3) $ pip install diagrams # using pipenv $ pipenv install diagrams # using poetry $ poetry add diagrams
For Graphviz in Linux:
# Ubuntu sudo apt install graphviz # or by yum: sudo yum install graphviz
For Windows or Mac, check this site: Download | Graphviz.
Show me the code (… and diagrams)!
Let’s start with a simple example. For obvious reasons, I will post only examples from the official website, but they are really good! Diagrams support clouds (AWS, Azure, GCP, Alibaba, Oracle), Kubernetes, OpenStack as well as on-prem solutions.
Let’s start with the “hello-world-like” example. Create a
diagram.py file with a few components:
from diagrams import Diagram from diagrams.aws.compute import EC2 from diagrams.aws.database import RDS from diagrams.aws.network import ELB with Diagram("Web Service", show=False): ELB("lb") >> EC2("web") >> RDS("userdb")
… and generate the diagram:
$ python diagram.py
The result will look as follows. Pretty good, isn’t it?
Below there is an example of Clustered Web Services in AWS. Below code:
from diagrams import Cluster, Diagram from diagrams.aws.compute import ECS from diagrams.aws.database import ElastiCache, RDS from diagrams.aws.network import ELB from diagrams.aws.network import Route53 with Diagram("Clustered Web Services", show=False): dns = Route53("dns") lb = ELB("lb") with Cluster("Services"): svc_group = [ECS("web1"), ECS("web2"), ECS("web3")] with Cluster("DB Cluster"): db_primary = RDS("userdb") db_primary - [RDS("userdb ro")] memcached = ElastiCache("memcached") dns >> lb >> svc_group svc_group >> db_primary svc_group >> memcached
will generate this diagram:
The result is also good!
Let’s see a different provider. Below there is an example of a Message Collecting System on GCP:
from diagrams import Cluster, Diagram from diagrams.gcp.analytics import BigQuery, Dataflow, PubSub from diagrams.gcp.compute import AppEngine, Functions from diagrams.gcp.database import BigTable from diagrams.gcp.iot import IotCore from diagrams.gcp.storage import GCS with Diagram("Message Collecting", show=False): pubsub = PubSub("pubsub") with Cluster("Source of Data"): [IotCore("core1"), IotCore("core2"), IotCore("core3")] >> pubsub with Cluster("Targets"): with Cluster("Data Flow"): flow = Dataflow("data flow") with Cluster("Data Lake"): flow >> [BigQuery("bq"), GCS("storage")] with Cluster("Event Driven"): with Cluster("Processing"): flow >> AppEngine("engine") >> BigTable("bigtable") with Cluster("Serverless"): flow >> Functions("func") >> AppEngine("appengine") pubsub >> flow
And the result:
Let’s see how what the library can do for us in terms of the on-prem architecture. Below is yet another beautiful example. Advanced Web Service with On-Premise:
from diagrams import Cluster, Diagram from diagrams.onprem.analytics import Spark from diagrams.onprem.compute import Server from diagrams.onprem.database import PostgreSQL from diagrams.onprem.inmemory import Redis from diagrams.onprem.aggregator import Fluentd from diagrams.onprem.monitoring import Grafana, Prometheus from diagrams.onprem.network import Nginx from diagrams.onprem.queue import Kafka with Diagram("Advanced Web Service with On-Premise", show=False): ingress = Nginx("ingress") metrics = Prometheus("metric") metrics << Grafana("monitoring") with Cluster("Service Cluster"): grpcsvc = [ Server("grpc1"), Server("grpc2"), Server("grpc3")] with Cluster("Sessions HA"): primary = Redis("session") primary - Redis("replica") << metrics grpcsvc >> primary with Cluster("Database HA"): primary = PostgreSQL("users") primary - PostgreSQL("replica") << metrics grpcsvc >> primary aggregator = Fluentd("logging") aggregator >> Kafka("stream") >> Spark("analytics") ingress >> grpcsvc >> aggregator
… and the result:
More examples like these can be found here: Examples · Diagrams (mingrammer.com).
Drawing diagrams like a boss?:)
Yes, I think so. I think even after looking at the above code examples we can understand how to more-or-less build diagrams that are maintainable yet beautiful in their simplicity. Just a few lines of code can generate a fantastic result. I think searching for dedicated components and placing them appropriately in a drawing tool would take me more time compared to learning the API of “Diagrams” and writing just a few lines of code. This can go even better with little practice.
If you have not used “Diagrams” before, I hope this post encouraged you to do so. Have fun!
Bonus – see the “Diagrams” in action
Some time ago, Paweł has made a great video explaining how to use “Diagrams” (in Polish). 6 mins of great content – enjoy!