What is a YAML? – A Box of DevOps?

I recently returned from a birthday trip to Napa Valley and got to spend some time with the Shipa Team in Palo Alto during the trip. Grabbing a coffee on my trek back to San Francisco, I overheard someone talking about YAML at the coffee shop and I had to hold back my laugh. You usually do not hear folks talking about YAML out in the public but this is San Francisco. For many engineers, YAML is a way of life. Especially if you are embracing a paradigm like GitOps, you will be flooded with them; the below picture from A Cloud Guru sums it up quite well. 

So how did we arrive here and what can we do about it? In one of our recent Shipa Office Hours, was asked “what is this YAML folks keep talking about”. If you start to witness you are writing more lines of YAML e.g YAML sprawl than your feature/general-purpose language, let’s take a look back at some history. 

XML and YAML Have “ML” In Their Names

A simple definition of YAML is that YAML is a human-readable data structure language. Similar to XML as a data structure, XML and JSON alike are designed to be machine-readable also. With JSON you can store large JSON objects in just one line. Today, YAML is a popular format for configuration files. The substantial growth of YAML is the rise in Kubernetes which leverages YAML is the primary language of manifests. 

For system-to-system communications e.g a restful service, sending XML or JSON across the wire is fine. Though from human to system communication, YAML is becoming the dominant format of many development and infrastructure tools, thanks to Kubernetes. Let us look at a few YAMLs. 

Here are some YAMLs

Here are a pair of YAMLs to see how the structure looks. As a human, they read pretty easily. Knowing some background where they came from, can assume they are trying to accomplish what they are designed to accomplish. Here is one from the recent ArgoCD 2.3 Release showing off Ignore Differences. 

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  ...
spec:
  ...
  ignoreDifferences:
  - group: "*"
    kind: "*"
    managedFieldsManagers:
    - rollouts-controller

Here is another YAML from Solo.io’s newly introduced Workspace. 

apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: default
  namespace: bar-ns
spec:
  exportTo:
    - selector:
        team: n-s-gateway
    - selector:
        team: foo
options:
  serviceIsolation:
    enabled: true
  federation:
    enabled: true
    hostSuffix: 'gminaction.io'
    serviceSelector:
      - labels:
          simple-federation: wanted
    ports:
      - number: 9080

Both are excellent capabilities and reading them seems easy enough to understand as a human especially if you have a background. With any piece of software that requires configuration, there is an inception problem. Imagine if you had to create all of these from scratch and you had limited experience in the tools. 

Configuration vs General-Purpose Programming 

Imagine you were given a task with a blank configuration.yaml. You have a blank slate which can be a positive or a negative depending on how you look at it. Items that require configuration require an understanding of the tool. Go back a decade or more to the JAVA/JEE days, a JAVA developer would write features in JAVA. 

These JAVA applications eventually would need to head to an Application Server like JBoss AS. JAVA is WORA e.g write-once-run-anywhere e.g “JAVA is JAVA”.  Though what is running your distributed JAVA web-facing application has nuances. Depending if there was a Middleware Engineer on a team, there would have to be deployment descriptors and Application Server specific configurations e.g a web.xml that would be pretty Application Server specific. There certainly was tooling around this, for example, if leveraging IBM WebSphere as an Application Server you could use an IBM development environment e.g Rational Software Architect that would generate a lot of the objects needed. 

But those objects were very opinionated and even had their own language, e.g in IBM WebSphere was TCL. Having to configure some things vs having to build something is different. The tools and resources available with a general-purpose language like JAVA or Go allow for you to create anything and are typically not bound by how a tool or system was implemented. Even though the number of lines-of-code [LoCs] is much lower in something like YAML, you still have to understand the abstraction. 

Take a look at the previous ArgoCD Ignore Difference example, how would you implement that as a developer? You would have a flag or conditional statement to skip over a block and regenerate the remaining config. That is most likely how the ArgoCD implementers designed that feature in Go. This is not an argument against abstraction, just the learning curve that abstractions sometimes require if you are presented with a blank slate. Though to combat the problem, templating is there to help.

Template Away     

The great thing about code is that you can generate code also. There is a host of templating engines and tools for YAML. For example, Ksonnett and Jsonnet can help you generate lots of YAMLs. Though again the inception problem, someone has to create those templates to be executed. Expertise needs to be disseminated and innovation vs control needs to be balanced with the templates. As the number of templates and files grows, the need for additional tools is warranted. 

Like any file-based approach, when there starts to be a large number of files, leveraging a package manager starts to make sense for distribution outside of source code management [SCM]. From the package manager side, you could create Helm Charts to package up all of these YAMLs. Even Helm has Go Templating to add another layer of templating. As the package and configuration management journey in Kubernetes continues to evolve, tools like Kustomize are gaining traction. If creating and maintaining this ecosystem seems daunting, there is another way with Shipa. 

Shipa, Another Way

At Shipa, we are trying to make Kubernetes and the dozens of potential YAMLs and associated templates an afterthought for developers. Shipa as a platform is an abstraction layer for your applications and a policy layer having the ability to enforce decisions made with the abstractions. Shipa’s approach is in plain English how to describe your application. Want some ingress/egress, no need to wire several Service Mesh CRDs with Shipa. 

By creating and enforcing abstractions with Shipa, you as a developer or a DevOps engineer can deploy to Kubernetes very quickly. With the art of the possible, you can even create a one-line Developer Experience [DX] with Shipa.

Cheers,

-Ravi