GitOps workflow using ArgoCD, Crossplane, and Shipa Cloud
On August 11th, 2021, we hosted a workshop to how you can implement a GitOps workflow using ArgoCD, Crossplane, and Shipa.
We were overwhelmed by the hundreds of people who registered for the session. It shows a few important challenges still have to be solved before GitOps will become mainstream across the enterprise.
While GitOps brings many benefits, we often see teams facing challenges around:
- Lack of visibility: Answering a simple question such as how often certain applications were deployed can become complex at scale
- Auditing isn’t as great: If you have a large number of applications and an enterprise-level audit requirement that you are responsible for, walking through a ridiculous amount of Git history and full-text search is not what your auditors, business, and the boss is looking for
- Controls: Implementing controls on what’s being deployed is not embedded as part of the approach
- Post-deployment: While people get really fixated on the deployment piece, you have to remember that you need to give your developers a connected workflow where they can manage their own applications, reducing the burden on the DevOps or platform engineering team
- Kubernetes only: Although you may want to move every app and service to Kubernetes unless you are a fairly new company, you already have a ton of existing infrastructure that your business wants you to leverage, and GitOps cannot attend that.
What We Learned
The workshop covered:
- How to install and configure ArgoCD, Crossplane, and Shipa Cloud.
- How to define a GitOps workflow for application controls and governance policies.
- How to improve the GitOps developer experience as part of the workflow with an integrated developer platform like Shipa.
During the session, we conducted a few polls where it became clear that about ⅔ of the attendants are still learning GitOps and that folks:
- Struggle with the developer experience for GitOps
- Are concerned about how to build security into the workflow.
While these are common challenges, it does not mean that GitOps is not a good solution.
When you detach GitOps from an infrastructure-specific tool, such as Kubernetes, and instead integrate it with an Application Operating Model, you can quickly solve the issues above.
Session Q&A
We had an overwhelming number of questions asked during the session.
For your reference, here are a few of the ones we addressed:
- Does Shipa use any OPA as part of the framework?
- No. Shipe uses its own engine to define and enforce policies across Kubernetes and VMs. The goal is to enable DevOps teams to define policies without learning, writing, and managing complex policies across different clusters. Shipa’s policy engine is also focused on broader infrastructure rather than Kubernetes only.
- Can you get the performance number for throughput on the network or CPU numbers?
- Yes. Once deployed, Shipa will display both resource utilization reports and application-related metrics such as latency, requests per second, and others. These can all be integrated with external monitoring tools.
- How does the shipa application quota relate to limits set in the pod-level quota?
- Shipa frameworks can enforce limits across how much resources applications deployed through the framework can consume and how much deployments can scale up to.
- How does Shipa integrate into ArgoCD?
- Shipa creates a set of CRDs in the cluster used to create and manage application-related objects, detect configuration drift, and manage the desired state of clusters, frameworks, and applications.
- What type of deployment method can we achieve with Shipa?
- Shipa supports regular deployments as well as more advanced techniques such as Canary rollouts. You can also customize deployments to address specific requirements such as ports, registries, and more.
- Does Shipa support Lambda and ECS?
- Shipa currently supports Kubernetes and Linux nodes (in its self-hosted version). Linux nodes are being released on Shipa Cloud soon. Moving forward, Shipa will work to make its application model work across additional platforms, such as serverless.
- Does Shipa support hybrid deployments?
- Yes. Shipa’s application operating model can be used to deploy apps and enforce policies across multiple Kubernetes clusters, across different versions and providers, and across virtual machines (on-premises or cloud)
- Can Shipa’s application model be used with FluxCD or just ArgoCD?
- Shipa’s application model can be used with both ArgoCD and FluxCD
- Can Shipa use my existing Istio ingress to create the application’s endpoints and network policies?
- Yes. Shipa supports Istio, and you can connect a Shipa framework to your existing Istio ingress.
- Can I use Shipa to replicate my desired state across users and permissions, or just frameworks and applications?
- ArgoCD can connect with Shipa to recreate your RBAC model, user, and team structure. This structure can then be applied across Kubernetes and Linux VMs.
What’s Next?
We will be scheduling these workshops on a regular basis and would love to get your ideas for the next workshops.
Here are some ideas that we are planning on. Which ones do you like best?
- Improving the developer experience for GitOps
- IaaS and AppOps with Terraform and Shipa
- Building security into your GitOps workflow
- Deploying applications on hybrid clusters
- Deploying applications across different architectures (for instance K8S & VMs)
