Crossplane, Infrastructure as Code
In this article, Miguel Ángel Chuecos, Technical Manager of Cloud and Architecture area of Dare Planet Technology, gives an overview of Crossplane, what it is, how to configure it and the main alternatives.
This article refers to a session we did on LinkedIn Live, we leave here the video in case you want to watch it and read this article at the same time (Only the Spanish version of the video is available).
Table of Contents
What is IaC and Crossplane?
In recent months, in the Cloud World and GitOps, there has been a strong echo of a tool that has come with strength to compete in the world of IaC (Infrastructure as code) being compared with some established tools and greater journey as Terraform.
But, what is IaC (Infrastructure as code)?
As its name suggests, the main objective of this type of tools is to manage and provision infrastructure from one or several providers (GCloud, Kubernetes, Azure, AWS, Oracle…) through code instead of a manual process through interface, CLI, API, etc.
In this way, we manage to provision infrastructures that are composed of multiple pieces/services (Compute, Networking, Storage…) in a matter of seconds being possible to multiply and “clone” this environment without having to perform repetitive actions and, at the same time, offering new environments to the development team with greater agility.
So, what is Crossplane?
It is an Open-Source project that provides a complement to Kubernetes, allowing it to act as a control plane to provide Cloud infrastructure. In addition, Crossplane offers and exposes high-level self-service APIs for teams to consume without having to write any code.
DevOps and GitOps
Crossplane is seen as a piece of the GitOps framework, but let’s first look at the definition of DevOps, GitOps and how they differ.
DevOps is a culture that focuses on giving development and operations teams the ability to work together collaboratively.
GitOps, on the other hand, is the tooling framework for DevOps practices to be applied using mainly Git as the version control system.
This framework is designed to meet the objectives of collaboration, CI/CD, infrastructure automation and application deployment.
The following example shows a GitOps model using some tools such as ECR (AWS Docker Repository), ArgoCD, Crossplane and Git as a building block.
Using the CI/CD tool Argo, you can deploy a component on multiple Kubernetes clusters regardless of where they are located (as long as there is accessibility); multi-cloud, multi-region, multi-account….
In this way, we avoid having to deploy the same component for each cluster manually and repetitively or through a custom script that requires maintenance.
The diagram shows the example of an application/component to be deployed in Kubernetes (packaged with Helm’s chart) that needs to be deployed in multiple clusters and also requires the creation of some cloud resources/services since the definition of these resources can be specified in YAML manifests with the Kubernetes format.
Cloud Native Compute Foundation and Crossplane
Crossplane is part of the Cloud Native Compute Foundation (CNCF) as an open-source tool.
The CNCF is a foundation that promotes the adoption of cloud-native whose adoption and objectives include:
Scalable technologies/applications in modern dynamic environments (public, private or hybrid cloud).
Containers, service grids, microservices, immutable infrastructure, declarative APIs…
Techniques to create resilient, manageable and observable systems.
All technologies under the CNCF are projects that are mostly hosted in Github repositories. We leave here the landscape with all the tools hosted under the CNCF, with several stages for each project:
- Sandbox stage: Initial entry point for projects.
- Incubation stage: Where a project must meet all sandbox requirements, as well as additional benchmarks, such as documenting at least three end users successfully deploying the project into production.
- Graduation stage: In which a project must meet all the requirements of the incubation stage, plus additional criteria, such as, for example, completion of an independent security audit.
Crossplane installation and setup
In order for the Kubernetes cluster to be able to use Crossplane, it is necessary to install and configure it together with all the resources (Composite, Resource, Definition, Composition…) for its performance.
In its official documentation we can find the steps to follow to do so.
Regardless of the cloud provider, it is required to configure a ProviderConfig resource that basically tells Crossplane how to connect to our cloud.
In the AWS case, if you don’t have an EKS cluster, you can create it manually from the management console or via eksctl with the following command:
eksctl create cluster
--name learn-crossplane \
--nodes 3 \
--with-oidc \
--ssh-access \
--ssh-public-key learn-crossplane \
--managed
To set the context with the created Kubernetes cluster:
# Obtener el listado de contextos
kubectl config get-context
# Establecer el contexto con el nombre del cluster EKS mostrado en el listado
kubectl config use-context demo@demo.com@learn-crossplane.eu-west-2.eksctl.ios
Remember to delete the cluster after all tests have been completed:
eksctl delete cluster --name learn-crossplane
Crossplane Sample Resource
Crossplane has a guide to create a sample resource (PostgreSQL database) together with the creation of a pod as a sample application to show the connection to the database through the secret created by the resource with all the necessary data (endpoint, port, user, password).
You can see more about the behavior and steps of the guide in the Youtube video, the link will take you to the exact moment when we discuss the topic.
How do I know which parameters to use in the definition of a resource and what type of resources can I create depending on the cloud provider?
In the API section of Crossplane’s official documentation, it is possible to find a list of resources per provider and a specification of all the parameters to be used to customize these resources.
Crossplane vs Terraform comparison
If we compare Crossplane with Terraform we will see that both have things in common but also with their differences obtaining a series of advantages and disadvantages in their multiple categories. These categories are classified in:
- Forecasting: Using the terraform plan command we can get a dry-run preview of what will be applied as infrastructure in the target environment, however, with crossplane, there is no high-level preview of what will be created, modified or deleted.
- Modifications: Whenever you need to apply a change in infrastructure, with Terraform it must be done ‘manually’ by making the appropriate changes in code and executing its commands (Unless it is integrated with some CI/CD and Git system that automates the launching of those commands when it detects new changes). Crossplane applies the changes automatically, that is, when you modify the manifest either by redeploying the YAML or “on the fly”.nte” directamente desde el recurso creado, Crossplane aplica esos cambios detectados. Por otro lado, cuando realizas un cambio manualmente, por ejemplo, modificar el tipo de instancia de una RDS, Crossplane detectará que el manifiesto que tiene no es igual a la infraestructura existente y la modificará con los valores que él dispone.
- CI/CD: Both tools are easy to integrate into CI/CD practices.
- Requirements: Terraform only requires having a system where to store the tfstate (Bucket, Terraform Cloud…) to be shared with the rest of the team. On the other hand, Crossplane requires a Kubernetes cluster since it only works on this technology and this may imply a high cost if it is created exclusively for this purpose.
- Language: Terraform uses its own HCL language (HashiCorp Configuration Language) that has a syntax very similar to JSON which allows you to easily read its code even though you have to learn how to develop in Terraform and its best practices. However, Crossplane works with YAML which is the language used by Kubernetes manifests so if you already have experience with Kubernetes, this makes new developments with Crossplane much easier.
- Community and support: Terraform has a larger community as it has been in the market for a longer time, as well as support, where it is possible to easily find troubleshooting threads. Crossplane, being a tool with less time on the market, does not have as much support, however, this is changing over time and already has different sources for it (Slack, Github, Stackoverflow…).
When to use Crossplane or Terraform?
After learning about two IaC tools, a question may arise… so, which tool should I use in each case?
Let’s look at a real case: A development team wants to create a new service, they create their repository, configure their Kubernetes manifests (Helm or not), generate the Docker image, pushing it and deploying it to the cluster.
It has no human interaction because CI/CD creates it automatically with its necessary resources (Ingress, Service, etc.). Even before the deployment of an application or component, it could automatically deploy its necessary infrastructure to run.
But what happens when a new functionality appears in the application that requires a new resource?
In many cases what happens is that you have to follow a procedure and a slow bureaucracy. Open a ticket, ask to create a resource (e.g. DB) and then the cloud team or the team that maintains the infrastructure, proceed with it.
This also involves a number of manual steps, for example setting the secrets as SSM Params or in a bucket, sharing it with the development team and having them modify the application to point to those secrets and connect to the DB. This process is slow and does not scale, especially if this process has to be repeated multiple times.
Here Crossplane appears, a tool that can be used for specific services/cases and accelerate this process.
Therefore, Crossplane and Terraform can complement each other being Terraform as a base infrastructure tool (the whole stack to have a working application) and Crossplane for new services/needs.
Do you want to tell us something?
CONTACT US!