A Crossplane Introduction

What is Crossplane and why should I care?

This can actually be a fairly difficult question to answer. Because it’s so flexible, Crossplane can be a lot of different things to a lot of different people. It could be looked at as a cloud-native replacement for Terraform or as a server-side Helm replacement. It is capable of replacing some aspects of your CD stack and also as a stepping stone to safe AI management of cloud infrastructure.

Regardless of how you look at it though, it is (probably) a better way to manage a platform than how you are doing it now.

Unfortunately for beginners, it can be a bit difficult to explain why you should care. In my experience, unless the person I’m talking to has a baseline understanding about how Crossplane works, it’s can be hard for them to visualize the major advantages it offers over the status quo specifically because it is so flexible and offers so much choice. There are some baseline things I can mention like constant reconciliation, elimination of a state file, templating in any language, etc. Those things don’t seem to be big enough motivators for engineers to consider adopting it though. The real eye opening stuff for me came once I got past the basics.

So to be able to better communicate the real benefits Crossplane introduces, I’m starting by writing this introduction to bring everyone up to speed before diving into more complex details, advanced implementation patterns, and longer term benefits of adoption. Hopefully then it’s easier to see why this is such a transformational piece of software for platform engineers.

The elevator pitch

Basically, Crossplane allows you to easily build custom resource types which automatically render templated collections of Kubernetes resources in your cluster. This provides a lot of advantages over things like internally authored Helm charts or Terraform modules and doesn’t require you to build or maintain your own Kubernetes operator.

Let’s start by getting a very simple example up and running. We can talk about the benefits and drawbacks later.

An example

Let’s say you want to create a custom resource type for managing databases. You could do that by creating an object with a schema like this:

apiVersion: crossplane.my-company.com/v1alpha1
kind: Database
metadata:
  name: my-test-database
  namespace: default
spec:
  engine: postgres
  version: 18.1

What happens behind the scenes when you apply this is entirely up to you. You could be creating a database with a managed provider like RDS or Cloud SQL. However, it could just as easily be an in-cluster database using CloudNativePG. It all depends on what resources you are templating.

What essentially happens is that the platform team can work out the lower level details for a given resource type, test that it reliably works, and offer it to the developers as a self-service object they can own. The platform team gets to implement best practices for reliability, security requirements, budget tracking, and they achieve platform consistency. The developer gets to create their own infrastructure, without going to an ops team for help, and doesn’t need to worry about the lower level details.

Developers also don’t need to worry about understanding the eccentricities of Terraform’s plan/apply workflow. They simply need to push their resources to git, the CD platform syncs the manifest to the cluster automatically, and Crossplane creates the resources on the user’s behalf.

With Terraform you would be running a plan, review, apply, and sometimes repeat if the apply doesn’t succeed. With Crossplane, you don’t need to babysit the changes. If there is a problem, as with any other resource or service, your monitoring platform should alert the responsible party. You assume it will work (because you tested it, right?) but monitor and alert when it doesn’t.

And because these resources are all basic yaml manifests with a standardized Kubernetes structure, it’s really easy to integrate this with an internal developer portal like Backstage too. Just have some kind of interface for developers to describe what they want (Backstage form, AI agent, etc), have it render yaml, commit it to a git branch, and open a PR. You can automatically validate the manifests through CI jobs and enforce guardrails through Kyverno policies.

Sounds good. How does it work?

It’s pretty simple actually.

The custom resource type is applied to a cluster.
Crossplane automatically recognizes the new resource and begins to render the template you defined using that custom resource as input. Similar to a Helm values file.
The new rendered resources are applied to the cluster.

This is essentially it. It’s not that scary, right? Following this simple pattern, you could create custom resources for things like InternalService, ClusterWorkload, Database, MessageQueue, or anything else that might be appropriate to your organization.

Implementing these things requires understanding just few basic components but that’s about it. They include:

composite resource definitions
composite resources
compositions
functions

Composite Resource Definitions

In the Crossplane ecosystem, these are commonly referred to as XRDs. This is the schema definition your new resource type will use. This is an XRD that implements the above Database example:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: database
spec:
  group: crossplane.my-company.com
  names:
    kind: Database
    plural: databases
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              engine:
                description: The DB engine to create
                type: string
                # The allowed values
                enum:
                  - postgres
                  - mysql
                  - mongo
              version:
                description: The DB version
                type: string
            required:
            - engine
            - version

With a defined spec, we can ensure that any requests to create a database will follow the same pattern. Regardless of who created it or how. It’s only valid if it follows this spec. Our time spent troubleshooting typos, whitespace inconsistencies, incorrect options copy and pasted from other locations, etc is greatly reduced. Those things can still happen of course but it would only happen with one manifest. Without this pattern you could be troubleshooting those same issues across 10-20 different resources.

And because it’s simply a kubernetes resource, any manifest we write can easily be tested with tools like kubeconform or simply by running kubectl apply --dry-run=server.

Composite Resources

In the Crossplane ecosystem, these are commonly referred to as XRs. These are resources which are based off of XRDs like the one we defined in the previous section. Each resource you create of kind: Database is considered a composite resource (or XR) because it is a resource that represents a collection of other resources, a composite. There can be multiple XRs for a single XRD per cluster.

Compositions

These are how you define the templates for the child resources that will be generated when someone applies an XR to the cluster. I’ll cover the details about compositions in a bit more detail later but for now here is a simplified example to implement the Database resource type above.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: database
spec:
  compositeTypeRef:
    apiVersion: crossplane.my-company.com/v1alpha1
    kind: Database
  mode: Pipeline
  pipeline:
    - step: database-templates
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            # Defining the XR so it can be more easily used in the template
            {{ $xr := .observed.composite.resource }}

            {{ if eq $xr.spec.engine "postgres" }}
              ---
              # Postgres is created in cluster using CNPG
              apiVersion: postgresql.cnpg.io/v1
              kind: Cluster
              metadata:
                name: {{ $xr.metadata.name }}
                namespace: {{ $xr.metadata.namespace }}
              spec:
                imageName: ghcr.io/cloudnative-pg/postgresql:{{ $xr.spec.version }}
                ...
            {{ else if eq $xr.spec.engine "mysql" }}
              ---
              # MySQL is created using AWS RDS
              apiVersion: rds.aws.m.upbound.io/v1beta1
              kind: Cluster
              metadata:
                name: {{ $xr.metadata.name }}
                namespace: {{ $xr.metadata.namespace }}
              spec:
                forProvider:
                  engine: mysql
                  engineVersion: {{ $xr.spec.version }}
                  region: us-west-1
                  ...
            {{ end }}

You can really template anything you want to. As long as it’s a Kubernetes resource, Crossplane can handle it. It could be custom resources from Kubernetes operators like CNPG or cert-manager, resources provided by Upbound’s cloud provider families, or kubernetes native resources like Deployment, ServiceAccounts, or Ingresses can all be grouped into the same composition.

Functions

Notice in our composition above that there is a functionRef field that references function-go-templating. This is one of the biggest advantages of Crossplane. That is that you can choose what language you want to use for writing your compositions. In Terraform, you only have HCL. In Helm, you only have Go templating. In Crossplane, you can mix and match different functions depending on the needs or preferences of the team maintaining a composition.

For example, let’s say I don’t like Go templating and I’ve been wanting to try out one of the new purpose built languages for managing yaml. CUE Lang has been getting some attention lately. I’ll try that. It looks like there is a function-cue already so I can write the exact same composition like this instead:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: database
spec:
  compositeTypeRef:
    apiVersion: crossplane.my-company.com/v1alpha1
    kind: Database
  mode: Pipeline
  pipeline:
    - step: database-templates
      functionRef:
        name: function-cue
      input:
        apiVersion: function-cue/v1
        kind: CueFunctionInput
        source: Inline
        script: |
          #request: {...}

          xr: #request.observed.composite.resource

          response: desired: resources: {
            "database": resource: {
              if xr.spec.engine == "postgres" {
                apiVersion: "postgresql.cnpg.io/v1"
                kind: "Cluster"
                metadata: {
                  name:      xr.metadata.name
                  namespace: xr.metadata.namespace
                }
                spec: {
                  imageName: "ghcr.io/cloudnative-pg/postgresql:\(xr.spec.version)"
                }
              }

              if xr.spec.engine == "mysql" {
                apiVersion: "rds.aws.m.upbound.io/v1beta1"
                kind: "Cluster"
                metadata: {
                  name: xr.metadata.name
                  namespace: xr.metadata.namespace
                }
                spec: {
                  forProvider: {
                    engine: "mysql"
                    engineVersion: xr.spec.version
                    region: "us-west-1"
                  }
                }
              }
            }
          }

Functions allow us to change how we build compositions and also allows for us to alter the resources that are generated.

For language support alone, there are these options available to you:

function-go-templating
function-cue
function-kcl
function-python
function-hcl
function-patch-and-transform

There are many other functions available that cover a lot of different use cases, not just templating languages.

Summary

These 3-4 components are all you really need to get started. A composition, using whatever templating function you want, and an XRD that defines options exposed to the end user. With those in place you can apply your XRs to the cluster. That’s it!

Next time I plan on covering some of the wider ecosystem in more detail. Things like providers plus more detail about compositions and functions. The patterns that emerge from that will be a lot more interesting so keep and eye out for it.

Tip

A really great tool for inspecting the state of child resources created through compositions is the kubectl krew plugin tree.