Please provide your feedback in this short Flings' survey.

Operator Builder for Tanzu

version v0.2.0-plugin.1 — October 04, 2021

Contributors 5
View All

Release Date: August 30, 2021

Summary

In Kubernetes, operators are the future of application and platform management. They allow consumers to extend the Kubernetes API in order to implement the business logic at the core of their running applications. Operator Builder for Tanzu is designed to facilitate the development and maintenance of operators on the Tanzu Kubernetes Grid (TKG) platform.

NOTE: This Fling was developed primarily to help TKG users, and it is entirely agnostic of the Kubernetes distribution in use. It can be used to develop operators compatible with any Kubernetes cluster.

Generally speaking, although operators are highly beneficial in a variety of use cases, it can commonly take weeks or even months to release an initial, working version of an operator. Operator Builder for Tanzu can reduce that time from weeks or months down to days or even hours in most cases. This allows platform engineers and developers to focus on other improvements and innovations by reducing routine toil and tedious maintenance of application deployments.

Operator Builder for Tanzu will produce a working operator given a set of known good YAML manifests with basic create, update, and delete functionality out of the box. It can be extended in any way the developer sees fit to include such things as adding business logic for other common tasks such as upgrades, backups, failovers, etc.

Requirements

VMware

  • VMware Tanzu Kubernetes Grid (or other Kubernetes distribution): this is where your operator will run.
  • Tanzu CLI >= 1.3: this plugin will be run on top of the Tanzu CLI.

Other

  • Kubernetes YAML Manifests: these are your known good manifests that are the input to the generated source code.
  • Go >= 1.16: this is used to test the generated source code.
  • Kubectl: this is to interact with the target Kubernetes cluster.
  • Git: used for version controlling the generated source code.

Suggested (Optional)

  • Docker: used for building images to run the source code in a Kubernetes cluster.
  • Make: used for running simple tasks within the project such as running the operator or building an image.
  • Bash: the underlying shell used for running make targets.
Instructions

For full documentation, please visit: https://github.com/vmware-tanzu-labs/operator-builder

Be sure to review the requirements section prior to proceeding with installation and configuration instructions.

Run the following command to install the plugin to be used by the Tanzu CLI:


            tanzu plugin install operator -l {PATH_TO_DOWNLOADED_PLUGIN}
            

Configuration

Mark Your YAML

Mark your YAML manifests with the appropriate workload markers. See workload markers for more details. These markers serve as options that you are exposing to end users to be able to modify your underlying application.

Create Your Workload Configuration

Once your YAML manifests have been marked, you need to tell the marked manfiests to be used in the generated source code output. See workload configuration for more details.

Initialize the Operator

Initialize the operator codebase. This will generate some base code for your operator:


        tanzu operator init --workload-config {CONFIG_FROM_LAST_STEP}.yaml --repo {GIT_REPO}
        
The above command will utilize the workload configuration created in the previous step and create the project to be stored in the source code repository at GIT_REPO. The source code is required and is a general Go convention that we follow, even if the code will never make it into a source code repository.

Create the Operator

Generate the source code for the operator's custom API and controller:


        tanzu operator create api --workload-config {CONFIG_FROM_LAST_STEP}.yaml --controller --resource
        
The above command will output all of the source code required to run your operator.

Run the Operator (Test)

Install the custom resource definition in your test cluster:


        make install
        
Run the controller's code locally for testing purposes:

        make run
        
Now install a sample of your custom resource:

        kubectl apply -f config/samples
        
The above command will submit sample manifests to your cluster and the operator will watch for these resources and act accordingly.

Run the Operator (In-Cluster)

This process is what you will use if you were to run the operator within a cluster on a persistent basis.


        export IMG={MY_IMAGE_REPO}/{MY_IMAGE}:{MY_IMAGE_VERSION}; make docker-build; make docker-push
        

The above command builds an image with your source code in it, and will push the image to a repository. The MY_IMAGE settings above will need to be replaced with your specific image repository information.


        make deploy
        

The above command will deploy the source code given the image you just pushed above into a Kubernetes cluster.

Supporting Information

The following is additional information that is helpful when configuring your operator.

Workload Markers

Workload Markers

Operator Builder uses commented markers as the basis for defining a new API. The fields for a custom resource kind are created when it finds a +operator-builder:field marker in a source manifest. Alternatively, for fields which are shared across workloads when using the workload collection feature, you can use +operator-builder:collection:field instead.

A workload marker is commented out so the manifest is still valid and can be used if needed. The marker must begin with +operator-builder followed by some comma-separated fields. Markers can either be at the end of a line or at the head of a line:

  • name: The name field is required. It should be provided as you want it to be in the spec of the resulting custom resource.
  • type: This field is provided as type=[value]. It is also a required field. The supported data types:
    • bool
    • string
    • int
    • int32
    • int64
    • float32
    • float64
  • default: This field is provided as default=[value]. It is an optional field. If provided it will make the field optional in the custom resource and when not included will get the default value provided.
  • description: This field is provided as description=[value]. It is an optional field. If provided, it will give documentation to the generated resource definition. For more information, see workload docs.

Consider the following Deployment:


            ---
            apiVersion: apps/v1
            kind: Deployment
            metadata:
              name: webapp-deploy
              labels:
                production: false  # +operator-builder:field:name=production,default=false,type=bool
            spec:
              # +operator-builder:field:name=webAppReplicas,default=2,type=int
              replicas: 2
              selector:
                matchLabels:
                  app: webapp
              template:
                metadata:
                  labels:
                    app: webapp
                spec:
                  containers:
                  - name: webapp-container
                    image: nginx:1.17  # +operator-builder:field:name=webAppImage,type=string
                    ports:
                    - containerPort: 8080
            

In this case, operator-builder will create and add three fields to the custom resource:

  • A production field that is a boolean. It will have a default of false and will inform the value of the label when the deployment is configured.
  • A webAppReplicas field that will default to 2 and allow the user to specify the number of replicas for the deployment in the custom resource manifest.
  • A webAppImage field that will set the value for the images used in the pods.

Now the end-user of the operator will be able to define a custom resource similar to the following to configure the deployment created:


            ---
            apiVersion: product.apps.acme.com/v1alpha1
            kind: WebApp
            metadata:
              name: dev-webapp
            spec:
              production: false
              webAppReplicas: 2
              webAppImage: acmerepo/webapp:3.5.3
            
Workload Document Markers

Workload document adds functionality to workload markers (see workload markers for more information) by allowing the user to inject documentation into the CRD that gets generated by using a description tag in the operator-builder marker. By injecting documentation to the CRD, the consumer of the custom resource gets the added benefit of being able to run kubectl explain against their resource and having documentation right at their fingertips without having to navigate to API documentation in order to see the usage of the API.

Documentation is injected by simply placing a single or multi-line description tag in the +operator-builder marker. Multi-line documentation can be used by using a backtick surrounding the documentation (`).

Considering the following simple resource which is being managed by operator-builder:

            ---
            apiVersion: tenancy.platform.cnr.vmware.com/v1alpha1
            kind: TanzuNamespace
            metadata:
              name: cnpo-security-system #+operator-builder:field:name=namespace,default="cnpo-security-system",type=string
              spec:
                #+operator-builder:field:name=namespace,default="cnpo-security-system",type=string,description=`
                # Defines the namespace in which all
                # resources for this component will be placed into.`
                tanzuNamespaceName: cnpo-security-system

In this case, operator-builder will simply manage the tanzuNamespaceName field as a workload marker which we are calling namespace in our custom CRD. In addition to managing that field, the +operator-builder marker with the description tag is used as the docmentation for that marker.

The end state of this configuration is templated Go code which looks like so:


            // SecurityCommonComponentSpec defines the desired state of SecurityCommonComponent.
            type SecurityCommonComponentSpec struct {
                // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
                // Important: Run "make" to regenerate code after modifying this file

                // +kubebuilder,default="cnpo-security-system"
                // +kubebuilder:validation:Optional
                // Defines the namespace in which all resources for this component will be placed into.
                Namespace string `json:"namespace"`

And when the make manifests command is run, the CRD is generated with the documentation in the description filed which looks like:

            ...
                          namespace:
                            default: cnpo-security-system
                            description: Defines the namespace in which all resources for this
                              component will be placed into.
                            type: string
            ...

Finally, the kubectl explain command now contains the documentation so that users may see the documentation by way of kubectl and not any other tooling:

            kubectl explain securitycommoncomponents.spec.namespace

            KIND:     SecurityCommonComponent
            VERSION:  security.cnpo.tanzulabs.vmware.com/v1alpha1

            FIELD:    namespace <string>

            DESCRIPTION:
                 Defines the namespace in which all resources for this component will be
                 placed into.

API documentation in markdown can also be generated after building an operator. To do so, simply run the command make docs from the root of your repository. This will build the documentation in the docs/apis.md file.



Workload Configuration

Standalone Workload

When you are building an operator for a single workload, you will just need a single standalone WorkloadConfig along with the source manifests to define the resources that will be created to fulfill an instance of your workload.

For example if your organization develops and maintains a web application as a part of its core business, you may consider using an operator to deploy and maintain that app in different environments. We refer to all the various resources that comprise that web app collectively as a "workload."

This is the simplest and most common implemetation of operator-builder.

If you have multiple workloads that have dependencies upon one another, and it makes sense to orchestrate them with an operator, a standalone workload will not suffice. For that you will need to leverage a workload collection.

Below is an example standalone workload:


        ---
        name: webstore
        kind: StandaloneWorkload
        spec:
          api:
            domain: acme.com
            group: apps
            version: v1alpha1
            kind: WebStore
            clusterScoped: false
          companionCliRootcmd:
            name: webstorectl
            description: Manage webstore application
          resources:
            - app.yaml
        
Component Workload

A component workload is identical to a standalone workload with one key difference. The difference is that the component workloads may have dependencies between them and are made up of a broader collection workload. The same is true for standalone versus component with the exception being a new dependencies field:

Below is an example component workload:

        ---
        name: webstore
        kind: ComponentWorkload
        spec:
          api:
            domain: acme.com
            group: apps
            version: v1alpha1
            kind: WebStore
            clusterScoped: false
          companionCliRootcmd:
            name: webstorectl
            description: Manage webstore application
          dependencies:
            - my-other-webstore-name
          resources:
            - app.yaml
        
Workload Collection

If you are building an operator to manage a collection of workloads that have dependencies upon one another, you will need to use a workload collection. If, instead, you have just a single workload to manage, you will want to use a standalone workload.

A workload collection is defined by a WorkloadConfig - just like any other workload used with Operator Builder. The only differences are that it will include collection: true and an array of workload definitions under the componentFiles field.


            ---
            name: webstore-collection
            kind: WorkloadCollection
            spec:
              api:
                domain: plugins.operator-builder.tanzulabs.vmware.com
                group: webstorecollection
                version: v1alpha1
                kind: WebStoreCollection
                clusterScoped: true
              companionCliRootcmd:
                name: wsc
                description: Manage webstore-collection component
              componentFiles:
                - webstore.yaml
                - my-other-webstore-name.yaml
            

Each of the componentFiles are ComponentWorkload configurations that may have dependencies upon one another which the operator will manage. This project will include a custom resource for each of the component workloads as well as a distinct controller for each component workload. Each of these controllers will run in a single containerized controller manager in the cluster when it is deployed.

Changelog
v0.2.0-plugin.1
  • Fixes bug https://flings.vmware.com/operator-builder-for-tanzu/bugs/1331 which excluded YAML metadata required for CLI plugin installation
  • Packaged as a single bundle rather than individual artifacts to make download process simpler
v0.2.0
  • Initial Public Release