DocsConcepts and ExpressionsTask RunnersTask Runner TypesKubernetes Task Runner

Kubernetes Task Runner​Kubernetes ​Task ​Runner

Available on: >= 0.16.0

Run tasks as Kubernetes pods.

This runner executes the tasks on the specified Kubernetes cluster. It is useful to declare resource limits and resource requests.

  • If your script task has inputFiles or namespaceFiles configured, an init container will be added to upload files into the main container.
  • If your script task has outputFiles configured, a sidecar container will be added to download files from the main container.

All containers will use an in-memory emptyDir volume for file exchange.

If a task is resubmitted (e.g. due to a retry or a Worker crash), the new Worker will reattach to the already running (or an already finished) pod instead of starting a new one.

Here is a simple example of Shell commands executed in a Kubernetes pod:

yaml
id: kubernetes_script_runner
namespace: company.team

description: |
  To get the kubeconfig file, run: `kubectl config view --minify --flatten`.
  Then, copy the `server`, `caCert`, `clientKey`, `clientCert`, `user`, and `namespace` values to the configuration below.
  Here is a mapping of the kubeconfig file to the Kubernetes task runner configuration:
  - clientKey: client-key-data
  - clientCert: client-certificate-data
  - caCert: certificate-authority-data
  - masterUrl: server
  - username: user

inputs:
  - id: file
    type: FILE

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    inputFiles:
      data.txt: "{{ inputs.file }}"
    outputFiles:
      - out.txt
    containerImage: centos
    taskRunner:
      type: io.kestra.plugin.kubernetes.runner.Kubernetes
    commands:
      - cp data.txt out.txt

And here is an example of a Python script executed in a Kubernetes pod:

yaml
id: kubernetes_script_runner
namespace: company.team

tasks:
  - id: send_data
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/kestra-io/pydata:latest
    taskRunner:
      type: io.kestra.plugin.kubernetes.runner.Kubernetes
      namespace: default
      pullPolicy: Always
      config:
        username: docker-desktop
        masterUrl: https://docker-for-desktop:6443
        caCertData: xxx
        clientCertData: xxx
        clientKeyData: xxx
      resources:
        request: # The resources the container is guaranteed to get
          cpu: "500m" # Request 1/2 of a CPU (500 milliCPU)
          memory: "128Mi" # Request 128 MB of memory
    outputFiles:
      - "*.json"
    script: |
      import platform
      import socket
      import sys
      import json
      from kestra import Kestra

      print("Hello from a Kubernetes runner!")

      host = platform.node()
      py_version = platform.python_version()
      platform = platform.platform()
      os_arch = f"{sys.platform}/{platform.machine()}"


      def print_environment_info():
          print(f"Host's network name: {host}")
          print(f"Python version: {py_version}")
          print(f"Platform info: {platform}")
          print(f"OS/Arch: {os_arch}")

          env_info = {
              "host": host,
              "platform": platform,
              "os_arch": os_arch,
              "python_version": py_version,
          }
          Kestra.outputs(env_info)

          filename = "environment_info.json"
          with open(filename, "w") as json_file:
              json.dump(env_info, json_file, indent=4)


      if __name__ == '__main__':
          print_environment_info()

Was this page helpful?