Kubedoop Operator for Apache Airflow

This is a Kubernetes operator to manage Apache Airflow ensembles.

It's part of the Kubedoop Data Platform, a modular open source data platform built on Kubernetes that provides Kubernetes native deployment and management of popular open source data apps like Apache Kafka, Apache Doris, Trino or Apache Spark, all working together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.

Quick Start

Add helm repository

Please make sure helm version is v3.0.0+

helm repo add kubedoop https://zncdatadev.github.io/kubedoop-helm-charts/

Add required dependencies

helm install commons-operator kubedoop/commons-operator
helm install listener-operator kubedoop/listener-operator
helm install secret-operator kubedoop/secret-operator

Add airflow-operator

helm install airflow-operator kubedoop/airflow-operator

Deploy Airflow cluster

kubectl apply -f config/samples

Kubedoop Data Platform Operators

These are the operators that are currently part of the Kubedoop Data Platform:

• Kubedoop Operator for Apache Airflow
• Kubedoop Operator for Apache DolphinScheduler
• Kubedoop Operator for Apache Doris
• Kubedoop Operator for Apache Hadoop HDFS
• Kubedoop Operator for Apache HBase
• Kubedoop Operator for Apache Hive
• Kubedoop Operator for Apache Kafka
• Kubedoop Operator for Apache Kyuubi
• Kubedoop Operator for Apache Nifi
• Kubedoop Operator for Apache Spark
• Kubedoop Operator for Apache Superset
• Kubedoop Operator for Trino
• Kubedoop Operator for Apache Zookeeper

And our internal operators: :

• Commons Operator
• Listener Operator
• Secret Operator

Contributing

If you'd like to contribute to Kubedoop, please refer to our Contributing Guide for more information. We welcome contributions of all kinds, including but not limited to code, documentation, and use cases.

License

Kubedoop is under the Apache 2.0 license. See the LICENSE file for details.