Skip to content

Commit 0e3acdc

Browse files
docs: cleanup readme (#399)
Signed-off-by: Mathew Wicks <[email protected]>
1 parent 4e0d931 commit 0e3acdc

File tree

1 file changed

+91
-80
lines changed

1 file changed

+91
-80
lines changed

charts/airflow/README.md

Lines changed: 91 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -4,118 +4,129 @@ __Previously known as `stable/airflow`__
44

55
[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/airflow-helm)](https://artifacthub.io/packages/helm/airflow-helm/airflow)
66

7-
---
7+
## About
88

9-
This chart provides a standard way to deploy [Apache Airflow](https://airflow.apache.org/) on your Kubernetes cluster,
10-
and is used by thousands of companies for their production deployments of Airflow.
9+
The `Airflow Helm Chart (User Community)` provides a standard way to deploy [Apache Airflow](https://airflow.apache.org/) on Kubernetes with Helm, and is used by thousands of companies for production deployments of Airflow.
1110

12-
> 🟦 __Discussion__ 🟦
13-
>
14-
> The `user community` chart is an alternative to the `official` chart found in the `apache/airflow` repo.<br>
15-
> There are differences between the charts, so you should evaluate which is better for your organisation.
16-
>
17-
> The `user community` chart has existed since 2018 and was previously called `stable/airflow` on the official [helm/charts](https://github.com/helm/charts/tree/master/stable/airflow) repo.
18-
>
19-
> The goals of the `user community` chart are:<br>
20-
> (1) be easy to configure<br>
21-
> (2) support older airflow versions<br>
22-
> (3) provide great documentation<br>
23-
> (4) automatically detect bad configs<br>
11+
### Goals:
2412

25-
## Quickstart Guide
13+
(1) Ease of Use<br>
14+
(2) Great Documentation<br>
15+
(3) Support for older Airflow Versions<br>
16+
(4) Support for Kubernetes GitOps Tools (like ArgoCD)
2617

27-
These steps will allow you to quickly install Apache Airflow on your Kubernetes cluster using the `community` chart.
18+
### History:
2819

29-
### 1. Install the Chart
20+
The `Airflow Helm Chart (User Community)` is a popular alternative to the official chart released in 2021 inside the `apache/airflow` git repository.
21+
It was created in 2018 and was previously called `stable/airflow` when it lived in the (now end-of-life) [helm/charts](https://github.com/helm/charts/tree/master/stable/airflow) repository.
3022

31-
> 🟨 __Note__ 🟨
32-
>
33-
> In production, we encourage using a tool like [ArgoCD](https://argoproj.github.io/argo-cd/), rather than running `helm install` manually
23+
### Airflow Version Support:
24+
25+
Chart Version → <br> Airflow Version ↓ | `7.X.X` | `8.X.X` |
26+
--- | --- | ---
27+
`1.10.X` | ✅ | ✅️ <sub>[1]</sub>
28+
`2.0.X`| ❌ | ✅
29+
`2.1.X`| ❌ | ✅
30+
31+
<sub>[1] you must set `airflow.legacyCommands = true` to use airflow version `1.10.X` with chart version `8.X.X`
32+
33+
### Airflow Executor Support:
34+
35+
Chart Version → <br> Airflow Executor ↓ | `7.X.X` | `8.X.X` |
36+
--- | --- | ---
37+
`CeleryExecutor` | ✅ | ✅
38+
`KubernetesExecutor` | ✅️ <sub>[1]</sub> | ✅
39+
`CeleryKubernetesExecutor` | ❌ | ✅
40+
41+
<sub>[1] we encourage you to use chart version `8.X.X`, so you can use the `airflow.kubernetesPodTemplate.*` values (note, requires airflow `1.10.11+`, as it uses [AIRFLOW__KUBERNETES__POD_TEMPLATE_FILE](https://airflow.apache.org/docs/apache-airflow/2.1.0/configurations-ref.html#pod-template-file))
42+
43+
44+
## Quickstart Guide
3445

46+
### Install:
47+
48+
__(Step 1) - Add this helm repository:__
3549
```sh
36-
# add this repo as "airflow-stable"
50+
## add this helm repository & pull updates from it
3751
helm repo add airflow-stable https://airflow-helm.github.io/charts
3852
helm repo update
53+
```
3954

40-
# set environment variables (to make `helm install ...` cleaner)
41-
export RELEASE_NAME=my-airflow-cluster # a name
42-
export NAMESPACE=my-airflow-namespace # a namespace
43-
export CHART_VERSION=8.X.X # a chart version - https://github.com/airflow-helm/charts/blob/main/charts/airflow/CHANGELOG.md
44-
export VALUES_FILE=./custom-values.yaml # your values file
55+
__(Step 2) - Install this chart:__
56+
```sh
57+
## set the release-name & namespace
58+
export AIRFLOW_NAME="airflow-cluster"
59+
export AIRFLOW_NAMESPACE="airflow-cluster"
4560

46-
# use helm 3+ to install
61+
## install using helm 3
4762
helm install \
48-
$RELEASE_NAME \
63+
$AIRFLOW_NAME \
4964
airflow-stable/airflow \
50-
--namespace $NAMESPACE \
51-
--version $CHART_VERSION \
52-
--values $VALUES_FILE
65+
--namespace $AIRFLOW_NAMESPACE \
66+
--version "8.X.X" \
67+
--values ./custom-values.yaml
5368

54-
# wait until the above command returns (there are some post-install Jobs which may take a while)
69+
## wait until the above command returns (may take a while)
5570
```
5671

57-
### 2. Login to the WebUI
58-
72+
__(Step 3) - Locally expose the airflow webserver:__
5973
```sh
60-
export NAMESPACE=my-airflow-namespace # set a namespace!
74+
## port-forward the airflow webserver
75+
kubectl port-forward svc/${AIRFLOW_NAME}-web 8080:8080 --namespace $AIRFLOW_NAMESPACE
6176

62-
export POD_NAME=$(kubectl get pods --namespace $NAMESPACE -l "component=web,app=airflow" -o jsonpath="{.items[0].metadata.name}")
63-
kubectl port-forward --namespace $NAMESPACE $POD_NAME 8080:8080
64-
65-
# open the web-ui in your browser: http://localhost:8080
66-
# default user login: admin/admin
77+
## open your browser to: http://localhost:8080
78+
## default login: admin/admin
6779
```
6880

69-
## Important Documentation
70-
71-
### Changelog
72-
73-
The [CHANGELOG.md](CHANGELOG.md) is found at the root of this chart folder.
74-
75-
### Airflow Version Support
81+
### Upgrade:
7682

77-
See [the guide here](#how-to-use-a-specific-version-of-airflow) on how to explicitly set your airflow version.
83+
> __WARNING__: always consult the [CHANGELOG](CHANGELOG.md) before upgrading chart versions
7884
79-
. | `1.10.X` | `2.0.X` | `2.1.X`
80-
--- | --- | --- | ---
81-
chart - `7.X.X` | ✅ | ❌ | ❌
82-
chart - `8.X.X` | ✅ <sub>[1]</sub> | ✅ | ✅
83-
84-
<sub>[1] you must set `airflow.legacyCommands = true` to use airflow `1.10.X` with chart `8.X.X`
85+
```yaml
86+
## pull updates from the helm repository
87+
helm repo update
8588

86-
### Airflow Executor Support
89+
## apply any new values // upgrade chart version to 8.X.X
90+
helm upgrade \
91+
$AIRFLOW_NAME \
92+
airflow-stable/airflow \
93+
--namespace $AIRFLOW_NAMESPACE \
94+
--version "8.X.X" \
95+
--values ./custom-values.yaml
96+
```
8797

88-
Set your airflow executor-type using the `airflow.executor` value.
98+
### Uninstall:
8999

90-
. | `CeleryExecutor` | `KubernetesExecutor` | `CeleryKubernetesExecutor`
91-
--- | --- | --- | ---
92-
chart - `7.X.X` | ✅ | ✅ <sub>[1]</sub> | ❌
93-
chart - `8.X.X` | ✅ | ✅ | ✅
100+
```yaml
101+
## uninstall the chart
102+
helm uninstall $AIRFLOW_NAME --namespace $AIRFLOW_NAMESPACE
103+
```
94104

95-
<sub>[1] we encourage you to upgrade the chart to `8.X.X`, so you can use the `airflow.kubernetesPodTemplate` values (which require airflow `1.10.11+`, as they set [AIRFLOW__KUBERNETES__POD_TEMPLATE_FILE](https://airflow.apache.org/docs/apache-airflow/2.1.0/configurations-ref.html#pod-template-file)) </sub>
105+
### Examples:
96106

97-
### Examples
107+
To help you create your `custom-values.yaml` file, we provide some examples for common situations:
98108

99-
We provide some example `values.yaml` files for common configurations:
109+
- ["Minikube - CeleryExecutor"](examples/minikube/custom-values.yaml)
110+
- ["Google (GKE) - CeleryExecutor"](examples/google-gke/custom-values.yaml)
100111

101-
- [Minikube/Kind - CeleryExecutor](examples/minikube/custom-values.yaml)
102-
- [Google Kubernetes Engine (GKE) - CeleryExecutor](examples/google-gke/custom-values.yaml)
112+
### Frequently Asked Questions:
103113

104-
### Further Reading
114+
> __NOTE:__ some values are not discussed in the `FAQ`, you can view the default [values.yaml](values.yaml) file for a full list of values
105115
106-
We recommend you review the following questions from the FAQ:
116+
Review the FAQ to understand how the chart functions, here are some good starting points:
107117

108-
- [How to use a specific version of airflow?](#how-to-use-a-specific-version-of-airflow)
109-
- [How to set airflow configs?](#how-to-set-airflow-configs)
110-
- [How to create airflow users?](#how-to-create-airflow-users) or [How to authenticate airflow users with LDAP/OAUTH?](#how-to-authenticate-airflow-users-with-ldapoauth)
111-
- [How to create airflow connections?](#how-to-create-airflow-connections)
112-
- [How to use an external database?](#how-to-use-an-external-database)
113-
- [How to persist airflow logs?](#how-to-persist-airflow-logs)
114-
- [How to setup an Ingres?](#how-to-set-up-an-ingress)
118+
- ["How to use a specific version of airflow?"](#how-to-use-a-specific-version-of-airflow)
119+
- ["How to set airflow configs?"](#how-to-set-airflow-configs)
120+
- ["How to create airflow users?"](#how-to-create-airflow-users)
121+
- ["How to authenticate airflow users with LDAP/OAUTH?"](#how-to-authenticate-airflow-users-with-ldapoauth)
122+
- ["How to create airflow connections?"](#how-to-create-airflow-connections)
123+
- ["How to use an external database?"](#how-to-use-an-external-database)
124+
- ["How to persist airflow logs?"](#how-to-persist-airflow-logs)
125+
- ["How to set up an Ingress?"](#how-to-set-up-an-ingress)
115126

116127
## FAQ - Airflow
117128

118-
> These are some frequently asked questions related to airflow configs:
129+
> __Frequently asked questions related to airflow configs__
119130
120131
### How to use a specific version of airflow?
121132
<details>
@@ -1022,7 +1033,7 @@ serviceAccount:
10221033

10231034
## FAQ - Databases
10241035

1025-
> These are some frequently asked questions related to database configs:
1036+
> __Frequently asked questions related to database configs__
10261037

10271038
### How to use the embedded Postgres?
10281039
<details>
@@ -1146,7 +1157,7 @@ externalRedis:
11461157

11471158
## FAQ - Kubernetes
11481159

1149-
> These are some frequently asked questions related to kubernetes configs:
1160+
> __Frequently asked questions related to kubernetes configs__
11501161

11511162
### How to mount ConfigMaps/Secrets as environment variables?
11521163
<details>
@@ -1254,7 +1265,7 @@ ingress:
12541265

12551266
We expose the `ingress.web.precedingPaths` and `ingress.web.succeedingPaths` values, which are __before__ and __after__ the default path respectively.
12561267

1257-
> 🟦 __Discussion__ 🟦
1268+
> 🟦 __Tip__ 🟦
12581269
>
12591270
> A common use-case is [enabling SSL with the aws-alb-ingress-controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/tasks/ssl_redirect/), which needs a redirect path to be hit before the airflow-webserver one
12601271

@@ -1412,7 +1423,7 @@ extraManifests:
14121423

14131424
## Values Reference
14141425

1415-
> The list of values provided by this chart (see the [values.yaml](values.yaml) file for more details):
1426+
> __Values provided by this chart (for more info see [values.yaml](values.yaml))__
14161427

14171428
### `airflow.*`
14181429
<details>

0 commit comments

Comments
 (0)