Commit 2d83698e authored by BO ZHANG's avatar BO ZHANG 🏀
Browse files

feat: 新增Worker标签与DAG槽位显示,优化部署配置

parent d844b898
Loading
Loading
Loading
Loading
+30 −7
Original line number Diff line number Diff line
@@ -18,10 +18,19 @@

如果您需要在真实的集群(1个 Master 节点 + 多个 Worker 节点)上部署该系统,我们提供了开箱即用的 Ansible Playbooks。由于我们有多个不同的部署环境(如 `p368`, `csu`, `zjlab`),我们为每个环境准备了专属的 Ansible Inventory 文件。

**💡 进阶:自定义 Worker 节点的并发度**
您可以在 Inventory 文件(例如 `inventory.p368.ini`)中,为每台 Worker 机器灵活配置变量:
- `worker_concurrency`: 设置该台机器上 Worker 容器的 Slots 数量(例如 16 或 64)。
*(通过这种方式,您可以让高配物理机加大并发,低配物理机少跑,充分压榨集群性能!)*
**💡 进阶:为每个 Worker 单独指定并发度**
您可以直接在 Inventory 文件(例如 `inventory.p368.ini`)的每个 Worker 节点条目上设置 `worker_concurrency`,从而让不同机器使用不同的 Celery 并发数:

```ini
[worker]
worker-a ansible_host=192.168.25.19 ansible_user=root deploy_dir=/opt/csst-airflow-worker-a worker_concurrency=16
worker-b ansible_host=192.168.25.20 ansible_user=root deploy_dir=/opt/csst-airflow-worker-b worker_concurrency=64
worker-c ansible_host=192.168.25.21 ansible_user=root deploy_dir=/opt/csst-airflow-worker-c worker_concurrency=8
```

- `worker_concurrency`:该节点上 Airflow Celery Worker 的并发槽位数。
- 建议按机器 CPU / 内存资源分别设置,不要所有 Worker 都使用同一个值。
- 如无特殊需要,请在每个 Worker 上显式写出该值,避免依赖默认值。

#### 1. 准备工作
- **安装 Ansible**:如果您的执行机(通常是您的本地电脑或 Master 节点)尚未安装 Ansible,请先通过以下命令安装:
@@ -56,8 +65,8 @@
node-master ansible_host=192.168.25.18 ansible_user=root deploy_dir=/opt/csst-airflow-master

[worker]
node-worker1 ansible_host=192.168.25.19 ansible_user=root deploy_dir=/opt/csst-airflow-worker1
node-worker2 ansible_host=192.168.25.20 ansible_user=root deploy_dir=/opt/csst-airflow-worker2
node-worker1 ansible_host=192.168.25.19 ansible_user=root deploy_dir=/opt/csst-airflow-worker1 worker_concurrency=16
node-worker2 ansible_host=192.168.25.20 ansible_user=root deploy_dir=/opt/csst-airflow-worker2 worker_concurrency=32

[all:vars]
env_name=csu
@@ -65,6 +74,20 @@ env_name=csu
deploy_dir=/opt/csst-airflow
```

如果多个 Worker 机器规格相同,也可以在 `[worker:vars]` 中统一配置默认值,再在个别节点上单独覆盖:

```ini
[worker:vars]
worker_concurrency=16

[worker]
node-worker1 ansible_host=192.168.25.19 ansible_user=root deploy_dir=/opt/csst-airflow-worker1
node-worker2 ansible_host=192.168.25.20 ansible_user=root deploy_dir=/opt/csst-airflow-worker2
node-worker3 ansible_host=192.168.25.21 ansible_user=root deploy_dir=/opt/csst-airflow-worker3 worker_concurrency=32
```

上述配置会在部署时写入目标节点的 `.env`,并在启动 Worker 容器时传递给 `AIRFLOW__CELERY__WORKER_CONCURRENCY`,因此只需要修改 Inventory 后重新执行 Ansible 部署或更新即可生效。

#### 3. 配置环境变量
为了让集群中的所有组件(特别是 Worker 节点)能够正确找到主节点和其他依赖,您需要为该环境配置专用的 `.env` 文件。
`envs/` 目录下,复制或编辑对应环境的 `.env` 文件(例如 `envs/csu.env`)。您**必须**修改 `MASTER_IP``HARBOR` 这两个关键变量:
+20 −0
Original line number Diff line number Diff line
@@ -59,6 +59,16 @@
    _env_name: "{{ env_name | default('p368') }}"
    _deploy_dir: "{{ deploy_dir | default('/opt/csst-airflow') }}"
  tasks:
    - name: Ensure envs directory exists
      file:
        path: "{{ _deploy_dir }}/envs"
        state: directory

    - name: Render .env file from template
      template:
        src: "{{ playbook_dir }}/roles/common/templates/env.j2"
        dest: "{{ _deploy_dir }}/envs/{{ _env_name }}.env"

    - name: Start Master services (postgres, redis, elasticsearch, airflow core, api, etc.)
      shell: |
        {{ docker_compose_cmd }} --env-file envs/{{ _env_name }}.env --profile master up -d
@@ -72,6 +82,16 @@
    _env_name: "{{ env_name | default('p368') }}"
    _deploy_dir: "{{ deploy_dir | default('/opt/csst-airflow') }}"
  tasks:
    - name: Ensure envs directory exists
      file:
        path: "{{ _deploy_dir }}/envs"
        state: directory

    - name: Render .env file from template
      template:
        src: "{{ playbook_dir }}/roles/common/templates/env.j2"
        dest: "{{ _deploy_dir }}/envs/{{ _env_name }}.env"

    - name: Start Worker services (celery-worker, filebeat, monitoring)
      shell: |
        export AIRFLOW__CELERY__WORKER_CONCURRENCY={{ worker_concurrency | default(16) }}
+5 −4
Original line number Diff line number Diff line
[master]
# Replace with your actual master node IP
# Replace with your actual master node IP or hostname
csu-master ansible_host=192.168.25.18 ansible_user=root

[worker]
# Add all your worker node IPs here
# e.g., csu-worker1 ansible_host=192.168.25.19 ansible_user=root
# Add one worker node per line
# e.g., csu-worker1 ansible_host=192.168.25.19 ansible_user=root deploy_dir=/opt/csst-airflow-worker1 worker_concurrency=16
# e.g., csu-worker2 ansible_host=192.168.25.20 ansible_user=root deploy_dir=/opt/csst-airflow-worker2 worker_concurrency=32

[all:vars]
# The environment name to deploy (e.g., p368, csu, zjlab)
+5 −4
Original line number Diff line number Diff line
[master]
# Replace with your actual master node IP
# Replace with your actual master node IP or hostname
zjlab-master ansible_host=10.200.60.244 ansible_user=root

[worker]
# Add all your worker node IPs here
# e.g., zjlab-worker1 ansible_host=10.200.60.245 ansible_user=root
# Add one worker node per line
# e.g., zjlab-worker1 ansible_host=10.200.60.245 ansible_user=root deploy_dir=/opt/csst-airflow-worker1 worker_concurrency=16
# e.g., zjlab-worker2 ansible_host=10.200.60.246 ansible_user=root deploy_dir=/opt/csst-airflow-worker2 worker_concurrency=32

[all:vars]
# The environment name to deploy (e.g., p368, csu, zjlab)
+1 −0
Original line number Diff line number Diff line
@@ -13,3 +13,4 @@ HARBOR={{ harbor }}
# --- Celery Worker Config ---
# Defines the concurrency per worker node. Can be overridden in inventory.
WORKER_CONCURRENCY={{ worker_concurrency | default(8) }}
WORKER_LABEL={{ ansible_host }}
Loading