跳到主要内容

DolphinScheduler 3.1.9 Helm部署方案

核心规范:

  • 部署路径:统一存放至 Linux 标准工具目录 /opt/dolphinscheduler-deploy(便于运维管理)
  • 适配环境:Ubuntu 24.04 LTS + k3s 集群 + Longhorn 分布式存储 + 外部 PostgreSQL + MetalLB(固定 IP 10.2.10.154)
  • 部署方式:基于官方源码包本地 Helm 安装(无第三方仓库依赖,稳定不失效)
  • 核心特性:Helm 自动创建 Longhorn 存储类、MetalLB 固定 IP 访问、禁用内置 PostgreSQL

一、方案概述​

本方案将 DolphinScheduler 3.1.9 的源码包、配置文件等所有部署相关文件统一存放至 /opt/dolphinscheduler-deploy 目录(符合 Linux 系统工具目录规范),全程通过本地 Helm 安装,无需依赖第三方 Helm 仓库,同时实现:

  1. 自动创建适配 Longhorn 的 dolphinscheduler-longhorn 存储类(无手动配置);
  2. 通过 MetalLB 分配固定 IP 10.2.10.154 直接访问 Web UI;
  3. 对接外部 PostgreSQL 数据库,通过官方标准Secret管理数据库密码,保障数据安全;
  4. 完全对齐官方模板逻辑,避免密码读取失败问题。

二、前置环境确认​

2.1 基础依赖检查​

# 1. 确认 k3s 集群节点状态(所有节点需为 Ready 状态)
kubectl get nodes

# 2. 确认 Longhorn 存储集群正常(所有 Pod 为 Running)
kubectl get pods -n longhorn-system

# 3. 确认 Helm 3.x 版本可用
helm version

# 4. 确认 MetalLB 已部署且 IP 池包含 10.2.10.154(无需重复部署)
kubectl get ipaddresspool -n metallb-system

2.2 创建标准化部署目录​

# 创建 /opt 下的专属部署目录(统一管理所有文件)
sudo mkdir -p /opt/dolphinscheduler-deploy
sudo chmod 755 /opt/dolphinscheduler-deploy

# 确认目录创建成功
ls -ld /opt/dolphinscheduler-deploy

三、数据库准备(外部 PostgreSQL)​

3.1 初始化 PostgreSQL 数据库​

# 1. 建库
CREATE DATABASE dolphinscheduler;

# 2. 切换到dolphinscheduler数据库
\c dolphinscheduler

CREATE USER dolphinscheduler WITH PASSWORD 'dolphinscheduler';

# 3. 赋权public schema核心权限(逐条执行,缺一不可)
# 3.1 授予schema的USAGE权限(基础访问权限)
GRANT USAGE ON SCHEMA public TO dolphinscheduler;

# 3.2 授予schema的CREATE权限(创建表/序列等)
GRANT CREATE ON SCHEMA public TO dolphinscheduler;

# 3.3 授予当前schema下所有对象的权限
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO dolphinscheduler;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO dolphinscheduler;
GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN SCHEMA public TO dolphinscheduler;

# 3.4 授予未来创建对象的默认权限
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON TABLES TO dolphinscheduler;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON SEQUENCES TO dolphinscheduler;

# 4. 退出PostgreSQL
\q

四、核心部署步骤(基于官方源码包)​

4.1 下载并解压官方源码包(存放至 /opt)​

helm pull oci://registry-1.docker.io/apache/dolphinscheduler-helm --version 3.1.9
tar -xvf dolphinscheduler-helm-3.1.9.tgz -C /opt/dolphinscheduler-deploy/
cd /opt/dolphinscheduler-deploy/dolphinscheduler-helm/

4.2 验证 Longhorn 存储类​

# 1. 创建存储类配置文件
cat > /opt/dolphinscheduler-deploy/dolphinscheduler-helm/dolphinscheduler-longhorn-sc.yaml << EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: dolphinscheduler-longhorn
annotations:
storageclass.kubernetes.io/is-default-class: "false" # 不设为默认存储类
provisioner: driver.longhorn.io # Longhorn驱动(必须已安装Longhorn)
parameters:
numberOfReplicas: "2" # 副本数(测试环境1个即可)
staleReplicaTimeout: "30"
fromBackup: ""
reclaimPolicy: Delete # 删除PVC时自动删除PV
allowVolumeExpansion: true # 允许扩容
EOF

# 2. 应用存储类配置
kubectl apply -f /opt/dolphinscheduler-deploy/dolphinscheduler-helm/dolphinscheduler-longhorn-sc.yaml

# 3. 验证存储类创建成功
kubectl get sc dolphinscheduler-longhorn

4.4 编写自定义配置文件(/opt 目录下)​

# 在 /opt 部署目录创建 custom-values.yaml(核心配置文件)
sudo tee /opt/dolphinscheduler-deploy/dolphinscheduler-helm/custom-values.yaml << 'EOF'
# ========== 基础全局配置 ==========
timezone: "Asia/Shanghai"
image:
registry: "harbor.local/dolphinscheduler"
tag: "3.1.9-new-driver"
pullPolicy: "IfNotPresent"
pullSecret: ""
master: dolphinscheduler-master
worker: dolphinscheduler-worker
api: dolphinscheduler-api
alert: dolphinscheduler-alert-server
tools: dolphinscheduler-tools

# ========== 禁用内置数据库,启用外部PostgreSQL ==========
postgresql:
enabled: false
mysql:
enabled: false

externalDatabase:
enabled: true
type: "postgresql"
host: "192.168.192.111"
port: 5432
username: "dolphinscheduler"
database: "dolphinscheduler"
password: "CHANGE_ME_STRONG_PASSWORD" # 请替换为实际数据库口令,勿提交真实密码
params: "characterEncoding=utf8"
driverClassName: "org.postgresql.Driver"

# ========== Zookeeper配置(10Gi存储) ==========
zookeeper:
enabled: true
persistence:
enabled: true
size: "10Gi" # 测试环境10Gi足够
storageClass: "dolphinscheduler-longhorn"

# ========== Master配置(5Gi存储) ==========
master:
replicas: 1
persistentVolumeClaim:
enabled: true
storageClassName: "dolphinscheduler-longhorn"
size: "5Gi" # Master存储需求小
resources:
limits:
cpu: 2000m
memory: 2Gi
requests:
cpu: 1000m
memory: 1Gi
livenessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
readinessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1

# ========== Worker配置(数据5Gi+日志10Gi) ==========
worker:
replicas: 2
persistentVolumeClaim:
enabled: true
dataPersistentVolume:
enabled: true
storageClassName: "dolphinscheduler-longhorn"
size: "5Gi" # 数据缓存5Gi
logsPersistentVolume:
enabled: true
storageClassName: "dolphinscheduler-longhorn"
size: "10Gi" # 日志10Gi(核心路径)
resources:
limits:
cpu: 2000m
memory: 4Gi
requests:
cpu: 1000m
memory: 2Gi
livenessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
readinessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
keda:
enabled: false

# ========== Alert配置(5Gi存储) ==========
alert:
replicas: 1
persistentVolumeClaim:
enabled: true
storageClassName: "dolphinscheduler-longhorn"
size: "5Gi" # Alert存储需求小
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
livenessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
readinessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1

# ========== API配置(5Gi存储+MetalLB) ==========
api:
replicas: 1
persistentVolumeClaim:
enabled: true
storageClassName: "dolphinscheduler-longhorn"
size: "5Gi" # API存储需求小
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
service:
type: LoadBalancer
loadBalancerIP: 10.2.10.154
port: 80
targetPort: 12345
env:
JAVA_OPTS: "-Xms512m -Xmx512m -Xmn256m"
livenessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
readinessProbe:
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1

# ========== 通用配置 ==========
conf:
common:
# 基本路径配置
data.basedir.path: /tmp/dolphinscheduler
# 存储类型配置
resource.storage.type: S3
resource.upload.path: /dolphinscheduler
resource.storage.upload.base.path: /dolphinscheduler
resource.aws.access.key.id: dolphinscheduler
resource.aws.secret.access.key: dolph@663.A998aX
resource.aws.region: cn-north-1
resource.aws.s3.bucket.name: dolphinscheduler
resource.aws.s3.endpoint: http://192.168.192.119:9000
resource.hdfs.fs.defaultFS: s3a://dolphinscheduler

# 数据源加密
datasource.encryption.enable: false
datasource.encryption.salt: '!@#$%^&*'
appId.collect: log

common:
configmap:
DOLPHINSCHEDULER_OPTS: ""
DATA_BASEDIR_PATH: "/tmp/dolphinscheduler"
RESOURCE_UPLOAD_PATH: "/dolphinscheduler"

# 软件路径配置
HADOOP_HOME: "/opt/soft/hadoop"
HADOOP_CONF_DIR: "/opt/soft/hadoop/etc/hadoop"
SPARK_HOME: "/opt/soft/spark"
PYTHON_HOME: "/usr/bin/python"
JAVA_HOME: "/opt/java/openjdk"
HIVE_HOME: "/opt/soft/hive"
FLINK_HOME: "/opt/soft/flink"
DATAX_HOME: "/opt/soft/datax"
SEATUNNEL_HOME: "/opt/soft/seatunnel"
# 共享存储(无实际Hadoop/Spark则设10Gi,有则设30Gi)
sharedStoragePersistence:
enabled: true
mountPath: "/opt/soft"
accessModes:
- "ReadWriteMany"
storageClassName: "dolphinscheduler-longhorn"
storage: "50Gi"
fsFileResourcePersistence:
enabled: false
accessModes:
- "ReadWriteMany"
storageClassName: "dolphinscheduler-longhorn"
storage: "20Gi"

# ========== 禁用Ingress ==========
ingress:
enabled: false
EOF

4.5 本地 Helm 安装(核心命令)​

确保当前处于 Helm Chart 目录,执行以下命令完成部署:

# 1. 添加bitnami仓库(官方源,适配DolphinScheduler 3.1.9)
helm repo add bitnami https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami

# 2. 更新仓库索引(确保能拉取最新依赖)
helm repo update

# 本地 Helm 安装 DolphinScheduler 3.1.9
helm dependency build && \
helm upgrade --install dolphinscheduler . \
--create-namespace \
--namespace dolphinscheduler \
-f /opt/dolphinscheduler-deploy/dolphinscheduler-helm/custom-values.yaml

# 查看 Helm 发布状态(显示 STATUS=deployed 即为成功)
helm list -n dolphinscheduler

# 验证Secret创建成功
kubectl describe secret dolphinscheduler-externaldb -n dolphinscheduler

# 验证Secret密码正确性(解码后应与DB_PASS一致)
kubectl get secret dolphinscheduler-externaldb -n dolphinscheduler -o jsonpath='{.data.database-password}' | base64 -d

五、部署验证(新增Secret验证步骤)​

5.1 核心资源验证​

# 1. 验证 Helm 自动创建的存储类(参数匹配 Longhorn)
kubectl get sc dolphinscheduler-longhorn

# 2. 验证 Service 固定 IP(EXTERNAL-IP 必须为 10.2.10.154)
kubectl get svc dolphinscheduler-api -n dolphinscheduler

# 3. 验证 PVC 绑定状态(所有 PVC 需为 Bound)
kubectl get pvc -n dolphinscheduler

# 4. 验证 Secret 已被正确挂载(关键!)
POD_NAME=$(kubectl get pod -n dolphinscheduler -l job-name=dolphinscheduler-db-init-job -o jsonpath='{.items[0].metadata.name}')
kubectl describe pod $POD_NAME -n dolphinscheduler | grep "SPRING_DATASOURCE_PASSWORD"

# 5. 监控 Pod 启动(所有 Pod 为 Running 即可)
kubectl get pods -n dolphinscheduler -w

5.2 访问 Web UI​

无需配置 hosts,直接通过固定 IP 访问:

  • 浏览器地址:http://10.2.10.154
  • 默认账号:admin
  • 默认密码:dolphinscheduler123

六、常用运维操作(修复路径错误)​

7.1 配置升级(修改 custom-values.yaml 后)​

# 进入正确的 Helm Chart 目录(修复原路径错误)
cd /opt/dolphinscheduler-deploy/apache-dolphinscheduler-3.1.9-src/deploy/kubernetes/dolphinscheduler

# 执行升级
helm upgrade dolphinscheduler . \
-n dolphinscheduler \
-f /opt/dolphinscheduler-deploy/custom-values.yaml

7.2 查看组件日志(排查问题)​

# 查看 API 组件日志(访问/数据库问题优先查看)
kubectl logs -f deployment/dolphinscheduler-api -n dolphinscheduler

# 查看 Master 组件日志
kubectl logs -f deployment/dolphinscheduler-master -n dolphinscheduler

# 查看 Worker 组件日志
kubectl logs -f deployment/dolphinscheduler-worker -n dolphinscheduler

# 查看 db-init-job 日志(数据库初始化问题优先查看)
kubectl logs $POD_NAME -n dolphinscheduler

7.3 重置管理员密码​

# 进入 API 容器
kubectl exec -it deployment/dolphinscheduler-api -n dolphinscheduler -- bash

kubectl exec -it deployment/dolphinscheduler-worker -n dolphinscheduler -- bash


# 重置密码(示例:重置为 new-password-123)
./bin/dolphinscheduler-admin.sh reset-admin-password new-password-123

7.4 更新数据库密码(Secret方式)​

# 1. 删除旧Secret
kubectl delete secret dolphinscheduler-externaldb -n dolphinscheduler

# 2. 创建新Secret(替换为新密码)
export NEW_DB_PASS="new-password-123"
kubectl create secret generic dolphinscheduler-externaldb \
-n dolphinscheduler \
--from-literal=database-password=${NEW_DB_PASS} \
--from-literal=host=${DB_HOST} \
--from-literal=port=5432 \
--from-literal=username=${DB_USER} \
--from-literal=database=${DB_NAME} \
--dry-run=client -o yaml | \
sed \
-e 's/^metadata:/metadata:\n labels:\n app.kubernetes.io\/managed-by: Helm\n app.kubernetes.io\/instance: dolphinscheduler\n annotations:\n meta.helm.sh\/release-name: dolphinscheduler\n meta.helm.sh\/release-namespace: dolphinscheduler/' \
| kubectl apply -f -

# 3. 重启所有组件使新密码生效
kubectl rollout restart deployment/dolphinscheduler-api -n dolphinscheduler
kubectl rollout restart deployment/dolphinscheduler-master -n dolphinscheduler
kubectl rollout restart deployment/dolphinscheduler-worker -n dolphinscheduler
kubectl rollout restart deployment/dolphinscheduler-alert -n dolphinscheduler

7.5 卸载 DolphinScheduler​

# 卸载 Helm 发布
helm uninstall dolphinscheduler -n dolphinscheduler

# 可选:删除Secret
kubectl delete secret dolphinscheduler-externaldb -n dolphinscheduler

# 删除命名空间
kubectl delete ns dolphinscheduler

# 可选:清理部署目录(如需彻底删除)
sudo rm -rf /opt/dolphinscheduler-deploy

dolphinscheduler 插件镜像​

sudo ctr -n k8s.io images  pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-alert-server:3.1.9
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-alert-server:3.1.9 docker.io/apache/dolphinscheduler-alert-server:3.1.9

sudo ctr -n k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-api:3.1.9
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-api:3.1.9 docker.io/apache/dolphinscheduler-api:3.1.9

sudo ctr -n k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-tools:3.1.9
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-tools:3.1.9 docker.io/apache/dolphinscheduler-tools:3.1.9

sudo ctr -n k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-master:3.1.9
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-master:3.1.9 docker.io/apache/dolphinscheduler-master:3.1.9

sudo ctr -n k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-worker:3.1.9
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/apache/dolphinscheduler-worker:3.1.9 docker.io/apache/dolphinscheduler-worker:3.1.9

sudo ctr -n k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/bitnami/zookeeper:3.6.2-debian-10-r185
sudo ctr -n k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/bitnami/zookeeper:3.6.2-debian-10-r185 docker.io/bitnami/zookeeper:3.6.2-debian-10-r185

hadoop:https://hadoop.apache.org/release/ spark:https://spark.apache.ac.cn/downloads.html hive:https://hive.apache.org/general/downloads/ flink:https://flink.apache.org/downloads/ datax:https://github.com/alibaba/DataX?tab=readme-ov-file seatunnel:https://seatunnel.apache.org/download

以 Spark 2.4.7 为例:

  1. 下载 Spark 2.4.7 发布的二进制包 spark-4.0.0-bin-hadoop3.tgz

  2. 确保 common.sharedStoragePersistence.enabled 开启

  3. 部署 dolphinscheduler (详见安装 dolphinscheduler)

  4. 复制 Spark 3.1.1 二进制包到 Docker 容器中

kubectl cp /opt/dolphinscheduler-deploy/deps-packages/spark-4.0.0-bin-hadoop3.tgz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10
kubectl cp /opt/dolphinscheduler-deploy/deps-packages/hadoop-3.4.0.tar.gz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10
kubectl cp /opt/dolphinscheduler-deploy/deps-packages/flink-2.2.0-bin-scala_2.12.tgz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10
kubectl cp /opt/dolphinscheduler-deploy/deps-packages/apache-seatunnel-2.3.10-bin.tar.gz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10
kubectl cp /opt/dolphinscheduler-deploy/deps-packages/apache-hive-4.2.0-bin.tar.gz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10
kubectl cp /opt/dolphinscheduler-deploy/deps-packages/datax.tar.gz dolphinscheduler-worker-0:/opt/soft -n dolphinscheduler --retries=10

因为存储卷 sharedStoragePersistence 被挂载到 /opt/soft, 因此 /opt/soft 中的所有文件都不会丢失

  1. 登录到容器并确保 SPARK_HOME 存在
kubectl exec -n dolphinscheduler -it dolphinscheduler-worker-0 -- bash

cd /opt/soft

tar zxf spark-4.0.0-bin-hadoop3.tgz
mv spark-4.0.0-bin-hadoop3 spark
#rm -f spark-4.0.0-bin-hadoop3.tgz
$SPARK_HOME/bin/spark-submit --version

tar zxf apache-hive-4.2.0-bin.tar.gz
mv apache-hive-4.2.0-bin hive
#rm -f apache-hive-4.2.0-bin.tar.gz

tar zxf flink-2.2.0-bin-scala_2.12.tgz
mv flink-2.2.0 flink
#rm -f flink-2.2.0-bin-scala_2.12.tgz

tar zxf apache-seatunnel-2.3.10-bin.tar.gz
mv apache-seatunnel-2.3.10 seatunnel
#rm -f apache-seatunnel-2.3.10-bin.tar.gz

tar zxf datax.tar.gz
#rm -f datax.tar.gz

nohup tar zxf hadoop-3.4.0.tar.gz > tar_hadoop.log 2>&1 &
mv hadoop-3.4.0 hadoop
#rm -f hadoop-3.4.0.tar.gz

如果一切执行正常,最后一条命令将会打印 Spark 版本信息

  1. 在一个 Shell 任务下验证 Spark
$SPARK_HOME/bin/spark-submit --class org.apache.spark.examples.SparkPi $SPARK_HOME/examples/jars/spark-examples_2.11-2.4.7.jar

使用SeaTunnel的注意事项:

1.需要进入容器中进行依赖下载:

# 集群配置,需要调整localhost:5801为集群地址:5801(可通过查看seatunnel集群的配置获取)
kubectl cp -n dolphinscheduler config/hazelcast-client.yaml dolphinscheduler-worker-0:/opt/soft/seatunnel/config/hazelcast-client.yaml

# plugins依赖配置(diy,用于拉取plugins目录下的依赖包配置)
kubectl cp -n dolphinscheduler config/plugin-dependencies.conf dolphinscheduler-worker-0:/opt/soft/seatunnel/config/plugin-dependencies.conf

# plugins下载脚本(diy,用于拉取plugins目录下的依赖包脚本)
kubectl cp -n dolphinscheduler bin/install-plugins-deps.sh dolphinscheduler-worker-0:/opt/soft/seatunnel/bin/install-plugins-deps.sh

# 安装maven配置,用于下载plugins下载脚本
kubectl cp -n dolphinscheduler .mvn dolphinscheduler-worker-0:/opt/soft/seatunnel/.mvn
kubectl cp -n dolphinscheduler mvnw dolphinscheduler-worker-0:/opt/soft/seatunnel/mvnw

# 其实建议是自己拉取完之后拷贝进容器的共享节点即可,则无需考虑plugins下载,否则还得清理下.m2的缓存
#kubectl cp -n dolphinscheduler connectors dolphinscheduler-worker-0:/opt/soft/seatunnel --retries=20
#kubectl cp -n dolphinscheduler plugins dolphinscheduler-worker-0:/opt/soft/seatunnel --retries=20

# 进入容器(master/worker都行)
kubectl exec -n dolphinscheduler -it dolphinscheduler-worker-0 -- bash

# 下载插件
nohup bash /opt/soft/seatunnel/bin/install-plugin.sh > seatunnel_install_plugin.log 2>&1 &
nohup bash /opt/soft/seatunnel/bin/install-plugins-deps.sh > seatunnel_install_plugins_deps.log 2>&1 &