Deploying Apache DolphinScheduler in a Single-Node Pseudo-Cluster Environment

System Account and SSH Automation Setup

Establishing a dedicated service account with passwordless SSH access streamlines the deployment workflow for all cluster components.

# Provision the deployment user and assign passwordless sudo rights
sudo useradd -m -s /bin/bash ds_user
echo "ds_user:SecurePass123" | sudo chpasswd
sudo tee -a /etc/sudoers.d/ds_user << 'EOF'
ds_user  ALL=(ALL)  NOPASSWD: ALL
EOF
sudo sed -i 's/^Defaults.*requiretty.*/#Defaults    requiretty/' /etc/sudoers

# Switch context to generate and authorize SSH credentials
sudo -u ds_user -i bash << 'SCRIPT'
mkdir -p ~/.ssh && chmod 700 ~/.ssh
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N "" -q
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
SCRIPT

Core Runtime Dependencies

The platform requires specific versions of the Java runtime, relational database, and coordination service before initialization.

Java Environment Configuration Extract the JDK archive and configure global environment variables via a profile scrript to ensure persistence across sessions.

export JDK_ARCHIVE="jdk-8u401-linux-x64.tar.gz"
sudo mkdir -p /opt/module/java && sudo tar -xzf /tmp/$JDK_ARCHIVE -C /opt/module/java --strip-components=1
cat << 'JAVA_ENV' | sudo tee /etc/profile.d/custom_java.sh
export JAVA_HOME=/opt/module/java
export PATH=\$PATH:\$JAVA_HOME/bin
JAVA_ENV
source /etc/profile.d/custom_java.sh

ZooKeeper Coordination Layer A stable coordination service is mandatory for cluster metadata synchronization. Version 3.8+ is recommended to avoid legacy compatibility issues.

export ZK_RELEASE="apache-zookeeper-3.8.1-bin"
sudo mkdir -p /opt/soft/zk && sudo tar -xzf /tmp/$ZK_RELEASE.tar.gz -C /opt/soft/zk

# Apply configuration via targeted replacements
ZK_CFG="/opt/soft/zk/$ZK_RELEASE/conf/zoo.cfg"
sudo cp "$ZK_CFG.sample" "$ZK_CFG"
sudo sed -i 's|^dataDir=.*|dataDir=/tmp/zk-data|' "$ZK_CFG"
sudo sed -i 's|^clientPort=.*|clientPort=12181|' "$ZK_CFG"
sudo mkdir -p /tmp/zk-data && sudo chown -R ds_user:ds_user /tmp/zk-data

# Initialize the service
/opt/soft/zk/$ZK_RELEASE/bin/zkServer.sh start
/opt/soft/zk/$ZK_RELEASE/bin/zkServer.sh status

Relational Database Backend MySQL serves as the primary metadata store. Execute the following to prepare the instance, enforce a relaxed password policy for testing, and provision the required schema and credentials.

# Adjust validation constraints
mysql -u root -p'root' -e "SET GLOBAL validate_password_length=4;"
mysql -u root -p'root' -e "SET GLOBAL validate_password_policy=LOW;"

# Clean slate, schema creation, and access grants
mysql -u root -p'root' -e "DROP DATABASE IF EXISTS dolphinscheduler_db;"
mysql -u root -p'root' -e "CREATE DATABASE dolphinscheduler_db DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;"
mysql -u root -p'root' -e "CREATE USER 'ds_operator'@'%' IDENTIFIED BY 'ds_pass';"
mysql -u root -p'root' -e "GRANT ALL PRIVILEGES ON dolphinscheduler_db.* TO 'ds_operator'@'%'; FLUSH PRIVILEGES;"

Application Configuration and Driver Integration

Retrieve the binary distribution and modify the environment definitions to align with your infrastructure.

export DS_PACKAGE="apache-dolphinscheduler-3.1.4-bin"
mkdir -p /opt/soft/ds && tar -xzf /tmp/$DS_PACKAGE.tar.gz -C /opt/soft/ds
cd /opt/soft/ds/$DS_PACKAGE/bin/env

Deployement Environment (install_env.sh) Define the target topology. For a pseudo-cluster, map all roles to the local host.

ips="127.0.0.1"
masters="127.0.0.1"
workers="127.0.0.1:default"
alertServer="127.0.0.1"
apiServers="127.0.0.1"
installPath="/opt/soft/ds/dolphinscheduler_cluster"
deployUser="ds_user"
zkRoot="/ds/registry"

Runtime Configuration (dolphinscheduler_env.sh) Point the framework to the Java runtime, database endpoint, and coordination service.

export JAVA_HOME=${JAVA_HOME:-/opt/module/java}
export DATABASE="mysql"
export SPRING_PROFILES_ACTIVE="${DATABASE}"
export SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler_db?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
export SPRING_DATASOURCE_USERNAME="ds_operator"
export SPRING_DATASOURCE_PASSWORD="ds_pass"
export REGISTRY_TYPE="zookeeper"
export REGISTRY_ZOOKEEPER_CONNECT_STRING="127.0.0.1:12181"

JDBC and CLI Library Placement The framework requires explicit placement of the MySQL connector and Apache Commons CLI libraries within each service module.

export MVN_REPO="https://repo1.maven.org/maven2"
export JDBC_URL="$MVN_REPO/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar"
export CLI_URL="$MVN_REPO/commons-cli/commons-cli/1.5.0/commons-cli-1.5.0.jar"

wget $JDBC_URL -P /tmp/libs && wget $CLI_URL -P /tmp/libs

for svc in worker-server api-server alert-server master-server tools; do
    mkdir -p /opt/soft/ds/$DS_PACKAGE/$svc/libs
    cp /tmp/libs/mysql-connector-java-8.0.16.jar /opt/soft/ds/$DS_PACKAGE/$svc/libs/
    [ "$svc" != "standalone-server" ] && cp /tmp/libs/commons-cli-1.5.0.jar /opt/soft/ds/$DS_PACKAGE/$svc/libs/
done
cp /tmp/libs/mysql-connector-java-8.0.16.jar /opt/soft/ds/$DS_PACKAGE/standalone-server/libs/standalone-server

Initialization and Cluster Deployment

Transfer ownership of the extracted binaries to the service account before executing the schema upgrade and installation routines.

cd /opt/soft/ds
sudo chown -R ds_user:ds_user $DS_PACKAGE

# Execute as the designated service user
sudo su - ds_user -c "
  cd /opt/soft/ds/$DS_PACKAGE && \
  bash tools/bin/upgrade-schema.sh && \
  bash ./bin/install.sh
"

Service Lifecycle Management and Access

Once the installation script completes, the platform exposes its REST API and web interface on port 12345. Default administrative credentials are admin and dolphinscheduler123.

# Navigate to the installation directory
cd /opt/soft/ds/dolphinscheduler_cluster

# Full cluster operations
./bin/start-all.sh
./bin/stop-all.sh

# Granular component control
./bin/dolphinscheduler-daemon.sh start master-server
./bin/dolphinscheduler-daemon.sh stop master-server
./bin/dolphinscheduler-daemon.sh start worker-server
./bin/dolphinscheduler-daemon.sh stop worker-server
./bin/dolphinscheduler-daemon.sh start api-server
./bin/dolphinscheduler-daemon.sh stop api-server
./bin/dolphinscheduler-daemon.sh start alert-server
./bin/dolphinscheduler-daemon.sh stop alert-server

Access the dashboard via http://localhost:12345/dolphinscheduler/ui.

Tags: Apache DolphinScheduler Pseudo-Cluster Deployment System Administration Big Data Orchestration Linux Server Setup

Posted on Fri, 17 Jul 2026 16:36:28 +0000 by XtacY

Freaks City