Two-Node Deployment (Remote GPU) | Kamiwaza Docs

Kamiwaza 0.13.0 Documentation

This is documentation for Kamiwaza 0.13.0, which is no longer actively maintained. For the current GA release, see 1.0.1.

Version: 0.13.0

How it works

Requirements

Configuration

After installing Kamiwaza on the head node, add the following environment variables to env.sh and restart services.

Core Two-Node Variables

Variable Description Example
KAMIWAZA_PAIRED_NODE Hostname or IP of the worker node 10.77.0.5
KAMIWAZA_PAIRED_USER SSH username on the worker node jxstanford
KAMIWAZA_PAIRED_KEY SSH private key path (must be readable by kamiwaza user) /opt/kamiwaza/kamiwaza/runtime/ssh/id_ed25519_shared
KAMIWAZA_PAIRED_STRICT_KNOWN_HOSTS Enforce SSH host key checking true or false
KAMIWAZA_TWO_NODE_MODE Enable two-node deployment mode true

vLLM Ray Cluster Variables

These control the Ray cluster that vLLM uses for distributed tensor parallelism across nodes.

Variable Description Example
KAMIWAZA_VLLM_RAY_INTERNAL_PORT_BASE Base port for Ray internal services (avoid conflicts with system dashboards) 15000
KAMIWAZA_VLLM_RAY_MIN_WORKER_PORT Minimum port for Ray worker communication 5000
KAMIWAZA_VLLM_RAY_MAX_WORKER_PORT Maximum port for Ray worker communication 5199
KAMIWAZA_SPARK_NCCL_SOCKET_IFNAME Network interface for NCCL/Gloo cross-node communication enp1s0f0np0

SSH Key Setup

The kamiwaza system user runs the deployment process. The SSH key must be readable by this user:

sudo mkdir -p /opt/kamiwaza/kamiwaza/runtime/ssh

sudo cp ~/.ssh/id_ed25519_shared /opt/kamiwaza/kamiwaza/runtime/ssh/id_ed25519_shared

sudo chown kamiwaza:kamiwaza /opt/kamiwaza/kamiwaza/runtime/ssh/id_ed25519_shared

sudo chmod 600 /opt/kamiwaza/kamiwaza/runtime/ssh/id_ed25519_shared

Model Storage on the Worker Node

Both nodes must have the model storage directory at the same absolute path. On the worker node, manually create the directory and set ownership to match KAMIWAZA_PAIRED_USER — this is the SSH user that rsync runs as:

# On the worker node (e.g., spark-2):

sudo mkdir -p /opt/kamiwaza/models

sudo chown jxstanford:jxstanford /opt/kamiwaza/models

DGX Spark Example

The following env.sh snippet shows a complete two-node configuration for a DGX Spark pair (spark-1 as head, spark-2 as worker) connected via a 10.77.0.x direct link:

# Two-node pairing

export KAMIWAZA_PAIRED_NODE=10.77.0.5

export KAMIWAZA_PAIRED_USER=jxstanford

export KAMIWAZA_PAIRED_KEY=/opt/kamiwaza/kamiwaza/runtime/ssh/id_ed25519_shared

export KAMIWAZA_PAIRED_STRICT_KNOWN_HOSTS=true

export KAMIWAZA_TWO_NODE_MODE=true

# Ray cluster ports (avoid DGX dashboard conflict on port 11000)

export KAMIWAZA_VLLM_RAY_INTERNAL_PORT_BASE=15000

export KAMIWAZA_VLLM_RAY_MIN_WORKER_PORT=5000

export KAMIWAZA_VLLM_RAY_MAX_WORKER_PORT=5199

# Network interface for cross-node distributed communication

export KAMIWAZA_SPARK_NCCL_SOCKET_IFNAME=enp1s0f0np0

Deploying a Model with Tensor Parallelism

Step 1: Download the model

Download a safetensors model through the Kamiwaza UI. For example, search for Qwen/Qwen3-8B and initiate the download. The model files will be stored on the head node and automatically synced to the worker when you deploy.

Step 2: Create a deployment configuration

Create a model configuration with tensor_parallel_size=2 to split the model across both nodes. You can do this through the Kamiwaza SDK:

import os

os.environ["KAMIWAZA_VERIFY_SSL"] = "false"

from kamiwaza_sdk import KamiwazaClient

from kamiwaza_sdk.authentication import UserPasswordAuthenticator

from kamiwaza_sdk.schemas.models.model import CreateModelConfig

client = KamiwazaClient(base_url="https://localhost/api")

client.authenticator = UserPasswordAuthenticator(

"admin", "your-password", client._auth_service
)

# Find the model

models = client.models.list_models()

model = next(m for m in models if m.name == "Qwen3-8B")

# Create a TP=2 configuration

config = CreateModelConfig(

m_id=model.id,

name="2-Node TP2",

default=True,

description="Two-node tensor parallelism across DGX Spark pair",

config={

"tensor_parallel_size": 2,

"max_model_len": 4096,

"gpu_memory_utilization": 0.15,

},
)

result = client.models.create_model_config(config)

print(f"Created config: {result.name} (id: {result.id})")

Step 3: Deploy

Deploy through the UI by selecting the model and the "2-Node TP2" configuration, or via SDK:

deployment_id = client.serving.deploy_model(

model_id=model.id,

m_config_id=result.id,
)

print(f"Deployment: {deployment_id}")

Step 4: Test inference

curl -sk https://localhost/runtime/models/<deployment-id>/v1/chat/completions \

-H "Content-Type: application/json" \

-d '{

"model": "Qwen3-8B",

"messages": [{"role": "user", "content": "Hello!"}],

"max_tokens": 100

}'

Troubleshooting

Deployment stays in INITIALIZING

Check the vLLM container logs in the Kamiwaza UI under the deployment's "Logs" tab. Common causes:

SSH permission denied during deployment

The kamiwaza system user cannot read the SSH key. Ensure the key is at a path owned by kamiwaza:

ls -la /opt/kamiwaza/kamiwaza/runtime/ssh/

# Should show: -rw------- kamiwaza kamiwaza id_ed25519_shared

Stale lock files prevent startup

If kamiwaza was previously run as a different user, lock files in /tmp may have wrong ownership:

sudo rm -f /tmp/kamiwazad.starting /tmp/kamiwazad.lock /tmp/.kamiwaza_lock_*

Operational notes