System Requirements | Kamiwaza Docs

Hardware Requirements

CPU

• Minimum Cores: 8+ cores
• Recommended Cores: 16+ cores for CPU-based inference workloads
• Architecture:
  • Linux: x64/amd64 (64-bit)
  • macOS: ARM64 (Apple Silicon) only

Memory

System RAM

GPU Memory (vRAM)

• GPU Inference: 16GB+ vRAM required
• Recommended: 32GB+ vRAM for optimal GPU inference performance

GPU (Optional)

Kamiwaza supports multiple GPU and accelerator platforms:

Discrete GPUs:

• NVIDIA GPUs with compute capability 7.0+ (Linux)
• AMD GPUs via ROCm (Linux) — see Software Dependencies

Unified Memory Systems:

• NVIDIA DGX Spark - GB10 Grace Blackwell, 128GB unified memory
• AMD Ryzen AI Max+ 395 - "Strix Halo" platform, up to 128GB unified memory
• Apple Silicon M-series - Unified memory architecture (macOS only)

Storage

Storage requirements are the same across all platforms.

Storage Performance

• Required: SSD (Solid State Drive)
• Preferred: NVMe SSD for optimal performance
• Minimum: SATA SSD
• Note: Model weights can be on a separate HDD but load times will increase significantly

Storage Capacity

• Minimum: 100GB free disk space
• Recommended: 200GB+ free disk space
• Additional space for /opt/kamiwaza persistence

Capacity Planning

Storage Performance Requirements

Local Storage (Single Node):

• Minimum: SATA SSD (500 MB/s sequential read)
• Recommended: NVMe SSD (2000+ MB/s sequential read)
• Note: HDD is only recommended for non-dynamic model loads and low KV cache usage - model load times can be very long (15+ minutes); models are in memory after load

Performance Targets:

• Sequential Read: 2000+ MB/s (model loading)
• Sequential Write: 1000+ MB/s (model downloads, checkpoints)
• 4K Random Read IOPS: 50,000+ (database, concurrent access)
• 4K Random Write IOPS: 20,000+ (database writes, logs)

Why It Matters:

• 7B model (14GB): Loads in ~7 seconds on NVMe vs ~28 seconds on SATA SSD
• Concurrent model loads across Ray workers stress random read performance
• Database query performance directly tied to IOPS

Supported Operating Systems

Linux

• Ubuntu: 24.04 and 22.04 LTS (x64/amd64 architecture only) — online install
• Red Hat Enterprise Linux (RHEL) 9 and compatibles — online or offline install

macOS

• macOS 15.0 (Sequoia) or later, Apple Silicon (ARM64) only — online install
• Single-node deployments only

All platforms install via the Keygen-based installer and require a Kamiwaza Prod license key.

Software Dependencies

What You Provide

The Kamiwaza installer provisions the container runtime, local Kubernetes cluster, and platform dependencies for you. You only need:

GPU Drivers (Required for GPU Inference)

Install the appropriate driver for your GPU hardware:

NVIDIA GPUs:

AMD GPUs (ROCm):

Auto-Installed by Kamiwaza

The Kamiwaza installer automatically installs and configures the following — no manual installation required:

• The container runtime and local Kubernetes cluster
• The platform database and other backing services
• Python, Node.js, uv, and other platform-specific dependencies

Verifying System Requirements

Use these commands to verify your system meets the requirements before installation.

NVIDIA GPU (if applicable)

# Check NVIDIA driver

nvidia-smi

# Expected: Driver version 550 or later

# Should display GPU name, driver version, and CUDA version

# Check NVIDIA Container Toolkit

nvidia-ctk --version

# Expected: Any version indicates toolkit is installed

# Example output: NVIDIA Container Toolkit CLI version 1.17.3

AMD ROCm (if applicable)

# Check ROCm installation

rocm-smi

# Expected: Should display AMD GPU information

# Look for: GPU temperature, utilization, memory usage

# Check ROCm version

cat /opt/rocm/.info/version

# Expected: 7.1.1 or later (7.10.0+ for Strix Halo gfx1151)

# Verify GPU device access

ls -la /dev/kfd /dev/dri

# Expected: Both devices should exist and be accessible

System Resources

# Check available memory

free -h

# Expected: At least 16GB total (32GB+ recommended)

# Look for "Mem:" row, "total" column

# Check CPU cores

nproc

# Expected: 8 or more cores

# Check available disk space

df -h /

# Expected: At least 100GB free (200GB+ recommended)

Hardware Recommendation Tiers

Kamiwaza is a distributed AI platform built on Ray that supports both CPU-only and GPU-accelerated inference. Hardware requirements vary significantly based on:

• Model size: From 0.6B to 70B+ parameters
• Deployment scale: Single-node development vs multi-node production
• Inference engine: LlamaCpp (CPU/GPU), VLLM (GPU), MLX (Apple Silicon)
• Workload type: Interactive chat, batch processing, RAG pipelines

GPU Memory Requirements by Model Size

The table below provides real-world GPU memory requirement estimates for representative models at different scales. These estimates assume FP8 and include overhead for context windows and batch processing.

Key Considerations:

• Minimum vRAM: FP8 weights + 1-batch allocation at your target max context
• Headroom: For longer contexts, larger batch sizes, and concurrency, budget additional vRAM beyond minimums
• Vision Workloads: Image/video processing adds overhead; budget 20-30% more for vision-language models
• Tensor Parallelism: Distributing large models (120B+) across multiple GPUs requires high-bandwidth interconnects (NVLink 3.0+)

Tier 1: Development & Small Models

Use Case: Local development, testing, small to medium model deployment (up to 13B parameters)

Hardware Specifications:

• CPU: 8-16 cores / 16-32 threads
• RAM: 32GB (16GB minimum for development only)
• Storage: 200GB NVMe SSD (100GB minimum)
• GPU: Optional - Single GPU with 16-24GB VRAM
  • NVIDIA RTX 4090 (24GB)
  • NVIDIA RTX 4080 (16GB)
  • NVIDIA T4 (16GB)
• Network: 1-10 Gbps

Workload Capacity:

• Low-volume workloads: 1-10 concurrent requests (supports dozens of interactive users)
• Development, testing, and proof-of-concept deployments
• Light production workloads

Tier 2: Production - Medium to Large Models

Use Case: Production deployment of medium to large models (13B-70B parameters), high throughput

Hardware Specifications:

• CPU: 32 cores / 64 threads
• RAM: 128-256GB system RAM
• Storage: 1-2TB NVMe SSD
• GPU: 1-4 GPUs with 40GB+ VRAM each
  • 1-4x NVIDIA B200 (192GB HBM3e)
  • 1-4x NVIDIA H200 (141GB HBM3e)
  • 1-4x NVIDIA RTX 6000 Pro Blackwell (48GB)
  • 1-2x NVIDIA H100 (80GB)
  • 1-4x NVIDIA A100 (40GB or 80GB)
  • 1-2x NVIDIA L40S (48GB)
  • 2-4x NVIDIA A10G (24GB) for tensor parallelism
• Network: 25-40 Gbps

Workload Capacity:

• Medium-scale production: 100s to 1,000+ concurrent requests (supports thousands of interactive users)
• Example: Per-GPU batch size of 32 across 8 GPUs = 256 concurrent requests; batch size of 128 = 1,024 requests
• Production chat applications
• Complex RAG pipelines with embedding generation
• Batch inference

Tier 3: Enterprise Multi-Node Cluster

Use Case: Enterprise deployment with multiple models, high availability, horizontal scaling, 99.9%+ SLA

Cluster Architecture: Head Node (Control Plane):

• CPU: 16 cores / 32 threads
• RAM: 64GB
• Storage: 500GB NVMe SSD
• GPU: Same class as worker nodes (homogeneous cluster recommended)
• Role: Ray head, API gateway, scheduling, monitoring (head performs minimal extra work; Ray backend load is distributed across nodes)

Worker Nodes (3+ nodes for HA):

• CPU: 32-64 cores / 64-128 threads per node
• RAM: 256-512GB per node
• Storage: 2TB NVMe SSD per node (local cache)
• GPU: 4-8 GPUs per node (same class as head node)
• Network: 40-100 Gbps (InfiniBand for HPC workloads)

Note: For production clusters, avoid non-homogeneous hardware (e.g., GPU-less head nodes). Each node participates in data plane duties (ingress gateway, HTTP proxying, etc.), so matching GPU capabilities simplifies scheduling and maximizes throughput.

Shared Storage:

• High-performance NAS or distributed filesystem (Lustre, CephFS)
• 10TB+ capacity, NVMe-backed
• 10+ GB/s aggregate sequential throughput
• Low-latency access (< 5ms) from all nodes

Workload Capacity:

• Multiple models deployed simultaneously
• High-scale production: 1,000–10,000+ concurrent requests (supports tens of thousands of interactive users)
• Batch sizes scale with GPU count and model size; smaller requests enable higher throughput per GPU
• High availability with automatic failover
• Horizontal auto-scaling based on load
• Production SLAs (99.9% uptime)

Cloud Provider Instance Mapping

AWS EC2 Instance Types

Google Cloud Platform (GCP) Instance Types

Microsoft Azure Instance Types

Network Configuration

Network Bandwidth Requirements

Single Node Deployment

Network Bandwidth:

• Minimum: 1 Gbps (for model downloads, API traffic)
• Recommended: 10 Gbps (for high-throughput inference)

Multi-Node Cluster

Inter-Node Network:

• Minimum: 10 Gbps Ethernet
• Recommended: 25-40 Gbps Ethernet or InfiniBand
• Latency: < 1ms between nodes (same datacenter/availability zone)

Network Ports

Linux/macOS

• 443/tcp: HTTPS primary access
• 51100-51199/tcp: Deployment ports for model instances

Required Kernel Modules (Linux)

• overlay
• br_netfilter

System Network Parameters (Linux)

# Required sysctl settings for container networking

net.bridge.bridge-nf-call-iptables  = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.ipv4.ip_forward                 = 1

Directory Structure

The installer creates the Kamiwaza directories on the host:

/etc/kamiwaza/

├── config/

└── ssl/      # Cluster certificates

/opt/kamiwaza/

├── cluster/    # Cluster values and overrides

├── logs/

├── prereqs/    # Installer prerequisites (offline installs)

├── scripts/    # Installer and management scripts

└── runtime/    # Runtime files

Special Considerations

Apple Silicon (M-Series)

MLX Engine Support:

• Kamiwaza supports Apple Silicon via the MLX inference engine
• Unified memory architecture (shared CPU/GPU RAM)
• Excellent performance for models up to 13B parameters; reasonable performance for larger models when context is appropriately restricted and RAM is available.

NVIDIA DGX Spark

The NVIDIA DGX Spark is a compact AI workstation powered by the GB10 Grace Blackwell Superchip:

• CPU: 20-core ARM (10x Cortex-X925 + 10x Cortex-A725)
• GPU: Blackwell architecture with 6,144 CUDA cores
• Memory: 128GB LPDDR5x unified memory (273 GB/s bandwidth)
• AI Compute: Up to 1 PFLOP FP4 AI performance
• Storage: 4TB NVMe SSD
• Networking: Dual QSFP ports (up to 200 Gbps aggregate)

AMD Ryzen AI Max+ 395 "Strix Halo"

AMD's Strix Halo platform provides powerful AI inference in a compact form factor:

• CPU: 16-core Zen 5 (up to 5.1 GHz), 80MB cache
• GPU: Radeon 8060S iGPU (40 CUs, RDNA 3.5 architecture)
• NPU: 50 TOPS XDNA 2 neural engine
• Memory: Up to 128GB LPDDR5x unified memory (up to 112GB GPU-allocatable)
• AI Performance: 126 TOPS total
• TDP: 55W (highly power efficient)

Shared Storage (Multi-Node Clusters)

Network Filesystem Requirements:

• Protocol: NFSv4, Lustre, CephFS, or S3-compatible object storage
• Network Bandwidth: 10 Gbps minimum, 40+ Gbps for production
• Network Latency: < 5ms between nodes and storage
• Sequential Throughput: 5+ GB/s aggregate (10+ GB/s for large clusters)

Storage Configuration

• Primary mountpoint for persistent storage (/opt/kamiwaza)
• Scratch/temporary storage (auto-configured)
• Shared storage for multi-node clusters (see Shared Storage Options above)

Version Compatibility

• NVIDIA Driver: 550-server or later
• ETCD: 3.5 or later

Important Notes

• System Impact: Network and kernel configurations can affect other services
• Security: Certificate generation and management for cluster communications
• GPU Support: Available on Linux — NVIDIA GPUs (CUDA) and AMD GPUs (ROCm)
• Storage: Persistent and scratch storage are configured on the install host
• Network: Requires the network ports listed above for platform and model access
• License: A Kamiwaza Prod license key is required for all installs.