# Documentation for Kamiwaza 1.0.0

This is documentation for Kamiwaza **1.0.0**, which is no longer actively maintained. For the current GA release, see [**1.0.1**](https://docs.kamiwaza.ai/).

### Overview

Kamiwaza uses S3-compatible object storage for two platform workflows:

- workroom context file persistence
- Skills Library package import and download

If object storage is not configured, file-backed user workflows can fail with errors such as:

```text
Workroom storage is not configured for Skills Library.
```

Use this guide when your deployment stores workroom content in **external AWS S3** instead of the default in-cluster object storage.

### When You Need This

Configure AWS S3 if your workroom content must live in an external AWS S3 bucket and you want the following to work reliably:

- context file upload and persistence
- Skills Library package import
- Skills Library package download

### Prerequisites

Before you start, make sure you have:

- an AWS S3 bucket
- an IAM user or role with access to that bucket
- `kubectl` access to the target cluster
- a values override file for the deployment configuration

The IAM identity should be allowed to perform at least:

- `s3:ListBucket`
- `s3:GetObject`
- `s3:PutObject`
- `s3:DeleteObject`

### Choose a Configuration Surface

There are two ways to configure external S3 workroom storage, depending on how you deploy:

| Deployment                                | Configure via                                                 |
| ----------------------------------------- | ------------------------------------------------------------ |
| `deploy` repo Helmfile install (v1.0 storage platform) | `storage.workrooms` in `deploy/cluster/values/storage-overrides.yaml` — **preferred** |
| Direct `core` chart values (no umbrella chart / no Helmfile) | `context.objectStorage` in your core values file             |

### Configuration Model

Under the hood, the `core` chart exposes a single object-storage configuration block. The storage platform fills it in from `storage.workrooms.*`; direct chart consumers set it themselves:

```yaml
context:

objectStorage:

enabled: true

defaultBucket: ""

defaultRegion: ""

defaultPrefix: "context/raw"

endpointUrl: ""

credentialsSecretRef:

name: ""

accessKeyIdKey: "access_key_id"

secretAccessKeyKey: "secret_access_key"

sessionTokenKey: "session_token"
```

### Option 1: Static AWS Access Keys

Use this option when your cluster does not already provide AWS credentials to the pods.

#### 1. Create a Kubernetes Secret

Core pods wait on this Secret, so create it before deploying:

```bash
kubectl create namespace kamiwaza --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic core-s3 \

-n kamiwaza \

--from-literal=access_key_id="<aws-access-key-id>" \

--from-literal=secret_access_key="<aws-secret-access-key>"
```

If you are using temporary session credentials:

```bash
kubectl create secret generic core-s3 \

-n kamiwaza \

--from-literal=access_key_id="<aws-access-key-id>" \

--from-literal=secret_access_key="<aws-secret-access-key>" \

--from-literal=session_token="<aws-session-token>"
```

#### 2a. Helmfile installs: configure `storage.workrooms` (preferred)

Add this to `deploy/cluster/values/storage-overrides.yaml`:

```yaml
storage:

workrooms:

backend: s3

s3:

region: "us-west-2"

bucket: "my-kamiwaza-artifacts"

prefix: "context/raw"

endpoint: ""

existingSecret: "core-s3"
```

### Option 2: Ambient AWS Credentials

Use this option if your cluster already provides AWS credentials to pods through IAM roles or another AWS-native mechanism. In this case, do not create a Secret.

#### Helmfile installs

Add this to `deploy/cluster/values/storage-overrides.yaml`:

```yaml
storage:

workrooms:

backend: s3

s3:

region: "us-west-2"

bucket: "my-kamiwaza-artifacts"

prefix: "context/raw"

endpoint: ""

existingSecret: ""
```

### Restart Existing Deployments

If the cluster is already running, restart the core workloads after applying the updated config:

```bash
kubectl rollout restart deployment/core-scheduler -n kamiwaza

kubectl delete pod -n kamiwaza -l ray.io/node-type=head

kubectl delete pod -n kamiwaza -l ray.io/node-type=worker
```

### Verify the Configuration

#### Check the ConfigMap

Verify that the non-secret S3 settings are present:

```bash
kubectl get configmap core-config -n kamiwaza -o yaml | grep CONTEXT_SERVICE_S3
```

### Troubleshooting

#### Error: secret "core-s3" not found (CreateContainerConfigError)

Core pods (`core-scheduler`, `core-raycluster-head`) are stuck in `CreateContainerConfigError`.  This means core was configured to read S3 credentials from a Secret that does not exist in the `kamiwaza` namespace.

**Fix:**
1. If you intended the default RGW storage: remove the `core.context.objectStorage` block from your overrides and re-apply the release.
2. If you intended external S3: create the Secret (Option 1, step 1).

#### Security Notes

- Do not store AWS secrets in workroom attributes or user-editable metadata.
- Prefer Kubernetes Secrets or ambient AWS identity over plain-text credentials in values files.
- Leave `endpointUrl` empty for AWS S3. Set it only when using a non-AWS S3-compatible service.