Documentation for Kamiwaza 1.0.1

This is documentation for Kamiwaza 1.0.1, which is no longer actively maintained. For the current GA release, see 1.0.1.

Version: 1.0.1

Overview

The following walkthrough is based on the Kamiwaza 1.0.0 user interface. We'll walk through navigating the Kamiwaza admin console to search for a model, download model files, and then deploy and use an inference endpoint.

Step 1: Find and click the Models menu in the sidebar

Step 2: Under Model Hub Search, type keywords for your desired model

In this example, I'm looking for a GGUF of Llama3.1 8B Instruct, so I type a few of its keywords to narrow results.

Clicking the Search button will show the results, like this:

Step 3: Download models files from chosen model

From the results, let's choose the Bartowski GGUF. Clicking the Download button for bartowski/Meta-Llama-3.1-8B-Instruct-GGUF results in the following:

In this example, we are downloading GGUF models for a llamacpp inference engine deployment. Unlike normal Hugging Face models (safetensors), we only need to download one or a couple files - just the specific quantized model that we need.

For this example, we will uncheck everything and choose only the Q8_0 variant. Click **Select/Unselect