> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argmaxinc/WhisperKit/llms.txt
> Use this file to discover all available pages before exploring further.

# Local Server & Clients

> Set up a local transcription server with client examples

## Overview

WhisperKit includes a local server that implements the OpenAI Audio API, allowing you to use existing OpenAI SDK clients or generate new ones. The server supports transcription and translation with **output streaming** capabilities.

<Note>
  For **full-duplex real-time streaming**, check out [WhisperKit Pro Local Server](https://www.argmaxinc.com/blog/argmax-local-server) which provides live audio streaming.
</Note>

## Building the Server

First, build the CLI with server support:

```bash theme={null}
# Clone the repository if you haven't already
git clone https://github.com/argmaxinc/whisperkit.git
cd whisperkit

# Build with server support
make build-local-server

# Or manually with the build flag
BUILD_ALL=1 swift build --product whisperkit-cli
```

## Starting the Server

### Default Configuration

```bash theme={null}
# Start server on default port (50060)
BUILD_ALL=1 swift run whisperkit-cli serve
```

The server will:

* Listen on `localhost:50060`
* Use the default `tiny` model
* Download the model if not already available

### Custom Configuration

```bash theme={null}
# Custom host and port
BUILD_ALL=1 swift run whisperkit-cli serve \
    --host 0.0.0.0 \
    --port 8080

# With specific model
BUILD_ALL=1 swift run whisperkit-cli serve \
    --model base \
    --verbose

# See all options
BUILD_ALL=1 swift run whisperkit-cli serve --help
```

## API Endpoints

The server exposes two main endpoints:

* **POST** `/v1/audio/transcriptions` - Transcribe audio to text
* **POST** `/v1/audio/translations` - Translate audio to English

## Supported Parameters

| Parameter                   | Description                             | Default        |
| --------------------------- | --------------------------------------- | -------------- |
| `file`                      | Audio file (wav, mp3, m4a, flac)        | Required       |
| `model`                     | Model identifier                        | Server default |
| `language`                  | Source language code (e.g., "en", "es") | Auto-detect    |
| `prompt`                    | Text to guide transcription             | None           |
| `response_format`           | Output format: `json`, `verbose_json`   | `verbose_json` |
| `temperature`               | Sampling temperature (0.0-1.0)          | 0.0            |
| `timestamp_granularities[]` | Timing detail: `word`, `segment`        | `segment`      |
| `stream`                    | Enable streaming output                 | `false`        |
| `include[]`                 | Include additional data: `logprobs`     | None           |

## Python Client

Use the OpenAI Python SDK to connect to the local server:

### Installation

```bash theme={null}
cd Examples/ServeCLIClient/Python
uv sync  # or: pip install openai
```

### Quick Example

```python theme={null}
from openai import OpenAI

# Connect to local server
client = OpenAI(base_url="http://localhost:50060/v1")

# Transcribe audio file
with open("audio.wav", "rb") as audio_file:
    result = client.audio.transcriptions.create(
        file=audio_file,
        model="tiny"  # Model parameter is required
    )

print(result.text)
```

### Transcription with Options

```python theme={null}
from openai import OpenAI

client = OpenAI(base_url="http://localhost:50060/v1")

with open("audio.wav", "rb") as audio_file:
    result = client.audio.transcriptions.create(
        file=audio_file,
        model="tiny",
        language="en",
        response_format="verbose_json",
        timestamp_granularities=["word", "segment"]
    )

# Access detailed information
print(f"Language: {result.language}")
print(f"Duration: {result.duration}s")
print(f"Text: {result.text}")

# Word-level timestamps
for word in result.words:
    print(f"{word.start:.2f}s - {word.end:.2f}s: {word.word}")

# Segment-level timestamps
for segment in result.segments:
    print(f"[{segment.start:.2f}s]: {segment.text}")
```

### Translation

```python theme={null}
# Translate audio to English
with open("spanish_audio.wav", "rb") as audio_file:
    result = client.audio.translations.create(
        file=audio_file,
        model="tiny"
    )

print(f"Translation: {result.text}")
```

### Streaming Transcription

```python theme={null}
import requests
import json

# Use requests library for streaming
url = "http://localhost:50060/v1/audio/transcriptions"

with open("audio.wav", "rb") as audio_file:
    files = {"file": audio_file}
    data = {
        "model": "tiny",
        "stream": "true",
        "response_format": "verbose_json"
    }
    
    response = requests.post(
        url,
        files=files,
        data=data,
        headers={"Accept": "text/event-stream"},
        stream=True
    )
    
    # Process Server-Sent Events
    for line in response.iter_lines():
        if line:
            line_str = line.decode('utf-8')
            if line_str.startswith('data: '):
                data_str = line_str[6:]  # Remove 'data: ' prefix
                try:
                    event = json.loads(data_str)
                    if event.get('type') == 'transcript.text.delta':
                        print(event['delta'], end='', flush=True)
                    elif event.get('type') == 'transcript.text.done':
                        print(f"\nFinal: {event['text']}")
                except json.JSONDecodeError:
                    pass
```

### Command Line Usage

```bash theme={null}
# Using the provided Python client
cd Examples/ServeCLIClient/Python

# Transcribe
python whisperkit_client.py transcribe \
    --file audio.wav \
    --language en

# Translate
python whisperkit_client.py translate \
    --file audio.wav

# With streaming
python whisperkit_client.py transcribe \
    --file audio.wav \
    --stream
```

## Swift Client

The Swift client is generated from the OpenAPI specification:

### Installation

```bash theme={null}
cd Examples/ServeCLIClient/Swift
swift build
```

### Command Line Usage

```bash theme={null}
# Transcribe
swift run whisperkit-client transcribe audio.wav --language en

# Translate
swift run whisperkit-client translate audio.wav

# With word-level timestamps
swift run whisperkit-client transcribe audio.wav \
    --timestamp-granularities word,segment \
    --response-format verbose_json

# Streaming
swift run whisperkit-client transcribe audio.wav --stream
```

### Programmatic Usage

```swift theme={null}
import Foundation
import WhisperKitSwiftClient

// Initialize client
let client = WhisperKitClient(
    serverURL: "http://localhost:50060/v1"
)

// Transcribe audio
try await client.transcribeAudio(
    filePath: "audio.wav",
    language: "en",
    model: "tiny",
    responseFormat: "verbose_json",
    timestampGranularities: "word,segment",
    stream: false
)

// Translate audio
try await client.translateAudio(
    filePath: "audio.wav",
    language: "es",
    model: "tiny",
    responseFormat: "verbose_json"
)
```

## cURL Client

Use the provided shell scripts or raw cURL commands:

### Using Shell Scripts

```bash theme={null}
cd Examples/ServeCLIClient/Curl
chmod +x *.sh

# Transcribe
./transcribe.sh audio.wav --language en

# Translate
./translate.sh audio.wav --language es

# With all options
./transcribe.sh audio.wav \
    --model base \
    --language en \
    --timestamp-granularities word,segment \
    --stream true

# Run comprehensive test suite
./test.sh
```

### Raw cURL Commands

#### Basic Transcription

```bash theme={null}
curl -X POST http://localhost:50060/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model="tiny" \
  -F response_format="verbose_json"
```

#### With Word Timestamps

```bash theme={null}
curl -X POST http://localhost:50060/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model="tiny" \
  -F response_format="verbose_json" \
  -F timestamp_granularities[]="word,segment"
```

#### Streaming Output

```bash theme={null}
curl -N -X POST http://localhost:50060/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model="tiny" \
  -F stream="true" \
  -H "Accept: text/event-stream"
```

#### Translation

```bash theme={null}
curl -X POST http://localhost:50060/v1/audio/translations \
  -F file=@audio.wav \
  -F model="tiny" \
  -F response_format="verbose_json"
```

#### With Log Probabilities

```bash theme={null}
curl -X POST http://localhost:50060/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model="tiny" \
  -F response_format="json" \
  -F "include[]=logprobs"
```

## JavaScript/TypeScript Client

### Installation

```bash theme={null}
npm install openai
# or
yarn add openai
```

### Usage

```typescript theme={null}
import OpenAI from 'openai';
import fs from 'fs';

const client = new OpenAI({
  baseURL: 'http://localhost:50060/v1',
  apiKey: 'dummy-key'  // Not used by local server
});

// Transcribe
const transcription = await client.audio.transcriptions.create({
  file: fs.createReadStream('audio.wav'),
  model: 'tiny',
  language: 'en',
  response_format: 'verbose_json',
  timestamp_granularities: ['word', 'segment']
});

console.log(transcription.text);

// Access word timestamps
transcription.words?.forEach(word => {
  console.log(`${word.start}s: ${word.word}`);
});

// Translate
const translation = await client.audio.translations.create({
  file: fs.createReadStream('audio.wav'),
  model: 'tiny'
});

console.log(translation.text);
```

## Generating Custom Clients

You can generate clients for any language using the OpenAPI specification:

### Get the OpenAPI Spec

```bash theme={null}
# Generate the latest spec
make generate-server

# The spec is located at:
# scripts/specs/localserver_openapi.yaml
```

### Generate Clients

#### Python Client

```bash theme={null}
swift run swift-openapi-generator generate \
  scripts/specs/localserver_openapi.yaml \
  --output-directory python-client \
  --mode client \
  --mode types
```

#### TypeScript Client

```bash theme={null}
npx @openapitools/openapi-generator-cli generate \
  -i scripts/specs/localserver_openapi.yaml \
  -g typescript-fetch \
  -o typescript-client
```

#### Go Client

```bash theme={null}
openapi-generator-cli generate \
  -i scripts/specs/localserver_openapi.yaml \
  -g go \
  -o go-client
```

## API Limitations

Compared to the official OpenAI API:

* **Response formats**: Only `json` and `verbose_json` supported (no plain text, SRT, VTT)
* **Model selection**: Server must be launched with desired model via `--model` flag

## Fully Supported Features

The local server fully supports:

* **Log probabilities**: `include[]=logprobs` parameter for token-level confidence
* **Streaming responses**: Server-Sent Events (SSE) for real-time transcription
* **Timestamp granularities**: Both `word` and `segment` level timing
* **Language detection**: Automatic language detection or manual specification
* **Temperature control**: Sampling temperature for transcription randomness
* **Prompt text**: Text guidance for transcription style and context

## Server Configuration

### Environment Variables

```bash theme={null}
# Set custom model cache directory
export WHISPERKIT_CACHE_DIR="/path/to/models"

# Enable debug logging
BUILD_ALL=1 swift run whisperkit-cli serve --verbose
```

### Model Management

```bash theme={null}
# Download a model before starting server
make download-model MODEL=base

# Start server with downloaded model
BUILD_ALL=1 swift run whisperkit-cli serve \
    --model-path "Models/whisperkit-coreml/openai_whisper-base"
```

## Docker Deployment

Create a `Dockerfile`:

```dockerfile theme={null}
FROM swift:5.9

# Install dependencies
RUN apt-get update && apt-get install -y \
    git \
    git-lfs

# Clone and build WhisperKit
WORKDIR /app
RUN git clone https://github.com/argmaxinc/whisperkit.git
WORKDIR /app/whisperkit

RUN make setup
RUN make download-model MODEL=tiny
RUN BUILD_ALL=1 swift build --product whisperkit-cli -c release

EXPOSE 50060

CMD ["BUILD_ALL=1", "swift", "run", "-c", "release", "whisperkit-cli", "serve", "--host", "0.0.0.0"]
```

Build and run:

```bash theme={null}
# Build image
docker build -t whisperkit-server .

# Run container
docker run -p 50060:50060 whisperkit-server
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Basic Transcription" icon="microphone" href="/examples/basic-transcription">
    Learn the basics of file transcription
  </Card>

  <Card title="Real-Time Streaming" icon="signal-stream" href="/examples/real-time-streaming">
    Transcribe audio in real-time
  </Card>
</CardGroup>