> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argmaxinc/WhisperKit/llms.txt
> Use this file to discover all available pages before exploring further.

# WhisperKit Overview

> Introduction to WhisperKit - On-device speech recognition powered by OpenAI Whisper

# WhisperKit Overview

WhisperKit is a Swift framework that brings OpenAI's Whisper speech recognition models to Apple devices. It leverages Core ML for efficient on-device inference, enabling privacy-focused, low-latency speech-to-text capabilities.

## Key Features

<CardGroup cols={2}>
  <Card title="On-Device Processing" icon="mobile">
    All transcription happens locally on the device, ensuring user privacy and enabling offline functionality.
  </Card>

  <Card title="Multiple Model Sizes" icon="layer-group">
    Choose from tiny, base, small, medium, and large variants to balance accuracy and performance.
  </Card>

  <Card title="Multilingual Support" icon="globe">
    Supports 99+ languages with automatic language detection for multilingual models.
  </Card>

  <Card title="Real-time Streaming" icon="podcast">
    Stream audio from the microphone and get transcriptions in real-time.
  </Card>
</CardGroup>

## Core Components

WhisperKit consists of several key components:

### WhisperKit Class

The main entry point for speech-to-text functionality. It orchestrates all the components needed for transcription.

```swift theme={null}
let whisperKit = try await WhisperKit()
```

See [WhisperKit.swift:12](~/workspace/source/Sources/WhisperKit/Core/WhisperKit.swift:12)

### Audio Processing Pipeline

1. **AudioProcessor** - Captures and processes audio input
2. **FeatureExtractor** - Converts audio to mel spectrograms
3. **AudioEncoder** - Encodes audio features using the Whisper encoder model
4. **TextDecoder** - Decodes encoded features into text tokens
5. **Tokenizer** - Converts tokens to readable text

### Model Architecture

WhisperKit uses Core ML models downloaded from Hugging Face Hub:

* **MelSpectrogram.mlmodelc** - Audio feature extraction
* **AudioEncoder.mlmodelc** - Audio encoding
* **TextDecoder.mlmodelc** - Text decoding
* **TextDecoderContextPrefill.mlmodelc** - Optional prefill optimization

## Quick Start

### Basic Initialization

```swift theme={null}
import WhisperKit

// Initialize with default model
let whisperKit = try await WhisperKit()

// Transcribe an audio file
let results = try await whisperKit.transcribe(audioPath: "path/to/audio.wav")
for result in results {
    print(result.text)
}
```

### Custom Configuration

```swift theme={null}
// Initialize with specific model and options
let config = WhisperKitConfig(
    model: "large-v3",
    computeOptions: ModelComputeOptions(
        audioEncoderCompute: .cpuAndNeuralEngine
    ),
    verbose: true,
    download: true
)

let whisperKit = try await WhisperKit(config)
```

See [WhisperKitConfig](~/workspace/source/Sources/WhisperKit/Core/Configurations.swift:7)

## Model States

WhisperKit models progress through several states:

* **unloaded** - Models not yet loaded
* **prewarming** - Models being specialized for the device (optional)
* **prewarmed** - Specialization complete
* **loading** - Models being loaded into memory
* **loaded** - Ready for transcription
* **unloading** - Models being removed from memory

See [WhisperKit.swift:15-19](~/workspace/source/Sources/WhisperKit/Core/WhisperKit.swift:15-19)

## Sample Rates and Constants

WhisperKit uses fixed audio parameters matching the Whisper model requirements:

```swift theme={null}
WhisperKit.sampleRate // 16000 Hz
WhisperKit.hopLength // 160 samples
WhisperKit.secondsPerTimeToken // 0.02 seconds
```

See [WhisperKit.swift:39-41](~/workspace/source/Sources/WhisperKit/Core/WhisperKit.swift:39-41)

## Next Steps

<CardGroup cols={2}>
  <Card title="Transcription" icon="file-audio" href="/whisperkit/transcription">
    Learn how to transcribe audio files
  </Card>

  <Card title="Streaming" icon="podcast" href="/whisperkit/streaming">
    Real-time audio transcription
  </Card>

  <Card title="Model Selection" icon="layer-group" href="/whisperkit/model-selection">
    Choose the right model for your needs
  </Card>

  <Card title="Configuration" icon="gear" href="/whisperkit/configuration">
    Advanced configuration options
  </Card>
</CardGroup>