> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argmaxinc/WhisperKit/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance Optimization

> Optimize transcription speed, memory usage, and quality with WhisperKit

Optimizing WhisperKit performance involves balancing speed, accuracy, and resource usage. This guide covers compute units, model selection, decoding options, and platform-specific optimizations.

## Compute Units

CoreML models can target different hardware accelerators on Apple devices:

<CardGroup cols={3}>
  <Card title="Neural Engine" icon="brain-circuit">
    Specialized ML accelerator (fastest, most efficient)
  </Card>

  <Card title="GPU" icon="microchip">
    Graphics processor (good balance)
  </Card>

  <Card title="CPU" icon="microchip">
    Central processor (most compatible)
  </Card>
</CardGroup>

### ModelComputeOptions

Configure compute units for each model component:

```swift theme={null}
import WhisperKit
import CoreML

let computeOptions = ModelComputeOptions(
    melCompute: .cpuAndGPU,              // Mel spectrogram extraction
    audioEncoderCompute: .cpuAndNeuralEngine,  // Audio encoder
    textDecoderCompute: .cpuAndNeuralEngine,   // Text decoder
    prefillCompute: .cpuOnly             // KV cache prefill
)

let config = WhisperKitConfig(
    model: "large-v3",
    computeOptions: computeOptions
)

let pipe = try await WhisperKit(config)
```

### Available Compute Units

<ResponseField name=".cpuOnly" type="MLComputeUnits">
  CPU only - most compatible, slowest
</ResponseField>

<ResponseField name=".cpuAndGPU" type="MLComputeUnits">
  CPU and GPU - good for macOS \< 14
</ResponseField>

<ResponseField name=".cpuAndNeuralEngine" type="MLComputeUnits">
  CPU and Neural Engine - recommended for macOS 14+, iOS 17+
</ResponseField>

<ResponseField name=".all" type="MLComputeUnits">
  All available compute units - lets CoreML decide
</ResponseField>

### Recommended Configurations

<Tabs>
  <Tab title="macOS 14+ (Recommended)">
    ```swift theme={null}
    let computeOptions = ModelComputeOptions(
        melCompute: .cpuAndGPU,
        audioEncoderCompute: .cpuAndNeuralEngine,
        textDecoderCompute: .cpuAndNeuralEngine,
        prefillCompute: .cpuOnly
    )
    ```

    **Best performance on M1/M2/M3 Macs**
  </Tab>

  <Tab title="macOS 13 and Earlier">
    ```swift theme={null}
    let computeOptions = ModelComputeOptions(
        melCompute: .cpuAndGPU,
        audioEncoderCompute: .cpuAndGPU,
        textDecoderCompute: .cpuAndNeuralEngine,
        prefillCompute: .cpuOnly
    )
    ```

    **Neural Engine support limited for audio encoder**
  </Tab>

  <Tab title="iOS 18+">
    ```swift theme={null}
    let computeOptions = ModelComputeOptions(
        melCompute: .cpuAndGPU,
        audioEncoderCompute: .cpuAndNeuralEngine,
        textDecoderCompute: .cpuAndNeuralEngine,
        prefillCompute: .cpuOnly
    )
    ```

    **Optimized for iPhone/iPad**
  </Tab>

  <Tab title="Low Memory">
    ```swift theme={null}
    let computeOptions = ModelComputeOptions(
        melCompute: .cpuOnly,
        audioEncoderCompute: .cpuOnly,
        textDecoderCompute: .cpuOnly,
        prefillCompute: .cpuOnly
    )
    ```

    **Minimize memory usage**
  </Tab>
</Tabs>

### CLI Configuration

Set compute units via command-line:

```bash theme={null}
swift run whisperkit-cli transcribe \
  --audio-path "audio.wav" \
  --model-path "Models/whisperkit-coreml/openai_whisper-large-v3" \
  --audio-encoder-compute-units cpuAndNeuralEngine \
  --text-decoder-compute-units cpuAndNeuralEngine
```

## Model Selection

Model size significantly impacts speed and accuracy:

<ResponseField name="tiny" type="39M parameters">
  **Speed**: \~32x real-time | **WER**: \~10-15% | **Size**: \~75 MB

  Best for: Real-time applications, low-end devices, quick prototyping
</ResponseField>

<ResponseField name="base" type="74M parameters">
  **Speed**: \~16x real-time | **WER**: \~8-12% | **Size**: \~142 MB

  Best for: Balanced speed/quality, general transcription
</ResponseField>

<ResponseField name="small" type="244M parameters">
  **Speed**: \~6x real-time | **WER**: \~5-8% | **Size**: \~466 MB

  Best for: Production applications requiring accuracy
</ResponseField>

<ResponseField name="medium" type="769M parameters">
  **Speed**: \~2x real-time | **WER**: \~4-6% | **Size**: \~1.5 GB

  Best for: High-accuracy requirements, offline processing
</ResponseField>

<ResponseField name="large-v3" type="1550M parameters">
  **Speed**: \~1x real-time | **WER**: \~3-5% | **Size**: \~3 GB

  Best for: Maximum accuracy, multilingual, post-processing
</ResponseField>

### Distil Models

Distilled models offer 2-3x speedup with minimal accuracy loss:

```swift theme={null}
let config = WhisperKitConfig(
    model: "distil*large-v3",  // Glob pattern
    modelRepo: "argmaxinc/whisperkit-coreml"
)
```

Benchmarks: [WhisperKit Benchmarks](https://huggingface.co/spaces/argmaxinc/whisperkit-benchmarks)

## Decoding Options

### Temperature and Sampling

Control randomness and diversity:

```swift theme={null}
var options = DecodingOptions()
options.temperature = 0.0  // Deterministic (default)
options.topK = 5           // Top-K sampling candidates
```

<ParamField path="temperature" type="Float" default="0.0">
  * `0.0`: Deterministic (always pick most likely token)
  * `0.0 - 1.0`: Increasing randomness
  * Higher values = more creative but less accurate
</ParamField>

<ParamField path="topK" type="Int" default="5">
  Number of top candidates to sample from when temperature > 0
</ParamField>

### Fallback Strategy

Automatically retry failed segments:

```swift theme={null}
var options = DecodingOptions()
options.temperatureIncrementOnFallback = 0.2  // Increment per retry
options.temperatureFallbackCount = 5          // Max retries
```

### Quality Thresholds

Detect and reject poor transcriptions:

```swift theme={null}
var options = DecodingOptions()
options.compressionRatioThreshold = 2.4       // Detect repetition
options.logProbThreshold = -1.0               // Average confidence
options.firstTokenLogProbThreshold = -1.5     // First token confidence
options.noSpeechThreshold = 0.6               // Silence detection
```

<Tip>
  **Lower thresholds = more rejections = better quality but longer processing**

  Default values work well for most use cases.
</Tip>

### Timestamp Control

```swift theme={null}
var options = DecodingOptions()
options.withoutTimestamps = false      // Include timestamps
options.wordTimestamps = true          // Word-level timing
options.maxInitialTimestamp = 1.0      // First timestamp limit
```

Word timestamps require \~10-15% more processing time.

## Parallel Processing

### Concurrent Workers

Process multiple audio segments in parallel:

```swift theme={null}
var options = DecodingOptions()

#if os(macOS)
options.concurrentWorkerCount = 16  // Default on macOS
#else
options.concurrentWorkerCount = 4   // Default on iOS/iPadOS
#endif
```

<Warning>
  iOS devices show regression with >4 workers. macOS handles 16+ workers efficiently.
</Warning>

### Chunking Strategy

```swift theme={null}
var options = DecodingOptions()
options.chunkingStrategy = .vad  // Voice Activity Detection
// or
options.chunkingStrategy = .none  // Process entire audio
```

**Voice Activity Detection (VAD)**:

* Automatically splits audio at silence
* Reduces unnecessary processing
* Better for long audio with pauses
* Slightly slower initialization

**No Chunking**:

* Process entire audio file
* Faster for short clips
* May hit token limits on very long audio

## Prefill Optimization

### KV Cache Prefill

Accelerate initial decoding with cached key-value pairs:

```swift theme={null}
var options = DecodingOptions()
options.usePrefillCache = true   // Use prefilled KV cache (faster)
options.usePrefillPrompt = true  // Force initial prompt tokens
```

<Info>
  **Prefill reduces first-token latency by 2-3x** but requires compatible models with prefill data.
</Info>

### Language and Task Prefill

```swift theme={null}
var options = DecodingOptions(
    task: .transcribe,
    language: "en",
    usePrefillPrompt: true,
    usePrefillCache: true
)
```

Automatic prefill when:

* Language is specified
* Task is `.translate`
* Custom prompt tokens provided

## Memory Management

### Model Prewarming

Reduce peak memory during model loading:

```swift theme={null}
let config = WhisperKitConfig(
    model: "large-v3",
    prewarm: true  // Load-unload-load pattern
)
```

<Accordion title="What is prewarming?">
  CoreML models need "specialization" on first load (compiling for your device). This specialized cache is maintained by Apple but evicted after OS updates.

  **With prewarm=true**:

  * Models loaded sequentially
  * Each model unloaded after specialization
  * Lower peak memory (1 model at a time)
  * 2x longer load time if cache is hit

  **With prewarm=false** (default):

  * Models loaded in parallel
  * Higher peak memory (all models + compilation)
  * Faster load time when cache is hit

  **Enable when**: Minimizing peak memory is critical (older devices, background apps)

  **Disable when**: Load time is critical (real-time apps, foreground processing)
</Accordion>

```swift theme={null}
let config = WhisperKitConfig(
    model: "large-v3",
    prewarm: true,
    verbose: true  // See timing breakdown
)

let pipe = try await WhisperKit(config)
// Logs: prewarmLoadTime, modelLoading, encoderLoadTime, decoderLoadTime
```

See [Memory Management](/advanced/memory-management) for detailed strategies.

## Performance Metrics

### Real-Time Factor

```swift theme={null}
let result = try await pipe.transcribe(audioPath: "audio.wav")
let rtf = result?.timings.realTimeFactor ?? 0

print("Real-time factor: \(rtf)")
// 0.25 = 4x faster than real-time
// 1.0 = same speed as audio duration
// 2.0 = 2x slower than real-time
```

### Tokens Per Second

```swift theme={null}
let tps = result?.timings.tokensPerSecond ?? 0
print("Tokens per second: \(tps)")
// Higher is better (typically 50-200 for large models)
```

### Speed Factor

```swift theme={null}
let speedFactor = result?.timings.speedFactor ?? 0
print("Speed factor: \(speedFactor)x")
// Inverse of real-time factor
// 4.0 = 4x faster than real-time
```

## Platform-Specific Tips

### macOS

<Card title="M1/M2/M3 Optimization">
  ```swift theme={null}
  let computeOptions = ModelComputeOptions(
      melCompute: .cpuAndGPU,
      audioEncoderCompute: .cpuAndNeuralEngine,
      textDecoderCompute: .cpuAndNeuralEngine,
      prefillCompute: .cpuOnly
  )

  var decodingOptions = DecodingOptions()
  decodingOptions.concurrentWorkerCount = 16
  decodingOptions.chunkingStrategy = .vad
  ```
</Card>

### iOS/iPadOS

<Card title="iPhone/iPad Optimization">
  ```swift theme={null}
  // Use smaller models on mobile
  let config = WhisperKitConfig(
      model: "small",  // or "distil*small"
      computeOptions: ModelComputeOptions(
          audioEncoderCompute: .cpuAndNeuralEngine,
          textDecoderCompute: .cpuAndNeuralEngine
      )
  )

  var decodingOptions = DecodingOptions()
  decodingOptions.concurrentWorkerCount = 4  // Don't exceed 4
  ```
</Card>

### watchOS

<Warning>
  WhisperKit on watchOS requires tiny/base models with CPU-only compute units due to memory constraints.
</Warning>

## Benchmarking

Measure performance on your target devices:

```bash theme={null}
make benchmark-devices
```

View results:

* [WhisperKit Benchmarks Space](https://huggingface.co/spaces/argmaxinc/whisperkit-benchmarks)
* Local: `fastlane/benchmark_data/`

See [BENCHMARKS.md](https://github.com/argmaxinc/WhisperKit/blob/main/BENCHMARKS.md) for details.

## Optimization Checklist

<Steps>
  <Step title="Choose the right model">
    Start with `small` for balance, `tiny` for speed, `large-v3` for accuracy
  </Step>

  <Step title="Configure compute units">
    Use `.cpuAndNeuralEngine` on macOS 14+ and iOS 17+
  </Step>

  <Step title="Enable prefill caching">
    Set `usePrefillCache = true` for faster first-token generation
  </Step>

  <Step title="Adjust concurrent workers">
    16 on macOS, 4 on iOS for optimal parallelism
  </Step>

  <Step title="Use VAD chunking">
    Enable `chunkingStrategy = .vad` for long audio files
  </Step>

  <Step title="Tune quality thresholds">
    Lower thresholds for better quality, higher for speed
  </Step>

  <Step title="Monitor metrics">
    Track real-time factor and tokens per second
  </Step>

  <Step title="Test on target devices">
    Always benchmark on actual hardware
  </Step>
</Steps>

## Next Steps

<CardGroup cols={2}>
  <Card title="Memory Management" icon="memory" href="/advanced/memory-management">
    Advanced memory optimization strategies
  </Card>

  <Card title="Custom Models" icon="brain" href="/advanced/custom-models">
    Fine-tune models for your domain
  </Card>
</CardGroup>