ModelComputeOptions

Configure compute units for WhisperKit model components

ModelComputeOptions configures which compute units (CPU, GPU, Neural Engine) are used for each component of the WhisperKit transcription pipeline.

Overview

WhisperKit models consist of multiple components that can run on different compute units. ModelComputeOptions allows fine-grained control over where each component executes for optimal performance.

Properties

melCompute · MLComputeUnits · default: ".cpuAndGPU"

Compute units for mel spectrogram generation.

audioEncoderCompute · MLComputeUnits

Compute units for audio encoding. Defaults to Neural Engine on newer OS versions for better performance.

textDecoderCompute · MLComputeUnits · default: ".cpuAndNeuralEngine"

Compute units for text decoding.

prefillCompute · MLComputeUnits · default: ".cpuOnly"

Compute units for KV cache prefill operations.

Initialization

public init(
    melCompute: MLComputeUnits = .cpuAndGPU,
    audioEncoderCompute: MLComputeUnits? = nil,
    textDecoderCompute: MLComputeUnits = .cpuAndNeuralEngine,
    prefillCompute: MLComputeUnits = .cpuOnly
)

Parameters

melCompute · MLComputeUnits · default: ".cpuAndGPU"

Compute units for mel spectrogram extraction.

audioEncoderCompute · MLComputeUnits · default: "nil"

Compute units for audio encoding. When nil, automatically selects .cpuAndNeuralEngine on iOS 17+/macOS 14+ and .cpuAndGPU on older versions.

textDecoderCompute · MLComputeUnits · default: ".cpuAndNeuralEngine"

Compute units for text decoding.

prefillCompute · MLComputeUnits · default: ".cpuOnly"

Compute units for prefill operations.

MLComputeUnits Values

Apple’s MLComputeUnits enum specifies which processors can execute a Core ML model:

• .cpuOnly - Only the CPU
• .cpuAndGPU - CPU and GPU
• .cpuAndNeuralEngine - CPU and Neural Engine (recommended for most models)
• .all - All available compute units

Usage Examples

Default Configuration

let options = ModelComputeOptions()
// Uses optimal defaults for your device

Custom Configuration

let options = ModelComputeOptions(
    melCompute: .cpuAndGPU,
    audioEncoderCompute: .cpuAndNeuralEngine,
    textDecoderCompute: .cpuAndNeuralEngine,
    prefillCompute: .cpuOnly
)

let config = WhisperKitConfig(
    model: "openai_whisper-base",
    computeOptions: options
)

let whisperKit = try await WhisperKit(config)

Force CPU-Only Execution

let cpuOnlyOptions = ModelComputeOptions(
    melCompute: .cpuOnly,
    audioEncoderCompute: .cpuOnly,
    textDecoderCompute: .cpuOnly,
    prefillCompute: .cpuOnly
)

Simulator Handling

When running on the iOS Simulator, all compute options are automatically overridden to .cpuOnly since the simulator doesn’t support GPU or Neural Engine execution.

Performance Considerations

Neural Engine

The Neural Engine provides the best performance for most Whisper models on Apple Silicon devices:

• Recommended for audioEncoderCompute and textDecoderCompute
• Requires iOS 17+/macOS 14+ for optimal audio encoder performance
• Most efficient for battery-powered devices

GPU

The GPU can be beneficial for certain operations:

• Recommended for melCompute on most devices
• May provide better throughput on some Mac models
• Consider for batch processing scenarios

CPU Only

CPU-only execution:

• Required for prefillCompute operations
• Use when Neural Engine/GPU are unavailable
• May be necessary for debugging or compatibility

Related Types

• WhisperKitConfig
• ModelState