> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argmaxinc/WhisperKit/llms.txt
> Use this file to discover all available pages before exploring further.

# WhisperKit

> Main class for speech recognition with WhisperKit

## Overview

The `WhisperKit` class is the main entry point for performing speech-to-text transcription using Apple's Core ML framework. It manages model loading, audio processing, and provides both synchronous and asynchronous transcription methods.

## Class Definition

```swift theme={null}
open class WhisperKit
```

## Initializers

### init(\_:)

Initializes WhisperKit with a configuration object.

```swift theme={null}
public init(_ config: WhisperKitConfig = WhisperKitConfig()) async throws
```

<ParamField path="config" type="WhisperKitConfig">
  Configuration object for WhisperKit initialization. See [WhisperKitConfig](/api/whisperkit/whisperkit-config) for details.
</ParamField>

**Throws:** An error if model setup or loading fails.

### Convenience Initializer

Initializes WhisperKit with individual parameters.

```swift theme={null}
public convenience init(
    model: String? = nil,
    downloadBase: URL? = nil,
    modelRepo: String? = nil,
    modelFolder: String? = nil,
    tokenizerFolder: URL? = nil,
    computeOptions: ModelComputeOptions? = nil,
    audioProcessor: (any AudioProcessing)? = nil,
    featureExtractor: (any FeatureExtracting)? = nil,
    audioEncoder: (any AudioEncoding)? = nil,
    textDecoder: (any TextDecoding)? = nil,
    logitsFilters: [any LogitsFiltering]? = nil,
    segmentSeeker: (any SegmentSeeking)? = nil,
    verbose: Bool = true,
    logLevel: Logging.LogLevel = .info,
    prewarm: Bool? = nil,
    load: Bool? = nil,
    download: Bool = true,
    useBackgroundDownloadSession: Bool = false
) async throws
```

<ParamField path="model" type="String?">
  Name of the Whisper model variant to use (e.g., "tiny", "base", "small", "medium", "large")
</ParamField>

<ParamField path="downloadBase" type="URL?">
  Base URL for downloading models
</ParamField>

<ParamField path="modelRepo" type="String?">
  Repository name for downloading models (default: "argmaxinc/whisperkit-coreml")
</ParamField>

<ParamField path="modelFolder" type="String?">
  Local folder path containing pre-downloaded models
</ParamField>

<ParamField path="tokenizerFolder" type="URL?">
  Folder containing tokenizer files
</ParamField>

<ParamField path="computeOptions" type="ModelComputeOptions?">
  Options for ML compute units (CPU, GPU, Neural Engine)
</ParamField>

<ParamField path="audioProcessor" type="AudioProcessing?">
  Custom audio processor implementation
</ParamField>

<ParamField path="featureExtractor" type="FeatureExtracting?">
  Custom feature extractor implementation
</ParamField>

<ParamField path="audioEncoder" type="AudioEncoding?">
  Custom audio encoder implementation
</ParamField>

<ParamField path="textDecoder" type="TextDecoding?">
  Custom text decoder implementation
</ParamField>

<ParamField path="logitsFilters" type="[LogitsFiltering]?">
  Array of logits filters to apply during decoding
</ParamField>

<ParamField path="segmentSeeker" type="SegmentSeeking?">
  Custom segment seeker implementation
</ParamField>

<ParamField path="verbose" type="Bool" default="true">
  Enable verbose logging
</ParamField>

<ParamField path="logLevel" type="Logging.LogLevel" default=".info">
  Maximum log level to display
</ParamField>

<ParamField path="prewarm" type="Bool?">
  Enable model prewarming to reduce peak memory during initialization
</ParamField>

<ParamField path="load" type="Bool?">
  Whether to load models immediately
</ParamField>

<ParamField path="download" type="Bool" default="true">
  Download models if not available locally
</ParamField>

<ParamField path="useBackgroundDownloadSession" type="Bool" default="false">
  Use background download session for model downloads
</ParamField>

## Properties

### Model State

<ResponseField name="modelVariant" type="ModelVariant">
  Currently loaded model variant (tiny, base, small, medium, large, etc.)
</ResponseField>

<ResponseField name="modelState" type="ModelState">
  Current state of the model (unloaded, loading, loaded, prewarming, etc.)
</ResponseField>

<ResponseField name="modelCompute" type="ModelComputeOptions">
  Compute options for the loaded models
</ResponseField>

<ResponseField name="tokenizer" type="WhisperTokenizer?">
  The tokenizer used for encoding/decoding text
</ResponseField>

### Processing Components

<ResponseField name="audioProcessor" type="AudioProcessing">
  Audio processor for handling audio input and preprocessing
</ResponseField>

<ResponseField name="featureExtractor" type="FeatureExtracting">
  Feature extractor for converting audio to mel spectrograms
</ResponseField>

<ResponseField name="audioEncoder" type="AudioEncoding">
  Audio encoder for encoding mel spectrograms to embeddings
</ResponseField>

<ResponseField name="textDecoder" type="TextDecoding">
  Text decoder for generating text from audio embeddings
</ResponseField>

<ResponseField name="segmentSeeker" type="SegmentSeeking">
  Segment seeker for managing audio window processing
</ResponseField>

<ResponseField name="voiceActivityDetector" type="VoiceActivityDetector?">
  Optional voice activity detector for chunking audio
</ResponseField>

### Configuration

<ResponseField name="audioInputConfig" type="AudioInputConfig">
  Configuration for audio input processing
</ResponseField>

<ResponseField name="modelFolder" type="URL?">
  Path to the folder containing model files
</ResponseField>

<ResponseField name="tokenizerFolder" type="URL?">
  Path to the folder containing tokenizer files
</ResponseField>

### Progress and Callbacks

<ResponseField name="currentTimings" type="TranscriptionTimings">
  Timing information for the current/last transcription
</ResponseField>

<ResponseField name="progress" type="Progress">
  Progress object for tracking transcription progress
</ResponseField>

<ResponseField name="segmentDiscoveryCallback" type="SegmentDiscoveryCallback?">
  Callback invoked when new transcription segments are discovered
</ResponseField>

<ResponseField name="modelStateCallback" type="ModelStateCallback?">
  Callback invoked when model state changes
</ResponseField>

<ResponseField name="transcriptionStateCallback" type="TranscriptionStateCallback?">
  Callback invoked when transcription state changes
</ResponseField>

### Constants

<ResponseField name="sampleRate" type="Int" default="16000">
  Sample rate used for audio processing (16 kHz)
</ResponseField>

<ResponseField name="hopLength" type="Int" default="160">
  Hop length for mel spectrogram computation
</ResponseField>

<ResponseField name="secondsPerTimeToken" type="Float" default="0.02">
  Duration in seconds represented by each time token (20ms)
</ResponseField>

## Static Methods

### deviceName()

Returns the device identifier string.

```swift theme={null}
public static func deviceName() -> String
```

<ResponseField name="return" type="String">
  Device identifier (e.g., "iPhone15,2")
</ResponseField>

### recommendedModels()

Returns recommended models for the current device.

```swift theme={null}
public static func recommendedModels() -> ModelSupport
```

<ResponseField name="return" type="ModelSupport">
  Model support information including default and supported model variants
</ResponseField>

### recommendedRemoteModels(from:downloadBase:token:remoteConfigName:endpoint:)

Fetches recommended models from a remote repository.

```swift theme={null}
public static func recommendedRemoteModels(
    from repo: String = "argmaxinc/whisperkit-coreml",
    downloadBase: URL? = nil,
    token: String? = nil,
    remoteConfigName: String = Constants.defaultRemoteConfigName,
    endpoint: String = Constants.defaultRemoteEndpoint
) async -> ModelSupport
```

<ParamField path="repo" type="String" default="argmaxinc/whisperkit-coreml">
  Repository to fetch model configuration from
</ParamField>

<ParamField path="downloadBase" type="URL?">
  Base URL for downloads
</ParamField>

<ParamField path="token" type="String?">
  Authentication token for the repository
</ParamField>

<ParamField path="remoteConfigName" type="String">
  Name of the remote configuration file
</ParamField>

<ParamField path="endpoint" type="String">
  API endpoint for the repository
</ParamField>

<ResponseField name="return" type="ModelSupport">
  Model support information from the remote repository
</ResponseField>

### fetchAvailableModels(from:matching:downloadBase:token:remoteConfigName:endpoint:)

Fetches list of available models from a remote repository.

```swift theme={null}
public static func fetchAvailableModels(
    from repo: String = "argmaxinc/whisperkit-coreml",
    matching: [String] = ["*"],
    downloadBase: URL? = nil,
    token: String? = nil,
    remoteConfigName: String = Constants.defaultRemoteConfigName,
    endpoint: String = Constants.defaultRemoteEndpoint
) async throws -> [String]
```

<ParamField path="repo" type="String" default="argmaxinc/whisperkit-coreml">
  Repository to fetch models from
</ParamField>

<ParamField path="matching" type="[String]" default="[&#x22;*&#x22;]">
  Glob patterns to filter model names
</ParamField>

<ResponseField name="return" type="[String]">
  Array of available model names
</ResponseField>

### download(variant:downloadBase:useBackgroundSession:from:token:endpoint:progressCallback:)

Downloads a specific model variant.

```swift theme={null}
public static func download(
    variant: String,
    downloadBase: URL? = nil,
    useBackgroundSession: Bool = false,
    from repo: String = "argmaxinc/whisperkit-coreml",
    token: String? = nil,
    endpoint: String = Constants.defaultRemoteEndpoint,
    progressCallback: ((Progress) -> Void)? = nil
) async throws -> URL
```

<ParamField path="variant" type="String">
  Model variant to download (e.g., "tiny", "base", "small")
</ParamField>

<ParamField path="progressCallback" type="((Progress) -> Void)?">
  Optional callback for download progress updates
</ParamField>

<ResponseField name="return" type="URL">
  Local URL of the downloaded model folder
</ResponseField>

## Instance Methods

### loadModels(prewarmMode:)

Loads the models into memory.

```swift theme={null}
open func loadModels(prewarmMode: Bool = false) async throws
```

<ParamField path="prewarmMode" type="Bool" default="false">
  If true, loads models in prewarm mode to reduce peak memory usage
</ParamField>

### prewarmModels()

Prewarms the models by loading them sequentially.

```swift theme={null}
open func prewarmModels() async throws
```

### unloadModels()

Unloads all models from memory.

```swift theme={null}
open func unloadModels() async
```

### clearState()

Clears the current transcription state.

```swift theme={null}
open func clearState()
```

### detectLanguage(audioPath:)

Detects the language of audio from a file path.

```swift theme={null}
open func detectLanguage(
    audioPath: String
) async throws -> (language: String, langProbs: [String: Float])
```

<ParamField path="audioPath" type="String">
  Path to the audio file
</ParamField>

<ResponseField name="return" type="(language: String, langProbs: [String: Float])">
  Tuple containing detected language code and probability distribution over all languages
</ResponseField>

### detectLangauge(audioArray:)

Detects the language of audio from sample array.

```swift theme={null}
open func detectLangauge(
    audioArray: [Float]
) async throws -> (language: String, langProbs: [String: Float])
```

<ParamField path="audioArray" type="[Float]">
  Array of 16kHz audio samples
</ParamField>

<ResponseField name="return" type="(language: String, langProbs: [String: Float])">
  Tuple containing detected language code and probability distribution
</ResponseField>

### transcribe(audioPath:decodeOptions:callback:)

Transcribes audio from a file path.

```swift theme={null}
open func transcribe(
    audioPath: String,
    decodeOptions: DecodingOptions? = nil,
    callback: TranscriptionCallback = nil
) async throws -> [TranscriptionResult]
```

<ParamField path="audioPath" type="String">
  Path to the audio file to transcribe
</ParamField>

<ParamField path="decodeOptions" type="DecodingOptions?">
  Options for transcription (language, task, temperature, etc.)
</ParamField>

<ParamField path="callback" type="TranscriptionCallback">
  Optional callback for progress updates during transcription
</ParamField>

<ResponseField name="return" type="[TranscriptionResult]">
  Array of transcription results. See [TranscriptionResult](/api/whisperkit/transcription-result) for details.
</ResponseField>

### transcribe(audioArray:decodeOptions:callback:segmentCallback:)

Transcribes audio from a sample array.

```swift theme={null}
open func transcribe(
    audioArray: [Float],
    decodeOptions: DecodingOptions? = nil,
    callback: TranscriptionCallback = nil,
    segmentCallback: SegmentDiscoveryCallback? = nil
) async throws -> [TranscriptionResult]
```

<ParamField path="audioArray" type="[Float]">
  Array of 16kHz mono audio samples
</ParamField>

<ParamField path="decodeOptions" type="DecodingOptions?">
  Options for transcription
</ParamField>

<ParamField path="callback" type="TranscriptionCallback">
  Optional callback for progress updates
</ParamField>

<ParamField path="segmentCallback" type="SegmentDiscoveryCallback?">
  Optional callback invoked when segments are discovered
</ParamField>

<ResponseField name="return" type="[TranscriptionResult]">
  Array of transcription results
</ResponseField>

### transcribe(audioPaths:decodeOptions:callback:)

Transcribes multiple audio files.

```swift theme={null}
open func transcribe(
    audioPaths: [String],
    decodeOptions: DecodingOptions? = nil,
    callback: TranscriptionCallback = nil
) async -> [[TranscriptionResult]?]
```

<ParamField path="audioPaths" type="[String]">
  Array of audio file paths to transcribe
</ParamField>

<ResponseField name="return" type="[[TranscriptionResult]?]">
  Array of optional transcription result arrays (nil if transcription failed for that file)
</ResponseField>

### transcribe(audioArrays:decodeOptions:callback:)

Transcribes multiple audio sample arrays.

```swift theme={null}
open func transcribe(
    audioArrays: [[Float]],
    decodeOptions: DecodingOptions? = nil,
    callback: TranscriptionCallback = nil
) async -> [[TranscriptionResult]?]
```

<ParamField path="audioArrays" type="[[Float]]">
  Array of audio sample arrays to transcribe
</ParamField>

<ResponseField name="return" type="[[TranscriptionResult]?]">
  Array of optional transcription result arrays
</ResponseField>

### loggingCallback(\_:)

Sets a custom logging callback.

```swift theme={null}
open func loggingCallback(_ callback: Logging.LoggingCallback?)
```

<ParamField path="callback" type="Logging.LoggingCallback?">
  Custom logging callback function
</ParamField>

## Example Usage

### Basic Transcription

```swift theme={null}
import WhisperKit

// Initialize WhisperKit with default settings
let whisperKit = try await WhisperKit()

// Transcribe an audio file
let results = try await whisperKit.transcribe(audioPath: "/path/to/audio.wav")

// Access the transcribed text
for result in results {
    print(result.text)
}
```

### Custom Configuration

```swift theme={null}
// Initialize with specific model and options
let config = WhisperKitConfig(
    model: "base",
    computeOptions: ModelComputeOptions(
        audioEncoderCompute: .cpuAndGPU,
        textDecoderCompute: .cpuAndNeuralEngine
    ),
    verbose: true,
    load: true
)

let whisperKit = try await WhisperKit(config)
```

### Language Detection

```swift theme={null}
let (language, probabilities) = try await whisperKit.detectLanguage(
    audioPath: "/path/to/audio.wav"
)

print("Detected language: \(language)")
print("Confidence: \(probabilities[language] ?? 0)")
```

### With Progress Callback

```swift theme={null}
let results = try await whisperKit.transcribe(
    audioPath: "/path/to/audio.wav",
    decodeOptions: DecodingOptions(task: .transcribe, language: "en")
) { progress in
    print("Progress: \(progress.text)")
    return true // Continue transcription
}
```