> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argmaxinc/WhisperKit/llms.txt
> Use this file to discover all available pages before exploring further.

# Memory Management

> Optimize memory usage for WhisperKit models on resource-constrained devices

Efficient memory management is critical when deploying large Whisper models on Apple devices. This guide covers prewarming, model caching, resource allocation, and strategies for minimizing memory footprint.

## Understanding Memory Usage

WhisperKit's memory footprint consists of:

<CardGroup cols={3}>
  <Card title="Model Weights" icon="database">
    75 MB (tiny) to 3 GB (large-v3)
  </Card>

  <Card title="KV Cache" icon="layer-group">
    Dynamic during decoding
  </Card>

  <Card title="Audio Buffers" icon="waveform">
    Mel spectrograms and features
  </Card>
</CardGroup>

### Model Size Reference

| Model    | Parameters | Disk Size | Memory (Loaded) |
| -------- | ---------- | --------- | --------------- |
| tiny     | 39M        | \~75 MB   | \~150 MB        |
| base     | 74M        | \~142 MB  | \~280 MB        |
| small    | 244M       | \~466 MB  | \~900 MB        |
| medium   | 769M       | \~1.5 GB  | \~3 GB          |
| large-v3 | 1550M      | \~3 GB    | \~6 GB          |

<Note>
  Loaded memory includes model weights, CoreML runtime overhead, and active computation buffers.
</Note>

## Model Prewarming

### What is Prewarming?

CoreML models are downloaded as device-agnostic `.mlmodelc` files and must be "specialized" (compiled) for your specific device chip before use. Apple caches these specialized models, but the cache is evicted:

* After OS updates
* When not used for extended periods
* When system storage is low

Prewarming triggers specialization sequentially to minimize peak memory.

### Configuration

```swift theme={null}
import WhisperKit

// Enable prewarming
let config = WhisperKitConfig(
    model: "large-v3",
    prewarm: true
)

let pipe = try await WhisperKit(config)
```

### Prewarming Workflow

<Steps>
  <Step title="Load Model 1">
    Load audio encoder → specialization if needed
  </Step>

  <Step title="Unload Model 1">
    Immediately release audio encoder memory
  </Step>

  <Step title="Load Model 2">
    Load text decoder → specialization if needed
  </Step>

  <Step title="Unload Model 2">
    Release text decoder memory
  </Step>

  <Step title="Final Load">
    Load all models together (now cached)
  </Step>
</Steps>

### Trade-offs

<Tabs>
  <Tab title="prewarm: true">
    **Pros:**

    * Lower peak memory (1 model at a time)
    * Safer for background/low-memory apps
    * Prevents crashes on older devices

    **Cons:**

    * 2x longer load time when cache is hit
    * Unnecessary overhead if cache is fresh
  </Tab>

  <Tab title="prewarm: false (default)">
    **Pros:**

    * Faster load time when cache is hit
    * Models loaded in parallel
    * Better for real-time applications

    **Cons:**

    * Higher peak memory during compilation
    * May cause memory pressure on older devices
    * Can trigger system warnings
  </Tab>
</Tabs>

### When to Use Prewarming

<AccordionGroup>
  <Accordion title="Enable prewarming when:">
    * Deploying on older iOS devices (iPhone 11 and earlier)
    * Using large models (medium, large-v3) on mobile
    * App runs in background or as extension
    * Memory pressure warnings occur
    * First launch after OS update
  </Accordion>

  <Accordion title="Disable prewarming when:">
    * Deploying on M1/M2/M3 Macs with ample RAM
    * Using small models (tiny, base)
    * Load time is critical (real-time apps)
    * Models are loaded once and cached
  </Accordion>
</AccordionGroup>

## CoreML Model Cache

### Cache Location

Apple maintains CoreML specialized model cache outside your app bundle:

```
~/Library/Caches/com.apple.CoreML/
```

This cache is:

* **Managed by the OS** (you cannot directly control it)
* **Device-specific** (different for each chip)
* **Persistent** across app launches
* **Evicted** unpredictably by the system

### Checking Cache Status

No official API exists, but you can measure load time:

```swift theme={null}
import WhisperKit

let start = Date()
let config = WhisperKitConfig(model: "large-v3", verbose: true)
let pipe = try await WhisperKit(config)
let elapsed = Date().timeIntervalSince(start)

print("Model load time: \(elapsed)s")
// < 2s = cache hit
// > 10s = cache miss (compilation)
```

### Prefilling KV Cache

Accelerate decoding with prefilled key-value cache:

```swift theme={null}
var options = DecodingOptions(
    language: "en",
    task: .transcribe,
    usePrefillCache: true,   // Enable prefill
    usePrefillPrompt: true   // Force initial tokens
)

let result = try await pipe.transcribe(
    audioPath: "audio.wav",
    decodeOptions: options
)
```

<Info>
  KV cache prefill reduces first-token latency by 2-3x by preloading common decoder states.
</Info>

## Resource Allocation

### Compute Units and Memory

Different compute units have different memory characteristics:

<ResponseField name="CPU Only" type=".cpuOnly">
  * **Memory**: Uses system RAM
  * **Usage**: 200-400 MB additional overhead
  * **Best for**: Extreme memory constraints
</ResponseField>

<ResponseField name="GPU" type=".cpuAndGPU">
  * **Memory**: Uses unified memory (shared with CPU)
  * **Usage**: 300-600 MB additional overhead
  * **Best for**: Balanced performance
</ResponseField>

<ResponseField name="Neural Engine" type=".cpuAndNeuralEngine">
  * **Memory**: Uses dedicated ANE memory + system RAM
  * **Usage**: 400-800 MB additional overhead
  * **Best for**: Maximum speed on supported devices
</ResponseField>

### Optimizing Compute Units for Memory

```swift theme={null}
import CoreML
import WhisperKit

// Minimal memory configuration
let computeOptions = ModelComputeOptions(
    melCompute: .cpuOnly,           // Lightest
    audioEncoderCompute: .cpuOnly,  // Most memory-efficient
    textDecoderCompute: .cpuOnly,
    prefillCompute: .cpuOnly
)

let config = WhisperKitConfig(
    model: "tiny",  // Smallest model
    computeOptions: computeOptions,
    prewarm: true   // Lower peak memory
)
```

## Memory Monitoring

### Runtime Memory Tracking

```swift theme={null}
import WhisperKit
import os

func logMemoryUsage(_ label: String) {
    var info = mach_task_basic_info()
    var count = mach_msg_type_number_t(MemoryLayout<mach_task_basic_info>.size) / 4
    
    let result = withUnsafeMutablePointer(to: &info) {
        $0.withMemoryRebound(to: integer_t.self, capacity: 1) {
            task_info(mach_task_self_, task_flavor_t(MACH_TASK_BASIC_INFO), $0, &count)
        }
    }
    
    if result == KERN_SUCCESS {
        let usedMB = Double(info.resident_size) / 1024 / 1024
        print("\(label): \(String(format: "%.2f", usedMB)) MB")
    }
}

// Usage
logMemoryUsage("Before init")
let pipe = try await WhisperKit()
logMemoryUsage("After init")

let result = try await pipe.transcribe(audioPath: "audio.wav")
logMemoryUsage("After transcription")
```

### Xcode Instruments

<Steps>
  <Step title="Open Instruments">
    Product → Profile (⌘I) in Xcode
  </Step>

  <Step title="Select Allocations">
    Choose "Allocations" template
  </Step>

  <Step title="Record Session">
    Run your app and transcribe audio
  </Step>

  <Step title="Analyze">
    Look for:

    * Peak memory usage
    * Memory growth over time
    * Allocation backtrace
  </Step>
</Steps>

## Strategies for Large Models

### Model Splitting

Load encoder and decoder separately:

```swift theme={null}
import WhisperKit

// Load only encoder first
let encoderConfig = WhisperKitConfig(
    modelFolder: "models/large-v3",
    load: false  // Don't load automatically
)
let pipe = try await WhisperKit(encoderConfig)

// Manually load encoder
try await pipe.loadModels()  // Load encoder only

// Process audio
let features = try await pipe.audioEncoder.encodeFeatures(...)

// Later, load decoder when needed
try await pipe.loadTextDecoder()
```

### Lazy Loading

Defer model loading until needed:

```swift theme={null}
import WhisperKit

let config = WhisperKitConfig(
    model: "large-v3",
    load: false,      // Don't load on init
    download: true    // But download if missing
)

let pipe = try await WhisperKit(config)
// Models downloaded but not loaded

// Load when user triggers transcription
button.action = {
    try await pipe.loadModels()
    let result = try await pipe.transcribe(audioPath: "audio.wav")
}
```

### Unloading Models

Free memory after transcription:

```swift theme={null}
import WhisperKit

var pipe: WhisperKit? = try await WhisperKit()

// Use model
let result = try await pipe?.transcribe(audioPath: "audio.wav")

// Release memory
pipe = nil
// Or explicit unload if keeping reference
// pipe.unloadModels()  // If implemented
```

## Concurrent Processing

### Worker Count and Memory

More workers = more memory:

```swift theme={null}
var options = DecodingOptions()

// High memory available
options.concurrentWorkerCount = 16  // macOS with >16GB RAM

// Medium memory
options.concurrentWorkerCount = 8   // macOS with 8-16GB RAM

// Low memory
options.concurrentWorkerCount = 4   // iOS devices

// Minimal memory
options.concurrentWorkerCount = 1   // Sequential processing
```

<Warning>
  Each concurrent worker may hold its own audio buffer and KV cache state. Start with defaults and adjust based on memory pressure.
</Warning>

### Sequential Processing

```swift theme={null}
import WhisperKit

let pipe = try await WhisperKit(WhisperKitConfig(model: "large-v3"))

let audioFiles = ["file1.wav", "file2.wav", "file3.wav"]

// Process one at a time
for file in audioFiles {
    let result = try await pipe.transcribe(audioPath: file)
    print("\(file): \(result?.text ?? "")")
    
    // Optional: explicit cleanup
    // autoreleasepool { ... }
}
```

## Audio Buffer Management

### Chunking Strategy

VAD chunking reduces memory by processing smaller segments:

```swift theme={null}
var options = DecodingOptions()
options.chunkingStrategy = .vad  // Voice Activity Detection

// VAD splits long audio into manageable chunks
// Smaller chunks = less memory per segment
```

### Clip Timestamps

Manually segment long audio:

```swift theme={null}
var options = DecodingOptions()
// Split 10-minute audio into 2-minute segments
options.clipTimestamps = [
    0.0,    // Start
    120.0,  // 2 min
    240.0,  // 4 min
    360.0,  // 6 min
    480.0,  // 8 min
    600.0   // 10 min (end)
]

let result = try await pipe.transcribe(
    audioPath: "long_audio.wav",
    decodeOptions: options
)
```

## Platform-Specific Guidance

### iOS Memory Limits

iOS has stricter memory limits than macOS:

<Tabs>
  <Tab title="iPhone 11 and earlier">
    ```swift theme={null}
    // Use tiny or base models only
    let config = WhisperKitConfig(
        model: "tiny",
        computeOptions: ModelComputeOptions(
            audioEncoderCompute: .cpuAndGPU,
            textDecoderCompute: .cpuOnly
        ),
        prewarm: true
    )
    ```
  </Tab>

  <Tab title="iPhone 12-14">
    ```swift theme={null}
    // Small models work well
    let config = WhisperKitConfig(
        model: "small",
        computeOptions: ModelComputeOptions(
            audioEncoderCompute: .cpuAndNeuralEngine,
            textDecoderCompute: .cpuAndNeuralEngine
        ),
        prewarm: true
    )
    ```
  </Tab>

  <Tab title="iPhone 15+">
    ```swift theme={null}
    // Can handle medium models
    let config = WhisperKitConfig(
        model: "medium",
        computeOptions: ModelComputeOptions(
            audioEncoderCompute: .cpuAndNeuralEngine,
            textDecoderCompute: .cpuAndNeuralEngine
        ),
        prewarm: false  // Sufficient memory
    )
    ```
  </Tab>
</Tabs>

### macOS Memory Guidelines

<ResponseField name="8 GB RAM" type="M1/M2 Base">
  * Recommended: `small` or `distil*medium`
  * Max: `medium` with prewarming
  * Avoid: `large-v3` (may cause swapping)
</ResponseField>

<ResponseField name="16 GB RAM" type="M1 Pro/M2 Pro">
  * Recommended: `medium` or `distil*large-v3`
  * Max: `large-v3` comfortably
  * Concurrent workers: 8-16
</ResponseField>

<ResponseField name="32+ GB RAM" type="M1 Max/M2 Max/M3 Max">
  * Use any model including `large-v3`
  * Multiple instances possible
  * Concurrent workers: 16+
</ResponseField>

## Background Execution

### App Extensions

```swift theme={null}
// In app extension (widget, share extension, etc.)
import WhisperKit

// Use smallest model and enable prewarming
let config = WhisperKitConfig(
    model: "tiny",
    computeOptions: ModelComputeOptions(
        melCompute: .cpuOnly,
        audioEncoderCompute: .cpuOnly,
        textDecoderCompute: .cpuOnly,
        prefillCompute: .cpuOnly
    ),
    prewarm: true
)

let pipe = try await WhisperKit(config)
```

### Background Tasks

```swift theme={null}
import BackgroundTasks
import WhisperKit

func scheduleTranscription() {
    let request = BGProcessingTaskRequest(identifier: "com.app.transcribe")
    request.requiresNetworkConnectivity = false
    request.requiresExternalPower = false  // Battery-friendly
    
    try? BGTaskScheduler.shared.submit(request)
}

BGTaskScheduler.shared.register(forTaskWithIdentifier: "com.app.transcribe") { task in
    task.expirationHandler = {
        // Clean up
        pipe = nil
    }
    
    Task {
        // Use small model for background
        let pipe = try await WhisperKit(WhisperKitConfig(
            model: "tiny",
            prewarm: true
        ))
        
        let result = try await pipe.transcribe(audioPath: "audio.wav")
        // Save result
        
        task.setTaskCompleted(success: true)
    }
}
```

## Troubleshooting Memory Issues

<AccordionGroup>
  <Accordion title="App crashes on model load">
    **Solutions:**

    * Enable prewarming: `prewarm: true`
    * Use smaller model: `tiny` or `base`
    * Switch to CPU-only: `computeOptions` with `.cpuOnly`
    * Close other apps to free memory
  </Accordion>

  <Accordion title="Memory warnings during transcription">
    **Solutions:**

    * Reduce concurrent workers: `concurrentWorkerCount = 1`
    * Enable VAD chunking: `chunkingStrategy = .vad`
    * Process files sequentially instead of in parallel
    * Use clip timestamps to segment long audio
  </Accordion>

  <Accordion title="Slow performance after first transcription">
    **May be memory pressure causing throttling:**

    * Monitor with Xcode Instruments
    * Explicitly release unused references
    * Consider smaller model or reduced worker count
  </Accordion>

  <Accordion title="Models recompiling frequently">
    **Cache being evicted:**

    * Check available disk space (cache requires \~2x model size)
    * Verify OS version (cache behavior varies)
    * Consider bundling pre-compiled models (advanced)
  </Accordion>
</AccordionGroup>

## Best Practices Summary

<Steps>
  <Step title="Profile first">
    Use Xcode Instruments to establish baseline memory usage
  </Step>

  <Step title="Choose appropriate model">
    Match model size to device capabilities and requirements
  </Step>

  <Step title="Enable prewarming on mobile">
    Always use `prewarm: true` on iOS devices
  </Step>

  <Step title="Optimize compute units">
    Balance performance and memory with appropriate compute units
  </Step>

  <Step title="Limit concurrency">
    Don't exceed recommended worker counts for platform
  </Step>

  <Step title="Use chunking">
    Enable VAD for long audio files
  </Step>

  <Step title="Monitor in production">
    Log memory metrics and watch for pressure warnings
  </Step>

  <Step title="Test on oldest devices">
    Ensure app works on minimum supported hardware
  </Step>
</Steps>

## Next Steps

<CardGroup cols={2}>
  <Card title="Performance Optimization" icon="gauge-high" href="/advanced/performance-optimization">
    Optimize speed and quality
  </Card>

  <Card title="Custom Models" icon="brain" href="/advanced/custom-models">
    Deploy optimized custom models
  </Card>
</CardGroup>