There is no single correct answer to “How much RAM does local AI need?” Memory use depends on model size, precision or quantization, context length, runtime, and operating system. You can still estimate it in a useful way.
The model file is only the starting point
If a quantized model file is several gigabytes, that does not mean exactly the same amount of RAM is sufficient. The runtime needs additional memory for context, intermediate data, caches, and its own structures. The operating system and other applications also need headroom.
Consider parameter count and quantization together
A 7B model in compact Q4 quantization uses much less memory than the same model at higher precision. A 14B model can still require more memory than a smaller model even after quantization.
Context length can increase memory use substantially
The more text kept in the active context, the more runtime memory may be required. Very long contexts are therefore not only a model capability but also a hardware consideration.
Practical ranges
Rather than using one hard minimum, it is more useful to think in ranges:
- 8 GB RAM: small quantized models and short contexts can work, but headroom is limited.
- 16 GB RAM: a practical starting point for many compact local models and normal desktop use.
- 24–32 GB RAM: substantially more room for medium models, longer contexts, and parallel applications.
- 64 GB or more: useful for larger models, heavier contexts, and more demanding local workflows.
These ranges are not guarantees. A particular model can require more or less depending on runtime and hardware.
On Macs, Unified Memory matters
Apple Silicon uses one shared memory pool for CPU and GPU. Sixteen gigabytes of Unified Memory therefore does not represent the same architecture as 16 GB of system RAM plus 8 GB of separate VRAM in a Windows PC.
On Windows, RAM and VRAM are separate
With a discrete GPU, much of a model may live in VRAM while Windows and other components use system memory. If the model does not fully fit on the GPU, the runtime may offload parts to RAM. That can work, but it may be slower.
Enough memory does not guarantee speed
A model can fit entirely in memory and still respond slowly. Memory primarily answers does it fit?; compute performance and memory bandwidth have a larger influence on how fast does it run?
Leave headroom
If loading the model already consumes almost all available memory, a few browser tabs or a large document may trigger swapping or severe slowdowns. A comfortable configuration should not sit right at the theoretical limit.
Priviot and model selection
When local models are used, Priviot should treat model size together with the device's available memory and selected quantization. The Local AI section is therefore focused on understandable hardware trade-offs rather than model names alone.
Conclusion
Local AI needs more than the raw model-file size. Account for the model, context, runtime, operating system, and headroom together. Sixteen gigabytes is a useful starting point for many compact models, while larger models and longer contexts benefit significantly from 24, 32, or more gigabytes.