Quantization
2 essays on this topic.
Run AI Models on Your Own Laptop: A Practical Guide
How to run open AI models locally: the memory arithmetic, picking a size for your machine, GGUF and MLX, Ollama, llama.cpp, LM Studio, and Python.
Shrinking AI: Quantization and Distillation Explained
How big AI models get small enough for a laptop or phone: quantization, distillation, pruning, what each one costs, and the licence questions.
