GGUF vs AWQ vs GPTQ: LLM quantization explained

GGUF is a file format, used by llama.cpp and Ollama, that can hold a model at many bit widths, from 2-bit to 16-bit. AWQ and GPTQ are 4-bit quantisation methods whose checkpoints GPU servers such as vLLM and Hugging Face Transformers load. So the first question is not which is best but which runtime you use: llama.cpp or Ollama means GGUF, and vLLM on NVIDIA GPUs usually means AWQ or GPTQ.

Quantization stores each weight in fewer bits than the 16 it was trained with, so a model needs less memory and runs on smaller cards, at some cost in quality.

What is GGUF?

GGUF is the file format of the GGML library behind llama.cpp. Its specification describes a single file that holds the weights, the tokenizer and the model's settings as key-value metadata, designed to load quickly. It replaced the older GGML, GGMF and GGJT formats.

GGUF is a container, not one method. One Hugging Face repository often offers the same model in a dozen GGUF files, from Q2_K to F16, and you pick one. llama.cpp makes them with its llama-quantize tool, and Ollama imports them through a Modelfile. Quantisation needs no training: llama.cpp can optionally use an importance matrix (imatrix), computed from sample text, to reduce the loss.

What do Q4_K_M and the other names mean?

The names come from llama.cpp. Read Q4_K_M as three parts:

• Q4: about 4 bits per weight for most tensors.
• K: a k-quant, introduced in llama.cpp pull request #1684 in June 2023. Weights are grouped into super-blocks; for Q4_K that is 8 blocks of 32 weights, each block with its own scale and minimum, for 4.5 bits per weight.
• M: the mix. Q4_K_S (small) uses Q4_K everywhere. Q4_K_M (medium) keeps half of two sensitive tensor types, attention.wv and feed_forward.w2, at 6-bit and uses Q4_K for the rest. L, where it exists, protects more.

Older names such as Q4_0 and Q8_0 are the original block formats. Names starting with IQ, such as IQ4_XS, are a later family that reach lower bit widths. F16 and BF16 are unquantised 16-bit.

File sizes for Llama 3.1 8B, from llama.cpp's own table:

What are GPTQ and AWQ?

Both are post-training methods that take a 16-bit model and write a 4-bit checkpoint, using a small set of calibration text.

GPTQ (Frantar and others, 2022, published at ICLR 2023) quantises the weights layer by layer, using approximate second-order information to correct the error each step introduces. The paper reports quantising 175 billion parameter models to 3 or 4 bits in about four GPU hours with negligible accuracy loss.

AWQ (Lin and others, 2023, MLSys 2024 Best Paper) starts from the observation that a small share of weights matter most. It finds those channels from activation statistics and scales them before quantising; the authors report that protecting only 1 percent of salient weights cuts the error a great deal. It needs no backpropagation.

A practical note: the AutoAWQ library was archived in May 2025, and the vLLM project now carries that work in llm-compressor.

Which runtime loads which format?

vLLM also serves FP8 and other formats on recent NVIDIA cards, which is worth a look for a server where 8-bit fits.

How much quality do you lose?

llama.cpp measures loss as perplexity (how surprised the model is by real text, lower is better) or as KL divergence from the 16-bit model. The k-quants pull request published these perplexities on a 7B model in 2023:

• F16: 5.9066
• Q6_K: 5.9110
• Q5_K_S: 5.9419
• Q4_K_S: 6.0215
• Q3_K_M: 6.1503
• Q2_K: 6.7764

In those figures, 6-bit is within 0.1 percent of 16-bit, 4-bit adds about 2 percent, and 2-bit adds about 15 percent. Today's models are different from that 2023 model, so treat these as a shape, not a promise. llama.cpp also warns that requantising an already quantised file can severely reduce quality; always start from the 16-bit weights.

We have not published quality tests of our own here. Perplexity also misses task-level problems, such as a model that starts to get table extraction or tool calls wrong, so test on your work.

How to choose

1. Pick the runtime first. One machine, a Mac or mixed CPU and GPU: GGUF. A Linux GPU server for many users: AWQ or GPTQ on vLLM.
2. Start at 4-bit. Q4_K_M, or a 4-bit AWQ or GPTQ checkpoint, is the common default.
3. Spend spare memory on bits. If Q5_K_M, Q6_K or Q8_0 fits with your context, use it. Our VRAM calculator shows what fits.
4. Use published checkpoints with care. Prefer the model maker's own quantised release, check the open-weight model licence, and run your own test set before staff rely on it.

Getting the model and format right for your data

Our private AI deployment work starts with this choice: which model, at which precision, on which runtime, for the documents and number of users you have. Book a consultation.