Best GPU for a local LLM: how to choose by VRAM

The best GPU for a local LLM is the one with enough memory (VRAM) to hold your model and its context. After that, memory bandwidth decides how fast it writes. As a rough guide at 4-bit: a 16 GB card runs models up to about 14 billion parameters, 24 to 32 GB runs 32B models, and a 70B model needs about 48 GB on one card or split over two.

Gaming speed matters much less than memory size and bandwidth.

Why does VRAM decide the choice?

A model runs at full speed only when its weights, its context and the runtime all fit in the card's memory. If they do not, Ollama and llama.cpp move some layers to system RAM; ollama ps then shows a CPU/GPU split, and replies slow down.

So start from the model, not the card. Ollama's 4-bit downloads give the size of the weights:

Add room for the conversation, which grows with context length and with the number of people using it at once. Our guide to how much VRAM an LLM needs has the formula and worked examples, and our glossary explains VRAM.

What does memory bandwidth add?

When a model writes, each new token needs its weights read from memory again. Arm's documentation gives the rule: tokens per second is roughly memory bandwidth divided by the bytes read per token. Two cards with the same memory can therefore differ a lot in speed.

NVIDIA's GeForce pages list the memory bus rather than bandwidth: the RTX 5090 has a 512-bit bus and the RTX 5060 Ti a 128-bit one, both with GDDR7. Data centre pages list bandwidth directly: 864 GB/s for the L40S and 4.8 TB/s for the H200. NVIDIA's DGX Spark desktop takes the other route, with 128 GB of unified memory that holds models up to 200 billion parameters but at 273 GB/s, so large models fit but write more slowly than on a fast discrete card.

Consumer, workstation or data centre card?

Memory and power from NVIDIA's spec pages. The last column is our arithmetic from the table above, not a benchmark.

How they differ in practice:

• Consumer cards are sold at retail and suit one person or a pilot. Power is high for the memory you get: the RTX 5090 needs a 1,000 W power supply for the whole system.
• Workstation cards list ECC memory and draw much less power: the RTX PRO 4000 Blackwell has 24 GB at 145 W. They suit an office machine that runs all day.
• Data centre cards go into rack servers. The L40S has no fans of its own and relies on the server's airflow, so it does not belong in a tower under a desk.

Check software support before you buy outside NVIDIA. Ollama supports AMD Radeon cards through ROCm v7 on Linux and Windows, and Apple M series through Metal.

Should you use two GPUs?

Two cards add their memory together: two 24 GB cards give 48 GB, enough for a 70B model at 4-bit with a short context. Ollama spreads a model across all available GPUs when it does not fit on one. vLLM uses tensor parallelism across GPUs in one server; its documentation recommends pipeline parallelism instead for cards without NVLink, such as the L40S.

The costs are power, heat, slots and complexity. Where one card with enough memory exists, it is usually the simpler build. For which server software suits several cards and many users, see Ollama vs vLLM vs llama.cpp.

Buying a GPU in India

• Prices. NVIDIA's India product pages showed no rupee prices when we checked on 10 November 2026, so get dated quotes and compare the same card and warranty across sellers.
• Workstation cards. NVIDIA's where-to-buy page sends buyers to OEM workstations (Dell, HP, HPE, Lenovo, Supermicro and others), to channel partners such as PNY, Leadtek and Ingram Micro, and to the NVIDIA Marketplace.
• Data centre cards are bought inside a server, so plan the rack, cooling and power with the card.
• Power backup. Size the UPS for the whole machine, not just the card's rated power.
• Laptops. Check the GPU memory, not the model name. Our lab's RTX 3070 Ti laptop GPU has 8 GB, enough for an 8B model at 4-bit with a modest context.
• Rent first. A rented cloud GPU for a week lets you test your exact model before you buy. Our comparison of paying per token or running your own server helps with the longer-term sums.

Choosing a GPU for your team

For a team, the number of people using the model at once and the length of their documents usually decide the card. Our GPU servers and AI hardware service works from your models and users to a build or a rental, then sets it up and hands over a runbook. Book a consultation.