CyberFreezeDev
Loading
Rent a model per token, or run an open-weight model on hardware you own.
Cloud APIs suit top quality and modest volumes. On-premise suits data that cannot leave, offline use or steady high volume. Many businesses use both.
Use a cloud API when you need the strongest reasoning and writing quality available, your volumes are modest or unpredictable, and the data you send can leave your premises under the provider's terms. It is the fastest way to prove an idea works.
Use an on-premise LLM when the data cannot leave your network (patient records, client books of accounts, legal files), when you need the system to work without internet, or when steady high volume makes per-token billing more expensive than owning a GPU server.
In the on-premise LLM vs cloud API decision, many businesses end up with both: a local model for sensitive documents and routine extraction, and a cloud model for the occasional hard task. We design and install the local side through our private AI deployment service.
| Compared on | On-premise LLM | Cloud AI API |
|---|---|---|
| Where data goes | Stays on your server or in your private cloud | Sent to the provider for processing under its data terms |
| Models available | Open-weight models such as Llama, Qwen, Mistral and Gemma | The provider's own models, including its most capable ones |
| Quality on hard reasoning | Good for extraction, classification, summaries and drafting; behind the top closed models on the hardest tasks | Strongest available on complex reasoning, long documents and code |
| Cost structure | Upfront hardware, power, space and maintenance; near zero per extra request | Pay per input and output token; nothing upfront |
| Scaling | Limited by your GPUs; more load means more hardware | Scales on demand within the provider's rate limits |
| Works offline | Yes | No |
| Latency | Predictable on a local network | Depends on internet and provider load |
| Customisation | Full: fine-tune weights, change quantization, pin a version forever | Prompting, provider fine-tuning where offered, model versions retired on the provider's schedule |
| Who maintains it | You or your vendor: drivers, serving software, updates | The provider |
Cloud pricing models: OpenAI API pricing, Anthropic pricing, Gemini API pricing.
The largest closed models are still ahead on reasoning, careful writing and long documents.
The largest closed models are still ahead on multi-step reasoning, careful writing and long-context work. If the task is drafting a contract clause or analysing a 200-page tender, a cloud model will usually do it better than anything that fits on a single office GPU.
An API key and a few lines of code can test an idea in a day.
An API key and a few lines of code are enough to test an idea in a day. There is no hardware lead time and no capacity planning, and pricing models with batch processing and prompt caching discounts can bring costs down for the right workloads.
Records on your own hardware avoid questions about processors, transfers and retention.
When the data is a clinic's patient notes, a CA firm's client ledgers or a manufacturer's drawings, keeping it on your own hardware removes a whole category of questions about processors, cross-border transfer and retention. That is simpler to explain to clients and auditors under the DPDP Act 2023, UK GDPR or GDPR.
A GPU server is a fixed cost that does not grow with every token.
If you classify thousands of documents every day, the per-token bill grows with every one. A GPU server is a fixed cost that does not grow with use until it runs out of capacity.
A model you host does not change or get retired on a provider's schedule.
A model you host does not change behind your back. Cloud providers retire model versions on their own schedule, and prompts tuned for one version sometimes need rework on the next.
We describe the cost shape rather than quote prices that change every quarter.
Cloud APIs bill per token. On-premise cost is mostly fixed hardware, sized by GPU memory, and the break-even depends on your real volumes.
Cloud APIs bill per token, with separate rates for input (what you send) and output (what the model writes), and much higher rates for the most capable models. Most providers also offer lower rates for batch jobs that can wait and for repeated prompt prefixes that can be cached. See the OpenAI, Anthropic and Gemini pricing pages. Your bill is roughly requests times tokens per request times rate, so long documents and chatty agents cost more than short classifications.
On-premise costs are mostly fixed: the GPU server or workstation, electricity and cooling, rack or office space, and engineering time to install and maintain the serving stack (llama.cpp, Ollama and vLLM are the common choices, depending on the load). The main sizing question is GPU memory. As a rough guide, an 8 GB GPU like the RTX 3070 Ti laptop card in our lab runs 7 to 8 billion parameter models at 4-bit quantization; 24 GB cards handle models in the 30 billion range; 70 billion parameter models need around 40 GB or more at 4-bit, usually across two GPUs. Our GPU servers and AI hardware service sizes this to your workload.
The break-even is not a universal number. It depends on how many tokens you process per month, which cloud model you would otherwise use, and whether a smaller local model is good enough for the task. We work it out from your real volumes during a consultation.
Prove the task on a cloud API, test open-weight models on the same samples, then route sensitive data locally and hard, non-sensitive tasks to the cloud.
Our usual path has three steps.
We avoid sending regulated data to a cloud API just because it is convenient, and we avoid buying a GPU server before anyone has shown the task works. Our local LLMs lab track is where we test models on real hardware before recommending them, and RAG vs fine-tuning covers the next question most teams ask.
Major providers state in their commercial terms that API data is not used for training by default, and several offer reduced retention on request. Read the current terms of the provider you choose, and check where the data is processed, before you send personal data.
For extraction from invoices and forms, classification, summaries, drafting replies and answering questions over your own documents, current open-weight models in the 7 to 30 billion parameter range are often good enough. For complex reasoning or long legal analysis, test carefully against a cloud model before deciding.
Not by itself. The Act sets duties around consent, purpose, security and breach notice, and allows the government to restrict transfers to specific countries. Whether a particular cloud API is acceptable depends on the data, your contracts and your clients' expectations. Take legal advice for regulated data; a local model avoids the question.
For a pilot, a single workstation with one modern NVIDIA GPU is usually enough. For a team using it all day, a server with one or more data-centre or workstation GPUs. We size it from your expected users, document volumes and the model that passed your tests.
Last updated