CyberFreezeDev
Loading
Open-weight models on your own hardware. Prompts, files and answers never leave your network.
A ChatGPT-style assistant whose model, data and logs all live on machines you control.
An open-weight model runs on your server, so nothing goes to an outside provider and heavy use does not raise a per-token bill.
Private AI, or on-premise LLM deployment, means the language model runs on hardware you control instead of a provider's cloud. When your team pastes a contract, a patient note or a GST working into a hosted chatbot, that text travels to someone else's servers. For a lot of work that is fine. For some of it, a board, an auditor or a client will ask where the data went, and "a US provider's API" is not the answer they want.
Private AI deployment removes the question. We take an open-weight model, meaning a model whose trained weights are published for anyone to download and run, and install it on a server in your office, your data centre or a cloud region you pick. Your staff use it through a web chat, an API or the tools they already work in. Nothing is sent to OpenAI, Anthropic, Google or anyone else unless you decide it should be.
The server answers questions, drafts documents, summarises files, extracts fields from PDFs and runs agents, all against your own data. You pay for the hardware or the rented GPU, not per token, so heavy internal use does not grow the bill. You also own the uptime, which is why sizing, monitoring and a clear runbook are part of every deployment we do.
This is one of the main things our engineers in Chennai build, and it is where most of our lab work on local models goes.
We will tell you in the first call if a hosted API is the better fit.
Clinic records, client books, HR files and legal drafts kept under your control.
Clinic records, client books at a CA firm, HR files, legal drafts, product designs. Anything a client contract or a regulator says must stay under your control.
Steady, heavy internal use where a fixed server cost beats per-token billing.
Hundreds of staff querying a knowledge base all day, or a nightly batch that classifies thousands of documents. A fixed server cost beats a per-token bill once usage is steady.
Factories and branches where the model must keep working when the line drops.
Factories, warehouses and branch offices where the model has to keep working when the line drops.
Where top hosted models are still ahead, we design a private and hosted hybrid.
The largest hosted models are still ahead of open-weight models on some hard reasoning and long-document tasks. If that gap matters for your use case, we will say so and design a hybrid: private for sensitive data, hosted for the rest.
A few questions a week leave a GPU idle; a hosted API is cheaper.
If ten people ask a few questions a week, a GPU server sits idle. A hosted API with a data processing agreement is cheaper and simpler.
Memory for the weights is roughly parameters multiplied by bytes per parameter. Quantization lowers the bytes.
| Model size | 16-bit (FP16/BF16) | 8-bit (Q8_0) | 4-bit (Q4_K_M) | Practical GPU for 4-bit with an 8K context |
|---|---|---|---|---|
| 7 to 8B parameters | about 16 GB | about 8.5 GB | about 5 GB | One 8 to 12 GB card |
| 14B | about 30 GB | about 16 GB | about 9 GB | One 16 GB card |
| 32B | about 66 GB | about 35 GB | about 20 GB | One 24 GB card, or 32 GB for longer context |
| 70B | about 141 GB | about 75 GB | about 43 GB | Two 24 GB cards or one 48 GB card, short context only; 64 GB or more for long context |
Weights only, worked from the bits per weight that llama.cpp publishes for its quantization types (Q4_K_M is about 4.9 bits, Q8_0 about 8.5 bits, F16 is 16). Add the KV cache and about 1 GB of runtime overhead on top. See the Hugging Face quantization overview for the other methods.
Model weights are only part of the memory bill. Context length and concurrent users decide the rest.
Long contexts and concurrent users need memory beyond the model weights, so we size from users, context and latency, not the model name.
Every token a model is holding in its context needs space in the KV cache. For Llama 3.1 8B that is about 128 KB per token at 16-bit, so an 8,000 token conversation adds about 1 GB. For a 70B model of the same family it is about 320 KB per token: roughly 2.5 GB at 8,000 tokens and roughly 40 GB at the full 128,000. Ten people chatting at once multiplies that again.
So we size from your workload, not from the model name:
We then pick the smallest model that passes your evaluation set (more on that below), quantize it, and choose a GPU with headroom. For the hardware itself, see GPU servers and AI hardware builds. For the terms used here, see our glossary entries on VRAM and quantization.
Licence, language and task fit matter more than leaderboard rank.
Every model's licence read first and recorded in your runbook.
Read the licence on the model card before anything else. Many Qwen and Mistral releases use Apache 2.0, Phi and DeepSeek R1 use MIT, Llama uses Meta's community licence with its own conditions, and Gemma uses Google's Gemma terms. We record the licence of every model we deploy in your runbook.
Tamil, Hindi or Arabic tested directly, since coverage varies widely between models.
If staff or customers write in Tamil, Hindi or Arabic, we test those languages directly. Coverage varies a lot between families and sizes, and a model that is strong in English can be weak in Tamil.
A tuned 14B model can beat a general 70B on a narrow task.
Field extraction from invoices, answering from a policy manual, drafting replies and writing SQL are different jobs. A 14B model tuned for instructions can beat a 70B general model on a narrow task, at a quarter of the hardware.
Every candidate scored on 50 to 200 real questions from your documents.
We build 50 to 200 real questions from your documents with the answers your team expects, and score every candidate model against it. That set stays with you and is rerun whenever a model is swapped or updated.
Running the model yourself does not make you compliant on its own, but it makes the data map short and easy to explain.
A server in India, the UK or the Gulf keeps personal data in that country. We document the data flows; your legal adviser decides.
India. The Digital Personal Data Protection Act 2023 and the DPDP Rules notified in November 2025 make you, as the data fiduciary, answerable for how personal data is processed, including by vendors. The Act does not impose a general localisation rule: Section 16 lets the government restrict transfers to specific countries by notification, and most of the Act's operational duties are being phased in, with the transfer provisions expected to apply from May 2027 (summary by King Stubb and Kasiva). Sector rules can be stricter: payment system data, for example, must be stored in India under RBI rules. A model on a server in India means prompts and documents containing personal data never leave the country, so a future restriction or a client's contract clause does not force a rebuild.
United Kingdom. Under UK GDPR, sending personal data to a provider outside the UK is a restricted transfer that needs adequacy regulations or a safeguard such as the International Data Transfer Agreement (ICO guidance on international transfers). A model running on your UK server, or in a UK cloud region you control, keeps inference out of that analysis.
Gulf. The same design works for teams in the UAE, Saudi Arabia and Qatar who want data held in-country.
What we deliver for your records: a diagram of where data flows, which systems store prompts and logs and for how long, who can read them, and how to delete a person's data on request. Your legal adviser decides what that means for your obligations; we make sure the technical answer is clear.
Tasks, documents, users, peak load and location limits, summed up in a one-page scope.
We list the tasks, the documents involved, who uses it, peak load and any data that must not leave a location. Output: a one page scope and a sizing estimate.
Three to five open-weight models scored on your evaluation set, side by side.
We build your evaluation set and test three to five open-weight models at different sizes and quantizations, on our lab machine for small models and on rented GPUs for large ones. You see the scores and sample answers side by side.
Buy, rent or private cloud, with memory, power and cost reasoning written down.
Buy, rent a dedicated GPU server, or use a private cloud GPU in a region you choose. We give you the options with the memory, power and running cost reasoning written down.
Drivers, serving engine and chat front end behind your firewall with TLS and sign-in.
Linux, NVIDIA drivers, CUDA, the serving engine and the chat front end, behind your firewall with TLS, sign-in and per-team access. No inbound access from the internet unless you ask for it.
Document search over your files and connections to n8n, Desk, WhatsApp or your systems.
Document search over your files (retrieval-augmented generation), connections to n8n, Desk, WhatsApp or your own systems, and agents where they help.
One department uses it for two to four weeks while we tune it.
One department uses it for two to four weeks. We watch response times, wrong answers and what people actually ask, then adjust the model, prompts and document scopes.
Runbook, monitoring, configuration backups and a procedure for swapping in newer models.
Runbook, monitoring, backup of configuration and indexes, and the procedure for swapping in a newer model. Ongoing support is available if you want us on call.
Staff ask about SOPs and policies and get answers linked to the source page.
Staff ask questions of SOPs, policy manuals, past proposals and product sheets, and get answers with links to the source page.
Invoices, statements and KYC forms turned into rows, with low-confidence fields reviewed.
Invoices, purchase orders, bank statements and KYC forms turned into structured rows, with a person reviewing low-confidence fields. Pairs with our OCR and document intelligence work.
Replies, meeting notes and contract comparisons drafted without text leaving the building.
Replies, meeting notes, audit working paper summaries and contract clause comparisons, without the text leaving the building.
Lookups, tasks and draft invoices through approved tools, with a person confirming record changes.
A model that can look up a customer, create a task or draft an invoice through tools you approve, with a person confirming anything that changes a record. See custom AI systems.
For focused business tasks such as answering from your documents, extracting fields and drafting routine text, a well chosen 8B to 32B open-weight model often does the job. For the hardest open-ended reasoning, the largest hosted models are still stronger. We test on your own questions before you buy anything, so you decide from evidence. Our on-premise LLM vs cloud API comparison sets out the trade-offs in full.
A 7 or 8B model at 4-bit fits in about 5 GB, so a desktop or laptop with an 8 GB NVIDIA GPU can run it for one or two users. Our own lab machine has an 8 GB laptop GPU. For a team, plan on a workstation or server with 24 GB or more.
No single system does that. It removes the cross-border transfer and third-party processor questions for the AI step, and we document the data flows. Consent, notices, retention and rights requests are still your processes, and your legal adviser should confirm the overall position.
The serving layer exposes an OpenAI-compatible API, so your apps do not change. We rerun your evaluation set on the new model, and if it scores better and fits the hardware, we swap it in using the procedure in your runbook.
Yes, but most teams do not need to start there. Retrieval over your documents solves the majority of "it should know our policies" requests. Fine-tuning helps when you need a fixed output format, a house writing style or a narrow classification task, and we recommend it only after retrieval has been tried.
Last updated