Can I run an LLM on a dedicated server without a GPU?

Can an LLM run on a CPU without a GPU?

Yes: a quantized LLM runs on an ordinary server CPU, from system RAM, with no GPU at all. Quantization stores the model's weights in fewer bits, so the file shrinks and fits in normal memory.

• llama.cpp is a plain C/C++ inference engine. Its README lists AVX, AVX2, AVX512 and AMX support on x86 CPUs. It also lists 1.5-bit to 8-bit integer quantization "for faster inference and reduced memory use".
• Ollama wraps a model runner with a model library and an API. Its FAQ says ollama ps shows "100% CPU" when a model is loaded entirely in system memory.

Both read GGUF model files. Both run on Linux, which is all Ironatoll installs.

How much RAM do I need to run an LLM on CPU?

Rule of thumb: RAM must hold the model file, plus the context memory, plus a few gigabytes for the system. The download size of a quantized model is close to what it occupies in RAM. The context (the KV cache) comes on top.

Ollama's FAQ gives the levers. Its default context window is 4,096 tokens. Required RAM scales with the number of parallel requests times the context length. With Flash Attention on, quantizing the KV cache to q8_0 uses about half the memory of the default f16.

The download sizes below come from the Ollama library, read on 30 October 2026.

Why does memory bandwidth matter more than core count?

Writing an answer is limited by how fast the CPU reads the weights from RAM, not by how many cores it has. A llama.cpp collaborator puts it plainly: for these models, "PP is compute-bound and TG is memory-bound". PP is prompt processing, TG is text generation.

Each new token reads the model's weights from memory. So, on the same machine, a model twice as large writes about half as fast. Two consequences for renting:

• Memory channels count. Server CPUs with more memory channels move more data per second. Our Ryzen or EPYC comparison lists the channels per platform.
• Cores still help with prompts. Long prompts, documents and retrieval context are compute-bound, so more cores shorten the wait before the first token.

We publish no tokens-per-second figure: we have not measured our servers. Run llama-bench from llama.cpp on your own server to measure your model.

When do I need a GPU instead?

You need a GPU for large models at interactive speed, for many users at once, and for training. GPU memory is far faster than server RAM, which is exactly what text generation needs.

Ironatoll has no GPU plans. A CPU server fits:

• a private assistant or coding model for one person or a small team;
• batch jobs that run overnight, such as summaries, tagging or translation;
• embeddings and retrieval for a private search index;
• an API front end, a vector database or document storage next to a GPU rented elsewhere.

For 70B models in real time, many concurrent chats or fine-tuning, rent GPU capacity from a host that sells it.

How do I set up Ollama or llama.cpp on a dedicated server?

Install with the official commands, keep the API on 127.0.0.1, and reach it through an SSH tunnel. Ollama listens on 127.0.0.1:11434 by default, and its Linux guide runs it as a systemd service under its own ollama user. llama-server also listens on 127.0.0.1 by default and accepts an --api-key.

curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8b
ollama ps          # shows 100% CPU when the model sits in system RAM

# from your own computer: reach the API through SSH
ssh -L 11434:127.0.0.1:11434 root@YOUR_SERVER_IP
curl http://localhost:11434/api/chat -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Hello"}]}'

llama-server -m ./model.gguf -c 8192 --host 127.0.0.1 --api-key CHANGE_ME

Which Ironatoll plan should I pick for an LLM?

Pick by RAM first, then by CPU. Check the CPU model of each plan before you order: newer platforms and more memory channels mean faster generation.

According to Ironatoll's check of 30 October 2026, a dedicated server comparable to Atoll 256 cost a median $1065.88 at 8 hosts that accept crypto without ID; Ironatoll charges $406.37. Market prices for every tier are in the dedicated server price index.

A self-hosted model also keeps prompts on hardware you rent alone. Read what a host can see on your server before you load private documents. Every plan is on how much an Ironatoll dedicated server costs.