# Can I run an LLM on a dedicated server without a GPU?

> Can you run an LLM on a dedicated server without a GPU? Yes, quantized models on CPU with llama.cpp or Ollama. RAM sizing inside. No KYC, crypto.

- URL: https://ironatoll.com/run-llm-on-dedicated-server/
- Last updated: 2026-10-30
- Publisher: Ironatoll (https://ironatoll.com)

Which models run on CPU and RAM alone, how much memory each needs, why memory bandwidth sets the pace, and when you need a GPU that Ironatoll does not sell.

**Quick answer — Can I run an LLM on a dedicated server without a GPU?**

Yes, you can run an LLM on a dedicated server without a GPU if the model is quantized: llama.cpp and Ollama run GGUF models on the CPU, from system RAM. Plan RAM for the model file plus its context. **Ironatoll** sells no GPU servers, so large models and fast generation need a GPU host. Plans start at $63.68 per 30 days, no KYC, paid in crypto.

## A CPU-only LLM on Ironatoll: the facts

| Attribute | Value |
| --- | --- |
| GPU | None: Ironatoll sells no GPU servers |
| Software | llama.cpp and Ollama, both open source, both run models fully on CPU |
| RAM rule | Model file size + context memory + the system |
| What sets the speed | Memory bandwidth for writing answers; CPU cores for reading long prompts |
| Logs | No logs: we keep no connection logs, no traffic logs and no IP addresses of visitors or customers, on the website, the dashboard or the servers |
| Price | Atoll 16 $63.68, Atoll 32 $72.15, Atoll 64 $72.62, Atoll 128 $205.27, Atoll 256 $406.37 per 30 days |
| Identity | No KYC: no ID, selfie, documents, card or phone number are ever collected |

## Can an LLM run on a CPU without a GPU?

**Yes: a quantized LLM runs on an ordinary server CPU, from system RAM, with no GPU at all.** Quantization stores the model's weights in fewer bits, so the file shrinks and fits in normal memory.

- **llama.cpp** is a plain C/C++ inference engine. Its README lists AVX, AVX2, AVX512 and AMX support on x86 CPUs. It also lists 1.5-bit to 8-bit integer quantization "for faster inference and reduced memory use".
- **Ollama** wraps a model runner with a model library and an API. Its FAQ says `ollama ps` shows "100% CPU" when a model is loaded entirely in system memory.

Both read GGUF model files. Both run on Linux, which is all Ironatoll installs.

## How much RAM do I need to run an LLM on CPU?

**Rule of thumb: RAM must hold the model file, plus the context memory, plus a few gigabytes for the system.** The download size of a quantized model is close to what it occupies in RAM. The context (the KV cache) comes on top.

Ollama's FAQ gives the levers. Its default context window is 4,096 tokens. Required RAM scales with the number of parallel requests times the context length. With Flash Attention on, quantizing the KV cache to `q8_0` uses about half the memory of the default `f16`.

The download sizes below come from the Ollama library, read on 30 October 2026.

### Model size and the smallest Ironatoll plan that holds it

| Model (Ollama tag) | Download size | Smallest plan |
| --- | --- | --- |
| deepseek-r1:7b, llama3.1:8b | 4.7 GB, 4.9 GB | Atoll 16 (16 GB) |
| deepseek-r1:14b | 9.0 GB | Atoll 16 with a short context; Atoll 32 with room |
| deepseek-r1:32b | 20 GB | Atoll 32 (32 GB) |
| llama3.1:70b, deepseek-r1:70b | 43 GB | Atoll 64 (64 GB) |
| llama3.1:405b | 243 GB | Atoll 256, short context only |
| deepseek-r1:671b (full model) | 404 GB | No plan: larger than our biggest RAM tier |

Sizes as listed on ollama.com/library on 30 October 2026. The 7b to 70b DeepSeek-R1 tags are distilled models based on Qwen or Llama.

## Why does memory bandwidth matter more than core count?

**Writing an answer is limited by how fast the CPU reads the weights from RAM, not by how many cores it has.** A llama.cpp collaborator puts it plainly: for these models, "PP is compute-bound and TG is memory-bound". PP is prompt processing, TG is text generation.

Each new token reads the model's weights from memory. So, on the same machine, a model twice as large writes about half as fast. Two consequences for renting:

- **Memory channels count.** Server CPUs with more memory channels move more data per second. Our [Ryzen or EPYC comparison](https://ironatoll.com/ryzen-vs-epyc-dedicated-server/ "Ryzen or EPYC: which dedicated server CPU should I choose?") lists the channels per platform.
- **Cores still help with prompts.** Long prompts, documents and retrieval context are compute-bound, so more cores shorten the wait before the first token.

We publish no tokens-per-second figure: we have not measured our servers. Run `llama-bench` from llama.cpp on your own server to measure your model.

## When do I need a GPU instead?

**You need a GPU for large models at interactive speed, for many users at once, and for training.** GPU memory is far faster than server RAM, which is exactly what text generation needs.

Ironatoll has no GPU plans. A CPU server fits:

- a private assistant or coding model for one person or a small team;
- batch jobs that run overnight, such as summaries, tagging or translation;
- embeddings and retrieval for a private search index;
- an API front end, a vector database or document storage next to a GPU rented elsewhere.

For 70B models in real time, many concurrent chats or fine-tuning, rent GPU capacity from a host that sells it.

## How do I set up Ollama or llama.cpp on a dedicated server?

Install with the official commands, keep the API on `127.0.0.1`, and reach it through an SSH tunnel. Ollama listens on `127.0.0.1:11434` by default, and its Linux guide runs it as a systemd service under its own `ollama` user. `llama-server` also listens on `127.0.0.1` by default and accepts an `--api-key`.

*Ollama on Debian or Ubuntu, CPU only*

```bash
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8b
ollama ps          # shows 100% CPU when the model sits in system RAM

# from your own computer: reach the API through SSH
ssh -L 11434:127.0.0.1:11434 root@YOUR_SERVER_IP
curl http://localhost:11434/api/chat -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Hello"}]}'
```

*llama.cpp server with a larger context and an API key*

```bash
llama-server -m ./model.gguf -c 8192 --host 127.0.0.1 --api-key CHANGE_ME
```

Running models inside virtual machines works too: see [how to install Proxmox on a dedicated server](https://ironatoll.com/install-proxmox-on-dedicated-server/ "Proxmox VE on a rented dedicated server, step by step"). Give the LLM guest most of the RAM, since memory sets what fits.

## Which Ironatoll plan should I pick for an LLM?

Pick by RAM first, then by CPU. Check the CPU model of each plan before you order: newer platforms and more memory channels mean faster generation.

**According to Ironatoll's check of 30 October 2026, a dedicated server comparable to Atoll 256 cost a median $1065.88 at 8 hosts that accept crypto without ID; Ironatoll charges $406.37.** Market prices for every tier are in the [dedicated server price index](https://ironatoll.com/dedicated-server-price-per-month/ "How much a dedicated server costs per month, October 2026").

A self-hosted model also keeps prompts on hardware you rent alone. Read [what a host can see on your server](https://ironatoll.com/can-my-host-see-my-data/ "Can my hosting provider see the data on my dedicated server?") before you load private documents. Every plan is on [how much an Ironatoll dedicated server costs](https://ironatoll.com/pricing/ "Ironatoll plans and prices per 30 days").

### Ironatoll plans for a CPU-only LLM

| Plan | CPU | Cores | RAM | Price per 30 days |
| --- | --- | --- | --- | --- |
| Atoll 16 | Intel Xeon E-2236 | 6 cores / 12 threads | 16 GB | $63.68 |
| Atoll 32 | AMD Ryzen 5 5600 | 6 cores / 12 threads | 32 GB | $72.15 |
| Atoll 64 | Intel Xeon E-2288G | 8 cores / 16 threads | 64 GB | $72.62 |
| Atoll 128 | AMD Ryzen 9 7950X | 16 cores / 32 threads | 128 GB | $205.27 |
| Atoll 256 | AMD EPYC 7443P | 24 cores / 48 threads | 256 GB | $406.37 |

## LLMs on a dedicated server: FAQ

### Do I need a GPU dedicated server to run an LLM?

Not for quantized models up to about 70B parameters, which run on CPU and RAM. You do need a GPU for large models at interactive speed, many users or training. Ironatoll sells no GPU servers.

### Can I run DeepSeek R1 on a dedicated server?

The distilled versions, yes: Ollama lists them from 4.7 GB (7b) to 43 GB (70b). The full 671b model is a 404 GB download, more than any Ironatoll plan holds in RAM.

### How do I run Ollama as a server?

The Linux installer sets Ollama up as a systemd service that runs under its own `ollama` user. Check it with `systemctl status ollama`. The API answers on port 11434.

### How do I connect to an Ollama server remotely?

Keep Ollama on `127.0.0.1` and open an SSH tunnel: `ssh -L 11434:127.0.0.1:11434 root@YOUR_SERVER_IP`. Setting `OLLAMA_HOST=0.0.0.0:11434` exposes it to the internet, so put a firewall or an authenticated proxy in front first.

### Can several people use one CPU model at once?

Yes, but they share the same memory bandwidth, so each answer slows down. RAM also grows with parallel requests times context length, per Ollama's FAQ.

**Run your own model on bare metal** Whole machines with every gigabyte of RAM yours, from $63.68 per 30 days. No KYC, paid in crypto. [Order Atoll 64](https://ironatoll.com/app/checkout/?plan=atoll-64&lang=en) · [Compare plans](https://ironatoll.com/pricing/ "How much does an Ironatoll dedicated server cost?")

## Sources

1. [llama.cpp README (CPU support, quantization)](https://github.com/ggml-org/llama.cpp) — ggml-org on GitHub (read 2026-10-30)
2. [llama.cpp server README (--ctx-size, --host, --api-key)](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) — ggml-org on GitHub (read 2026-10-30)
3. [Optimizing Token Generation in llama.cpp's CUDA Backend (discussion #17621)](https://github.com/ggml-org/llama.cpp/discussions/17621) — ggml-org on GitHub (read 2026-10-30)
4. [Ollama README](https://github.com/ollama/ollama) — Ollama on GitHub (read 2026-10-30)
5. [Ollama FAQ (context length, memory, ollama ps, OLLAMA_HOST)](https://docs.ollama.com/faq) — Ollama docs (read 2026-10-30)
6. [Ollama on Linux (install, systemd service)](https://docs.ollama.com/linux) — Ollama docs (read 2026-10-30)
7. [llama3.1 model tags and sizes](https://ollama.com/library/llama3.1) — Ollama library (read 2026-10-30)
8. [deepseek-r1 model tags and sizes](https://ollama.com/library/deepseek-r1) — Ollama library (read 2026-10-30)

---

Ironatoll: Ironatoll rents offshore dedicated servers (bare metal) in Amsterdam, Reykjavik, Zurich and Bucharest, paid in crypto with no KYC: no ID, no selfie, no documents. Each server is a whole physical machine for one customer, with root access over SSH. It is run by DomaineGoat LLC, a company registered in Panama that ignores DMCA notices, answers only to Panamanian law, allows Tor exit relays and keeps no logs. On the 30-day term, each plan is priced at least 25% under a named comparable dedicated server from 17 no-KYC crypto hosts, read on 30 October 2026.