Skip to content

Solutions · Document chat

A VPS for AnythingLLM

AnythingLLM is a private chat workspace over your own documents: upload files, they are embedded into a built-in vector database, and you chat with them through a hosted model behind an API key or a local one through Ollama. Its docs ask for 2 GB of RAM, a 2-core CPU with AVX2 and 5 GB of disk, and call AVX2 the one hard requirement. GL2-4 at $11.90 a month is the cheapest machine that meets that floor; BD-8 at $12.90 is the one we would buy, because the embedder runs on this machine's CPU.

Deploy BD-8Compare every plan

What it needs

Software from the docs, sizes labelled by source

The software and the security posture come from the project's own documentation. Where the project publishes no hardware figure, the sizes say whose they are.

From docs.anythingllm.com and the project's Docker guide

  • “RAM 2GB, 2-core CPU with AVX2, Storage 5GB”: the docs' minimum. On a cloud machine the Docker guide says at least 2 GB of RAM and a minimum of 10 GB of disk.
  • AVX2 is “the one hard requirement”: the default vector database is built for CPUs from 2013 on, and without it the server dies on start. Check inside the machine: lscpu | grep -o avx2. If it prints nothing, set VECTOR_DB to another vector database.
  • Install: docker run -d -p 3001:3001 --cap-add SYS_ADMIN … mintplexlabs/anythingllm with a storage directory mounted at /app/server/storage. The app is on port 3001.
  • Chat needs a model: an API key for a hosted provider, or Ollama on the same or another machine. Embedding runs on this machine's CPU by default. Multi-user mode with permissions is a Docker-version feature. No GPU.

Sizes (the project's minimum, then ours)

Sizing by workload
WorkloadvCPURAM
Documents, chats, a hosted model (official minimum)Two cores with AVX2. 5 GB of disk, 10 GB on a cloud machine; the images alone take a few gigabytes.22 GB
A real document library, the built-in CPU embedder, a few usersOurs. Embedding runs on this CPU; a big upload on two cores takes a long while.48 GB
Ollama beside it for a fully local stackOurs. That is a model-hosting question, not an AnythingLLM one: size it on the Ollama page.832 GB

The plans that fit

5 machines, priced live

Cheapest fitting machine first. Prices are today's, per month, read from the catalogue. Run the AVX2 check on whichever you buy, in the first minute.

  • GL2-4$11.90/mo

    2 vCPU · 4 GB · 40 GB disk · Unmetered

    Cheapest that meets the floor, unmetered. 2 vCPU / 4 GB / 40 GB, twice the memory the docs ask for, and unmetered traffic for uploads. Run the AVX2 check before anything else.

    Deploy
  • BD-8$12.90/mo

    4 vCPU · 8 GB · 100 GB disk · 32 TB

    The one we would buy: 4 cores for the embedder. 4 vCPU, 8 GB, 100 GB and 32 TB of transfer for a dollar more. Embedding a library on four cores instead of two is the difference you feel.

    Deploy
  • BD-24$31.90/mo

    8 vCPU · 24 GB · 300 GB disk · 32 TB

    8 vCPU / 24 GB for many users and Ollama beside it. Room for an 8B model in Ollama on the same machine, slowly, and a team's documents.

    Deploy
  • C2-4-80$37.90/mo

    2 vCPU · 4 GB · 80 GB disk · 3 TB

    The floor's shape, in dozens of cities. 2 vCPU / 4 GB / 80 GB. Pick it for the city. Hosted model, modest library.

    Deploy
  • C4-8-160$75.90/mo

    4 vCPU · 8 GB · 160 GB disk · 3 TB

    4 vCPU / 8 GB / 160 GB, in dozens of cities. The comfortable size with disk for a large library and its vectors.

    Deploy

There is no start-up preset for AnythingLLM yet: the install is the five steps below, typed by you. Step 3 differs from the project's command in one flag, which binds the port to loopback, and says so.

The install

Five steps, from the project's Docker guide

Check the CPU, install Docker, run the container on loopback, set it up over SSH, then give it a domain.

  1. Check AVX2 first. lscpu | grep -o avx2 It must print avx2. The docs are clear that this is the one hardware rule and that it cannot be emulated; a machine without it needs VECTOR_DB set to an external vector database before the first start.
  2. Docker. curl -fsSL https://get.docker.com | sh
  3. Run it, on loopback. export STORAGE_LOCATION=/opt/anythingllm && mkdir -p $STORAGE_LOCATION && touch $STORAGE_LOCATION/.env docker run -d --name anythingllm --restart unless-stopped -p 127.0.0.1:3001:3001 --cap-add SYS_ADMIN --add-host=host.docker.internal:host-gateway -v $STORAGE_LOCATION:/app/server/storage -v $STORAGE_LOCATION/.env:/app/server/.env -e STORAGE_DIR="/app/server/storage" mintplexlabs/anythingllm This is the project's command with one change: -p 127.0.0.1:3001:3001 instead of -p 3001:3001, so the chat is not on the open internet before you have set a password. The storage directory holds the database, the vectors and your documents; back it up.
  4. Set it up over SSH. ssh -N -L 3001:127.0.0.1:3001 root@your-server then http://127.0.0.1:3001. Choose the LLM: a hosted provider and its API key, or Ollama at http://host.docker.internal:11434 if it runs on this machine. Turn on multi-user mode or set the instance password in Settings before anyone else can reach it.
  5. Give it a domain. Point an A record at the server, install Caddy, and one Caddyfile block reverse_proxy 127.0.0.1:3001 under your domain gets the certificate and puts HTTPS in front of 3001. Open 80 and 443 on the firewall, nothing else.

Honestly

What we would actually buy

The floor is 2 cores and 2 GB, and the cheapest row meets it. We would still spend the extra dollar.

For a document library you will actually use

BD-8 · $12.90/mo

4 vCPU, 8 GB and 100 GB at $12.90, against GL2-4 at $11.90: a dollar a month buys two more cores for the embedder that runs on this machine every time you upload, and 100 GB for the documents and their vectors. The model is elsewhere, behind an API key, so no GPU; if you want Ollama beside it, read the Ollama page and buy for the model, not for AnythingLLM. On any of them, run the AVX2 check before you upload a single file.

Deploy BD-8GL2-4, the cheapest that meets the floor

Questions

What people ask before they buy one.

What are AnythingLLM's system requirements?
Its docs give a minimum of 2 GB of RAM, a 2-core CPU with AVX2 and 5 GB of storage, and say that is enough to store some documents, chat and use its features. The project's Docker guide adds that on a cloud machine you should aim for at least 2 GB of RAM and 10 GB of disk, because disk grows with documents, vectors and any local models. The docs also say the app is a wrapper around external services, so it is light when the model and the embedder are elsewhere, and heavier when they run on the same machine.
What is the AVX2 requirement, and will my VPS have it?
The default vector database, LanceDB, is compiled for CPUs with the AVX2 instruction set (Intel Haswell, 2013, and newer). Without it the server process is killed the moment the database loads, and the docs call it the one hard hardware requirement. Virtual machines are the catch: a hypervisor can present a generic CPU model that hides AVX2 even on a modern host, so the docs say to check inside the guest with lscpu | grep -o avx2, not on the host. We do not promise a CPU flag we cannot see from the catalogue, so run that one line in the first minute. If it prints nothing, set VECTOR_DB to one of the other supported vector databases, which run as separate services.
Does it need a GPU?
No. AnythingLLM itself runs the web app, the document pipeline and, by default, a CPU-only embedder. Chat answers come from whichever LLM you connect: a hosted provider through an API key, with what the docs call almost zero overhead, or a local one through Ollama. The docs' own tip is to host a local model on a different machine that has a GPU if the AnythingLLM machine does not. That is the Ollama page and the GPU line, not this one.
Is it open on the internet after install?
The project's command publishes port 3001 on every interface, and a fresh instance has no password until you set one. This page's install binds 3001 to 127.0.0.1 instead, so until you add a domain the only way in is an SSH tunnel. Then, in Settings, set the instance password or turn on multi-user mode, which the README lists as a Docker-version feature, before you put HTTPS in front of it.
Where does my data live?
In the storage directory you mount at /app/server/storage: the SQLite database, the LanceDB vectors, uploaded documents and any locally downloaded models. The Docker guide tells you to mount it on the host precisely so you can pull a new image without losing anything. Back up that directory and the .env beside it; together they are the whole instance.
Can it use Ollama on the same machine?
Yes. From inside the container the host is http://host.docker.internal:11434, which on Linux needs --add-host=host.docker.internal:host-gateway on the run command; the step above includes it. Size the machine for the model then: an 8B model wants 8 GB of its own on top of AnythingLLM, and answers at a few tokens a second on CPU. The 24 GB row is the smallest on this page where that is comfortable.

Keep reading

Related