System Requirements
AnythingLLM is fully customizable in every regard.
Given this customizable nature, your exact requirements to run AnythingLLM depend on many factors. You can use the tables below to get a rough idea of what it will take to run AnythingLLM.
AnythingLLM can be a wrapper around many external services that all accomplish some task - making AnythingLLM so lightweight it can run on the smallest machines - even Raspberry Pis!
Recommended configuration for AnythingLLM
The minimum requirements for running AnythingLLM vary based on your use case. Fundamentally, if want to use a local LLM on-device this will be the main factor in determining your requirements. AnythingLLM itself is very lightweight and can run on very small machines, but the LLM is often the bottleneck to a decent experience and will limit what kind and size of models you can use.
In general, better model = better AnythingLLM experience. If you are wondering what the minimum requirements are for a "basic" AnythingLLM experience, they are as follows:
| Property | Recommended Value |
|---|---|
| RAM | 16GB |
| CPU | 8-core CPU (any) |
| Storage | varies |
On Windows, a GPU is recommended to leverage your GPU for faster processing of local LLMs (8-12GB+ VRAM is great!) On MacOS, any M-Series chip will be able to handle local LLMs with no additional hardware. Intel-based Macs will be a bit slower - mostly limited by RAM.
The storage requirements are based on the size of the local LLM model you want to use since AnythingLLM stores the models on your PC.
If you are using a cloud-based LLM, the requirements will be much lower since the AnythingLLM client is very lightweight and does not need to store the models on your PC.
LLM selection impact
This is how you get chat responses. Popular hosted solutions like OpenAI (opens in a new tab) tend to provide state-of-the-art responses with almost zero overhead. However, you will need an API key for any cloud-based LLM provider.
Tip: Host a local LLM on another machine that has a GPU if the device running AnythingLLM does not have a GPU. AnythingLLM can connect to any LLM running anywhere via API.
Embedder selection impact
This is the model which you use to "embed" or vectorize text. Likewise, external services connected to AnythingLLM have zero overhead impact.
Tip: Host a local embedder on another machine that has a GPU if the device running AnythingLLM does not have a GPU. AnythingLLM can connect to to a provider via API.
Vector database selection impact
All supported vector databases either have no impact as they are externally hosted or can scale to hundreds of millions of vectors at the minimum recommended settings.
the default LanceDB vector database can handle anything you can throw at it