Importing custom LLMs into AnythingLLM
AnythingLLM allows you to easily load into any valid GGUF file and select that as your LLM with zero-setup. Please only use text based LLMs
for this process. Embedder models will not function in this capacity.
Import model into AnythingLLM.
Desktop only!
This LLM provider is only available in the desktop version. If you are using the browser based version you will need to import the model into your local LLM provider. We recommend Ollama or LMStudio.
Importing any GGUF file into AnythingLLM for use as you LLM is quite simple. On the LLM selection screen you will see an Import custom model button.
Clicking this button will open a file picker. Simply select your GGUF file and wait 2-3 minutes while the model is imported. Now you can select and save this model as your LLM!
After import you should see your model displayed.
How to import into Ollama
Importing to Ollama is also quite simple and we provide instructions in your download email on how to accomplish this. Please refer to this video segment.
How to import into LMStudio
Importing to LMStudio is even more simple to load. Simply drag the instructions + GGUF folder into the LMStudio model location. Please refer to this
video segment.