Ollama¶
Ollama allows you to get up and running with large language models locally.
Description¶
It provides a simple CLI and API for running models like Llama 3, Mistral, and others on your own hardware.
Links¶
Alternatives¶
Backlog¶
- Benchmarking performance on TrueNAS SCALE.
- Setup GPU passthrough for faster inference.