Run LLMs locally

From LemonWiki共筆
Revision as of 14:33, 29 April 2024 by Planetoid (talk | contribs) (Created page with "Run LLMs (Large language model) locally == List of LLMs software == [https://ollama.com/ Ollama]<ref>[https://blog.miniasp.com/post/2024/03/04/Useful-tool-Ollama 介紹好用工具：Ollama 快速在本地啟動並執行大型語言模型 | The Will Will Web]</ref> * Docker support: Available<ref>[https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image Ollama is now available as an official Docker image · Ollama Blog]</ref> {{Gd}} * Supported OS: {{...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Run LLMs (Large language model) locally

List of LLMs software

Ollama[1]

  • Docker support: Available[2] Good.gif
  • Supported OS: Win   , macOS icon_os_mac.png & Linux  
  • User Interface: GUI
  • Support API endpoint: Available[3]
  • License: OSS, MIT[4]

LM Studio - Discover, download, and run local LLMs

  • Docker support: N/A Icon_exclaim.gif
  • Supported OS: Win   , macOS icon_os_mac.png & Linux  
  • User Interface: GUI
  • Support API endpoint: Available[5]
  • License: Mixed; Apache License 2.0 for lmstudio.js[6]. But Terms of Use[7] mentioned "using the Services for your personal, non-commercial purposes."

janhq/jan: Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

  • Docker support: Available
  • Supported OS: Win   , macOS icon_os_mac.png & Linux  
  • User Interface: CLI (Command-line interface)
  • Support API endpoint: Available
  • License: GAPL[8]

GPT4All

  • Docker support: N/A
  • Supported OS: Win   , macOS icon_os_mac.png & Linux  
  • User Interface: GUI
  • Support API endpoint: Available[9]
  • License: OSS, MIT[10]

oobabooga/text-generation-webUser Interface: A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

  • Docker support: Available
  • Supported OS: Win   , macOS icon_os_mac.png & Linux  
  • User Interface: GUI
  • Support API endpoint: Available[11]
  • License: OSS, GNU v.3[12]

(left blank intentionally) 

* Docker support: Available or not
* Supported OS: {{Win}}, {{Mac}} & {{Linux}}
* User Interface: GUI or CLI
* Support API endpoint: Available or not
* License: Mixed, OSS or copyright

Further reading

References

  1. ↑ 介紹好用工具：Ollama 快速在本地啟動並執行大型語言模型 | The Will Will Web
  2. ↑ Ollama is now available as an official Docker image · Ollama Blog
  3. ↑ ollama/ollama: Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  4. ↑ ollama/LICENSE at main · ollama/ollama
  5. ↑ Local LLM Server | LM Studio
  6. ↑ lmstudio.js/LICENSE at main · lmstudio-ai/lmstudio.js
  7. ↑ https://lmstudio.ai/terms
  8. ↑ jan/LICENSE at dev · janhq/jan
  9. ↑ Generation - GPT4All Documentation
  10. ↑ gpt4all/LICENSE.txt at main · nomic-ai/gpt4all
  11. ↑ 12 ‐ OpenAI API · oobabooga/text-generation-webui Wiki
  12. ↑ text-generation-webui/LICENSE at main · oobabooga/text-generation-webui
Retrieved from "https://wiki.planetoid.info/index.php?title=Run_LLMs_locally&oldid=24650"