Run LLMs locally: Difference between revisions

Jump to navigation Jump to search

Run LLMs locally (edit)

Revision as of 14:26, 7 August 2025

675 bytes added ,  7 August 2025
m
→‎List of LLMs software
(+ MLC LLM)
 
(6 intermediate revisions by the same user not shown)
Line 5: Line 5:
* Docker support: Available<ref>[https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image Ollama is now available as an official Docker image · Ollama Blog]</ref> {{Gd}}
* Docker support: Available<ref>[https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image Ollama is now available as an official Docker image · Ollama Blog]</ref> {{Gd}}
* Supported OS: {{Win}}, {{Mac}} & {{Linux}}
* Supported OS: {{Win}}, {{Mac}} & {{Linux}}
* User Interface: CLI (Command-line interface)
* User Interface: CLI (Command-line interface) & GUI (Graphical User Interface)
* Support API endpoint: Available<ref>[https://github.com/ollama/ollama?tab=readme-ov-file#rest-api ollama/ollama: Get up and running with Llama 3, Mistral, Gemma, and other large language models.]</ref>
* Support API endpoint: Available<ref>[https://github.com/ollama/ollama?tab=readme-ov-file#rest-api ollama/ollama: Get up and running with Llama 3, Mistral, Gemma, and other large language models.]</ref>
* License: OSS, MIT<ref>[https://github.com/ollama/ollama/blob/main/LICENSE ollama/LICENSE at main · ollama/ollama]</ref>
* Supported embedding models:
* License: Open-source license, MIT<ref>[https://github.com/ollama/ollama/blob/main/LICENSE ollama/LICENSE at main · ollama/ollama]</ref> {{Gd}}


[https://lmstudio.ai/ LM Studio - Discover, download, and run local LLMs]
[https://lmstudio.ai/ LM Studio - Discover, download, and run local LLMs]
Line 14: Line 15:
* User Interface: GUI
* User Interface: GUI
* Support API endpoint: Available<ref>[https://lmstudio.ai/docs/local-server Local LLM Server | LM Studio]</ref>
* Support API endpoint: Available<ref>[https://lmstudio.ai/docs/local-server Local LLM Server | LM Studio]</ref>
* Supported embedding models:
* License: Mixed; Apache License 2.0 for lmstudio.js<ref>[https://github.com/lmstudio-ai/lmstudio.js/blob/main/LICENSE lmstudio.js/LICENSE at main · lmstudio-ai/lmstudio.js]</ref>. But Terms of Use<ref>https://lmstudio.ai/terms</ref> mentioned "using the Services for your personal, non-commercial purposes."
* License: Mixed; Apache License 2.0 for lmstudio.js<ref>[https://github.com/lmstudio-ai/lmstudio.js/blob/main/LICENSE lmstudio.js/LICENSE at main · lmstudio-ai/lmstudio.js]</ref>. But Terms of Use<ref>https://lmstudio.ai/terms</ref> mentioned "using the Services for your personal, non-commercial purposes."


Line 21: Line 23:
* User Interface: CLI (Command-line interface)
* User Interface: CLI (Command-line interface)
* Support API endpoint: Available
* Support API endpoint: Available
* Supported embedding models:
* License: GAPL<ref>[https://github.com/janhq/jan/blob/dev/LICENSE jan/LICENSE at dev · janhq/jan]</ref>
* License: GAPL<ref>[https://github.com/janhq/jan/blob/dev/LICENSE jan/LICENSE at dev · janhq/jan]</ref>


Line 28: Line 31:
* User Interface: GUI
* User Interface: GUI
* Support API endpoint: Available<ref>[https://docs.gpt4all.io/gpt4all_python.html Generation - GPT4All Documentation]</ref>
* Support API endpoint: Available<ref>[https://docs.gpt4all.io/gpt4all_python.html Generation - GPT4All Documentation]</ref>
* Supported embedding models: "SBert and Nomic Embed Text v1 & v1.5"<ref>[https://docs.gpt4all.io/gpt4all_help/faq.html#what-are-the-system-requirements FAQ - GPT4All]</ref>
* License: OSS, MIT<ref>[https://github.com/nomic-ai/gpt4all/blob/main/LICENSE.txt gpt4all/LICENSE.txt at main · nomic-ai/gpt4all]</ref>
* License: OSS, MIT<ref>[https://github.com/nomic-ai/gpt4all/blob/main/LICENSE.txt gpt4all/LICENSE.txt at main · nomic-ai/gpt4all]</ref>


Line 35: Line 39:
* User Interface: GUI
* User Interface: GUI
* Support API endpoint: Available<ref>[https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API#examples 12 ‐ OpenAI API · oobabooga/text-generation-webui Wiki]</ref>
* Support API endpoint: Available<ref>[https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API#examples 12 ‐ OpenAI API · oobabooga/text-generation-webui Wiki]</ref>
* Supported embedding models:
* License: OSS, GNU v.3<ref>[https://github.com/oobabooga/text-generation-webui/blob/main/LICENSE text-generation-webui/LICENSE at main · oobabooga/text-generation-webui]</ref>
* License: OSS, GNU v.3<ref>[https://github.com/oobabooga/text-generation-webui/blob/main/LICENSE text-generation-webui/LICENSE at main · oobabooga/text-generation-webui]</ref>


Line 42: Line 47:
* User Interface: GUI
* User Interface: GUI
* Support API endpoint: Available<ref>[https://docs.privategpt.dev/api-reference/overview/api-reference-overview API Reference overview – PrivateGPT | Docs]</ref>
* Support API endpoint: Available<ref>[https://docs.privategpt.dev/api-reference/overview/api-reference-overview API Reference overview – PrivateGPT | Docs]</ref>
* Supported embedding models:
* License: OSS, Apache v.2<ref>[https://github.com/zylon-ai/private-gpt/blob/main/LICENSE private-gpt/LICENSE at main · zylon-ai/private-gpt]</ref>
* License: OSS, Apache v.2<ref>[https://github.com/zylon-ai/private-gpt/blob/main/LICENSE private-gpt/LICENSE at main · zylon-ai/private-gpt]</ref>


Line 56: Line 62:
* User Interface: CLI
* User Interface: CLI
* Support API endpoint: N/A
* Support API endpoint: N/A
* Supported embedding models:
* License:
* License:


Line 63: Line 70:
* User Interface: CLI
* User Interface: CLI
* Support API endpoint: Available<ref>[https://llm.mlc.ai/docs/deploy/rest.html REST API — mlc-llm 0.1.0 documentation]</ref>
* Support API endpoint: Available<ref>[https://llm.mlc.ai/docs/deploy/rest.html REST API — mlc-llm 0.1.0 documentation]</ref>
* Supported embedding models:
* License: OSS, Apache License Version 2.0<ref>[https://github.com/mlc-ai/mlc-llm/blob/main/LICENSE mlc-llm/LICENSE at main · mlc-ai/mlc-llm]</ref>
* License: OSS, Apache License Version 2.0<ref>[https://github.com/mlc-ai/mlc-llm/blob/main/LICENSE mlc-llm/LICENSE at main · mlc-ai/mlc-llm]</ref>


Line 71: Line 79:
* User Interface: GUI or CLI
* User Interface: GUI or CLI
* Support API endpoint: Available or not
* Support API endpoint: Available or not
* Supported embedding models:
* License: Mixed, OSS or copyright
* License: Mixed, OSS or copyright
</pre>
</pre>
Line 77: Line 86:
* [https://www.infoworld.com/article/3705035/5-easy-ways-to-run-an-llm-locally.html 5 easy ways to run an LLM locally | InfoWorld]
* [https://www.infoworld.com/article/3705035/5-easy-ways-to-run-an-llm-locally.html 5 easy ways to run an LLM locally | InfoWorld]
* [https://github.com/Troyanovsky/Local-LLM-Comparison-Colab-UI Troyanovsky/Local-LLM-Comparison-Colab-UI: Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.]
* [https://github.com/Troyanovsky/Local-LLM-Comparison-Colab-UI Troyanovsky/Local-LLM-Comparison-Colab-UI: Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.]
* [https://abishekmuthian.com/how-i-run-llms-locally/ How I run LLMs locally - Abishek Muthian]
* [https://iamhlb.notion.site/HOWTO-Setup-Local-LLM-16757ba4def5805c9baad0c78d54d62f HOWTO: Setup Local LLM]


== References ==
== References ==
Retrieved from "https://wiki.planetoid.info/index.php/Special:MobileDiff/24845...25905"

Navigation menu