Summary of Reddit Comments for "computer for local llm Ollama"

LMStudio and Ollama Models

• LMStudio is recommended for working with LMStudio and Ollama models.
• Models are limited by the tools provided and cannot access the internet.
• Models vary in size depending on GPU VRAM, with Qwen3 (8b or 14b) recommended for STEM and Gemma3 12b for "World Knowledge" or rephrasing texts.
• Mistral 7B is outdated compared to Qwen 3 8B and Gemma 3 12B models.
• Ollama models are available on the local PC via API for use with tools like Openwebui.
• LMStudio and MSTY are user-friendly options for local LLMs.
• MSTY allows configuration of model settings and supports various models.
• Backyard.ai is a lightweight and easy-to-use option, especially for Vulkan(AMD/Intel GPU) support.
• NPC tools are recommended for organizing conversations and equipping local models.
• Pinokio.computer is praised for its easy setup and usage of AI tools.

Running Local Models

• Running local models requires significant VRAM from the GPU.
• RTX 5060 Ti 16GB or RTX 5090 GPUs are mentioned as suitable for running models.
• Laptops with low GPU capabilities may limit performance, while local GPUs are more efficient.
• Locally run LLMs are suggested for sensitive documents or for fun tinkering.
• Helix.ml offers a GUI for AI tools and models, with potential for free use based on company size.
• Ich hab eine RTX-5070 Ti is reported to run Ollama with good performance.

Specific Model Recommendations

• DeepSeek-R1-0528-Qwen3 is noted for its efficiency on a $300 GPU with 12GB VRAM.
• Shisa models are advised for Japanese language tasks.
• Qwen2.5 is recommended for LLMs.

Other Tools and Suggestions

• Open WebUI and NPC tools are mentioned for interacting with models.
• Nemotron-mini and Nvidia web version are provided as starting points for beginners.
• Kolosal AI is mentioned for its light size and compatibility with CPU or GPU.
• Word documents can utilize offline LLMs with specific features for use cases.
• Github projects like sgpt and Github Copilot are recommended for AI tasks.

These comments provide insights into various LLM models, tools, and setups for running local models efficiently, depending on GPU capabilities, model requirements, and user preferences.