Best Mini PC to Run LLM Locally

Recommended Mini PCs for Running Local LLMs

• Dell OptiPlex 7000 Micro Desktop Computer: Fast with maxed out RAM.
• Fujitsu Futro S940: Good, cheap, fast, and power-efficient.
• HP Proliant MicroServer Gen 8: Great for NAS with 4 front disk slots.
• Intel NUC: Works well for SIMD development.
• Lenovo ThinkCentre M900: Affordable and fast machine.
• Minisforum Series (e.g., MS-01, UM690, HX90): Good for different tasks like networking or processing.
• Pi 4: Suitable for website, DNS, and other services in jails.

Hardware Considerations

• Multiple GPUs like 2x3090s are better than a single 4090 for LLM performance.
• Better PSU (1000-1200w) needed for power consumption with multiple GPUs.
• Intel CPUs like i7 or similar recommended for running LLMs.
• Mac Mini or Ryzen APU for cost-effective local LLM running.

User Insights and Tips

• Check graphics and network hardware compatibility with FreeBSD for smooth operations.
• AMD pre-Ryzen CPUs are not recommended for LLM tasks.
• Drivers, especially for graphics, might take time to support FreeBSD.

Other Tips and Recommendations

• Consider used 3090 cards for cost-effective performance.
• Remember VRAM is crucial for LLM performance.
• Consider ARM SBCs like Odroid H3 or Raspberry Pi for experimentation.

This comprehensive summary highlights different mini PCs recommended by users for running Local LLMs, along with hardware considerations, user insights, and additional tips.