Local LLM AI Setup Series

Introduction

I’ll be discussing over a series of blog posts my local AI setup to make the most of open weight models running on my hardware at home. This will cover everything from the research I’ve done to get my environment setup, the hardware, researching open weight models that can be used, tools to find good models for one’s hardware, etc.

This is my journey into working with AI locally outside of the frontier models from Anthropic/OpenAI.

Breakdown

Future posts will be broken down into the following categories:

• Hardware Research: What hardware do I need or want to run models
• Open Weight Model Research: What models can I run with my hardware and what type will serve my needs
• Model Runners: What will serve my models so I can interact with them (LM Studio, Ollama, vLLM, etc)
• Front Ends: What front end systems will I use to interact with my models being served (Open WebUI, Ollama, LM Studio, etc)
• Useful MCPs: Useful MCP servers I am using to extend the capabilities of the open weight models
• Misc: Other random things

Why break everything down? Well most people don’t want to read a wall of text and screenshots. Some people don’t have the attention span and would also rather just get to the point.

Maybe once the series concludes I’ll make one succinct large post containing my full setup without all the additional research but I want to provide those looking to create a similar setup all the potential possibilities they can use outside of just mine.

Ultimately, this is my journey and I just want to write about it. If someone learns a thing or two, great!

Outro

I will be starting Parental Leave as my partner and I welcome a new baby into the world shortly. So posts maybe a bit sporadic or non-existent for a while.