Open-weight models and where to run them

Open-weight models publish their trained weights, so you can download them, run them where you like and even fine-tune them. Open weights is not the same as open source: each family has its own licence, and some limit commercial use or very large companies, so read it before you ship.

There are three common places to run one. On your own machine, through Ollama, for experiments and private data. On your own GPU server, with an inference server such as vLLM, when volume or data rules justify the work.

Or through a hosted provider that serves open models behind a normal API. Smaller and quantised versions (weights stored at lower precision) fit on modest hardware but lose some quality. Which model to choose is a separate question: the Choosing LLMs roadmap covers it.