Local model runtimes, inference engines, agent frameworks and coding systems that are useful beyond a demo.
8 long-form field guidesOpenHands / OpenHands
The current OpenHands repository centers on Agent Canvas, a self-hosted control surface for starting coding-agent conversations and automating engineering work across local, container, remote, and cloud backends. That scope makes it more ambitious than a chat panel and more operationally demanding than an editor extension.
ollama / ollama
Ollama packages model acquisition, local execution, configuration, a command-line interface, and an application-facing API into one approachable runtime. Its current documentation also supports larger cloud-hosted models, so the project is best understood as a consistent model interface with local execution as a major strength, not as an offline guarantee.
Aider-AI / aider
Aider works inside an existing Git repository, builds a map of the codebase, sends selected context to a chosen model, applies edits, and records changes through familiar version-control machinery. Its restraint is the point: the developer keeps the terminal, editor, test suite, and repository as the center of work.
continuedev / continue
Continue helped establish a configurable alternative to sealed coding assistants through editor extensions, a terminal client, model choice, rules, prompts, and tool integrations. The official repository now states that it is read-only and no longer actively maintained, with 2.0.0 presented as the final release. That status is the first fact any prospective adopter needs.
langchain-ai / langchainjs
LangChain.js v1 centers its main package on agents, model and tool abstractions, structured output, messages, and middleware, with agents running on LangGraph. That shared machinery can shorten integration work, but it does not remove the hard product work of permissions, context selection, evaluation, cost controls, and truthful failure handling.
vllm-project / vllm
vLLM is an inference and serving engine for large language models that started in UC Berkeley's Sky Computing Lab. People come to it for throughput. The better reason to study the repository is how it turns the attention cache, batching and parallelism into scheduling decisions that an operator has to understand and own.
ggml-org / llama.cpp
llama.cpp is a C and C++ inference engine built on the ggml library, with quantized GGUF models, a long list of CPU and GPU backends and an HTTP server that speaks OpenAI- and Anthropic-compatible APIs. Other projects embed it. Running it directly gives the most control, and the most flags to get right.
langchain-ai / langgraph
LangGraph is a low-level orchestration framework from LangChain Inc for long-running, stateful agents. It does not hide prompts or choose an architecture. It provides a graph of nodes and edges, a persistence layer that checkpoints state per thread, and interrupts for human review: the machinery agents tend to need once they leave a demo.