A new distributed computing platform called Mesh LLM has launched to let teams run large language models across their own hardware instead of relying on cloud providers.

The system pools GPUs and memory from multiple machines — laptops, workstations, servers — and exposes them through a single OpenAI-compatible API at localhost:9337. Users can start with one device and add more later, with the mesh automatically deciding whether to run models locally, route to peers, or split across several machines.

Breaking free from cloud dependency

Mesh LLM addresses a core frustration for AI teams: surrendering control to external providers. With traditional cloud APIs, companies cannot control when models change, where data goes, or what hardware runs their workloads. Costs scale linearly with usage, with no optimization lever except paying more.

The platform ships with 40+ models, from half-billion-parameter models that run on laptops to 235-billion-parameter mixture-of-experts giants. For the largest models, Mesh LLM uses "Skippy" mode — partitioning models by layer ranges across multiple nodes in a pipeline.

Built on iroh networking

The architecture runs on iroh, a peer-to-peer networking library that handles NAT traversal and direct connections between devices. Every node boots an iroh endpoint identified by a public key, with no central server required.

The protocol uses QUIC's ALPN negotiation with three channels: the main mesh for gossip and routing, an owner control plane for configuration, and a latency-sensitive transport for split model activations. All communication flows through authenticated, encrypted connections between public keys.

Two iroh relays in different regions provide fallback connectivity when devices cannot reach each other directly.

Immediate availability

The 18MB software is available now, letting users join the public mesh or configure private deployments. A mobile app built on iroh's Swift SDK is planned, along with support for ACP, the emerging agent communication protocol.

The project's code is open source on GitHub, with the company positioning it as part of a broader shift toward peer-to-peer computing and reduced dependence on centralized cloud infrastructure.