TensorZero builds open-source infrastructure for teams shipping large language model applications in production. Its stack unifies five components that are usually stitched together from disparate tools: a high-performance LLM gateway for unified provider access, observability for monitoring inference, evaluations for benchmarking quality, optimization for improving prompts and models, and experimentation for running A/B tests.
The core insight is a data and learning loop. By capturing structured inference and downstream feedback data, TensorZero can act like an 'automated AI engineer' that analyzes observability data, sets up evals, and optimizes prompts and model choices over time, instead of leaving teams to manually tune everything by hand.
The gateway is written in Rust for low latency and high throughput, reflecting the team's focus on industrial-grade reliability. The open-source repository has grown rapidly, at one point becoming the #1 trending repository of the week on GitHub as star counts jumped into the thousands.
TensorZero raised a $7.3M seed round in August 2025 led by FirstMark, with participation from Bessemer Venture Partners, Bedrock, DRW, Coalition, and a roster of strategic angels. CEO Gabriel Bianconi was previously chief product officer at Ondo, and CTO Viraj Mehta holds a Carnegie Mellon PhD focused on reinforcement learning and LLMs.
Based in New York City, the company targets engineering teams that have outgrown ad-hoc tooling and need a coherent, self-hostable stack to observe, evaluate, and continuously improve their LLM applications.