General Compute is pioneering a novel approach to AI inference by establishing a dedicated cloud infrastructure built around alternative chips. The company procures advanced hardware, including NVIDIA GPUs for the 'prefill' stage (reading prompts) and specialised ASICs from vendors like Cerebras, SambaNova, Positron AI, and d-Matrix for the 'decode' stage (writing answers). This disaggregated architecture allows for unparalleled speed and efficiency in processing AI inference requests.
General Compute handles the entire lifecycle of this bespoke hardware, from acquisition and deployment into colocation data centres to managing cooling, networking, energy, and the proprietary software stack. Their software is designed to intelligently split each inference request across these diverse chips, optimising for both throughput and latency. This results in significantly faster token generation per user and higher throughput per rack compared to traditional GPU-only cloud solutions, with benchmarks showing up to 16.2 times faster end-to-end performance.
The service is aimed at developers and organisations building high-performance AI applications, particularly those requiring real-time responses and complex agentic workflows, such as AI agents and coding tools. By offering either bare metal access or a single endpoint with comprehensive SLAs, General Compute simplifies the deployment and management of cutting-edge AI inference infrastructure. Their model allows clients to benefit from the latest chip innovations without the complexities of hardware procurement, data centre operations, or software integration for disaggregated systems. The company is rapidly expanding its capacity, with deployments going live from Q1 2027, and offers flexible pricing models based on contract terms, volume, and site requirements.