NVIDIA shipped NV-Raw2Insights-US, an AI model that processes raw ultrasound sensor data to create patient-specific sound speed maps for adaptive imaging.

The system bypasses traditional ultrasound reconstruction pipelines by learning directly from raw sensor measurements captured by ultrasound probes. This approach preserves information typically lost during conventional beamforming processes.

NVIDIA developed the technology in partnership with researchers from Siemens Healthineers. The model represents the first application in what NVIDIA calls the "Raw2Insights" class of AI systems for medical imaging.

Breaking the beamforming bottleneck

Traditional ultrasound imaging relies on hand-engineered reconstruction pipelines that make simplifying assumptions about physics, including constant sound speed throughout the body. NV-Raw2Insights-US estimates individual sound speed variations to correct images in real-time.

The system generates personalized sound speed maps for each patient during scanning. What previously required complex, time-intensive computation now happens in a single AI inference pass.

NVIDIA deploys the model using its Holoscan edge AI platform on systems including NVIDIA IGX Thor and NVIDIA DGX Spark. The inference runs on Blackwell-class GPUs.

Hardware integration challenges solved

Raw ultrasound channel data typically remains inaccessible on clinical scanners due to high bandwidth requirements. NVIDIA's Holoscan Sensor Bridge addresses this through an open-source FPGA IP that enables high-bandwidth, low-latency data transfer to GPUs.

The implementation uses an Altera Agilex-7 FPGA development kit paired with Holoscan Sensor Bridge to stream raw data from an ACUSON Sequoia ultrasound scanner's DisplayPort outputs. NVIDIA calls this approach "Data over DisplayPort."

The FPGA packetizes the data and transmits it over Ethernet to NVIDIA IGX systems for collection and AI inference. This demonstrates how modern computational capacity integrates with existing scanner architectures using high-bandwidth DisplayPort outputs.

Once processed, the patient-specific sound speed estimates stream back to the ultrasound scanner to improve focus in the live imaging feed. The architecture provides flexibility for both development and deployment scenarios.