A team of 13 researchers has open-sourced Hebatron, a Hebrew-specialized language model that outperforms significantly larger competitors while using a fraction of their computational resources during inference.

The model achieves a 73.8% Hebrew reasoning average, surpassing DictaLM-3.0-24B-Thinking's 68.9% score while activating only 3 billion parameters per forward pass across its 30 billion parameter sparse mixture-of-experts architecture.

Hebatron builds on NVIDIA's Nemotron-3 sparse MoE framework, making it the first language-specific adaptation of this architecture for any target language. The researchers employed a three-phase "easy-to-hard" curriculum training approach with continuous anti-forgetting anchoring, followed by supervised fine-tuning on 2 million bilingual Hebrew-English samples.

The curriculum ordering alone delivered a 3-point aggregate benchmark improvement over reversed training configurations, according to the research paper published on arXiv.

Performance and efficiency gains

The model remains competitive with Google's Gemma-3-27B-IT on GSM8K-HE mathematical reasoning and Israeli Trivia benchmarks while delivering approximately 9 times higher inference throughput. Hebatron supports native context lengths up to 65,536 tokens, making it the first open-weight Hebrew-specialized MoE model with long-context capabilities.

The sparse architecture allows the model to selectively activate relevant expert networks during inference, dramatically reducing computational requirements compared to dense models of similar capability.

The research team includes members from multiple Israeli institutions and represents a collaborative effort to advance Hebrew natural language processing capabilities. Model weights have been released openly to support further research in Hebrew and Semitic-language NLP applications.

The release addresses a significant gap in Hebrew language AI capabilities, providing researchers and developers with a high-performance, computationally efficient foundation model specifically optimized for Hebrew text understanding and generation tasks.