Nvidia releases a new small, open model Nemotron-Nano-9B-v2 with toggle on/off reasoning

In a significant advancement for AI language models, Nvidia has introduced the Nemotron-Nano-9B-v2, a small yet powerful model designed to optimize performance while fitting seamlessly onto a single Nvidia A10 GPU. This release follows a wave of innovative small models, including a recent AI vision model from MIT spinoff Liquid AI and a smartphone-compatible model from Google. The Nemotron-Nano-9B-v2 boasts the ability to toggle AI 'reasoning' on and off, allowing users to choose whether the model self-checks before delivering an answer. With a parameter count reduced from 12 billion to 9 billion, it stands out in its category, demonstrating impressive efficiency and speed—up to six times faster than similar-sized transformer models. Oleksii Kuchiaev, Nvidia's Director of AI Model Post-Training, emphasized that this reduction was tailored specifically for deployment on the widely used A10 GPU. This model supports multiple languages, including English, German, Spanish, French, Italian, and Japanese, as well as several others like Korean and Portuguese, making it versatile for various applications including instruction following and code generation. The Nemotron-Nano-9B-v2 is based on the hybrid Mamba-Transformer architecture, which combines traditional attention layers with selective state-space models to manage lengthy sequences without excessive memory demands. Nvidia's latest offering also features a unique runtime 'thinking budget' management. This allows developers to set limits on the number of tokens spent on internal reasoning, effectively balancing accuracy with latency—a crucial factor for applications such as customer support or autonomous systems. Evaluation results reveal competitive performance against other small-scale models, achieving notable scores on several benchmarks, including a 72.1% accuracy on AIME25 and 90.3% on IFEval. Released under the Nvidia Open Model License Agreement, the model is designed to be enterprise-friendly, allowing developers to use, create, and distribute derivative models without the burden of additional licensing fees or restrictions. This provides a significant advantage for enterprises looking to deploy AI solutions quickly and efficiently. With the Nemotron-Nano-9B-v2, Nvidia is clearly targeting developers who seek a blend of reasoning capabilities and deployment efficiency. The model's features are designed to facilitate experimentation and integration, reinforcing Nvidia's commitment to enhancing AI language model accessibility while maintaining high accuracy and manageable operational costs.