NVIDIA Open-Sources NitroGen: AI Learns Games from Video
NVIDIA releases NitroGen, an open-source AI system that learns to play video games by watching gameplay footage, advancing machine learning through visual
Someone found that NVIDIA open-sourced NitroGen, an AI that literally watches gameplay footage and learns to play games from scratch.
How it works:
- Feed it raw video frames from gameplay recordings
- Vision transformer (SigLip2) processes what’s happening on screen
- Diffusion model spits out controller inputs to mimic human players
Pretty wild for imitation learning experiments. Works best on gamepad-friendly stuff like platformers, racing games, and action titles. Struggles with RTS/MOBA games that need precise mouse control.
Grab it here:
https://huggingface.co/nvidia/NitroGenThe interesting bit? It’s trained purely on watching humans play—no reward signals or game-specific programming. Just learns patterns from observing controller inputs matched to what’s happening on screen. Could be useful for testing game AI or building gameplay automation tools without writing tons of game-specific logic.
Related Tips
Tencent Releases HY-Motion: Text-to-3D Animation
Tencent unveils HY-Motion, an innovative AI system that generates high-quality 3D character animations directly from text descriptions, advancing
NousResearch Boosts Qwen3-14B Coding to 68% Pass@1
NousResearch enhances Qwen3-14B's coding performance to achieve 68% pass@1 rate through advanced fine-tuning techniques and optimization strategies for
Falcon-H1R-7B: Small Model Beats 70B via Hybrid RL
Falcon-H1R-7B demonstrates how a compact 7-billion parameter language model achieves performance rivaling 70B models through innovative hybrid reinforcement