OpenFlyV2 introduces a unified onboard vision-language-action (VLA) architecture for aerial robots that integrates semantic understanding, language-grounded perception, spatial reasoning, and continuous flight control within a single model deployable on resource-constrained hardware. The system employs a shared vision-language representation processed by a compact action expert that outputs short-horizon flight waypoints, while motion-aware temporal memory and action priors minimize computational overhead for closed-loop execution. Researchers constructed a simulation-based dataset comprising approximately 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs to train the model. OpenFlyV2 generates action chunks of 12 waypoints at 10 Hz entirely onboard an NVIDIA Jetson Orin NX. The architecture demonstrates zero-shot sim-to-real transfer, achieving a 48% navigation success rate across four distinct physical environments without utilizing any real-world aerial training data. Results confirm that broad vision-language competence and continuous closed-loop flight can coexist effectively within a single model on an embedded aerial platform.
Read original
huggingface/daily-papers