High-capacity vision-language models (VLMs) are trained on large web datasets, enabling them to effectively recognize visual and language patterns and function across multiple languages. However, for robots to reach a similar level of proficiency, they would need to gather firsthand data across various objects, environments, tasks, and situations. In this context, researchers have introduced Robotic Transformer 2 (RT-2), a vision-language-action (VLA) model that learns from both web and robotics

2.3K views · 61 reactions | Vision Language Models (VLMs) understand natural language prompts and perform visual question answering. ➡️ https://nvda.ws/4cTW5Ox Learn how you can build VLM-powered visual AI agents for a wide range of apps. #SIGGRAPH2024 | NVIDIA AI | Facebook

Vison-language pretraining is pushing AI forward in novel object captioning and image caption generation. Learn about powerful new VLP methods in this webinar and how advances permit captioning without image-text pairs on February 11 at 10 AM PT. Register now: https://aka.ms/AAaz6bh | Microsoft Research

In vision-and-language pretraining (VLP), objects can be used as anchor points to make aligning semantics between image-text pairs easier. Learn how Oscar, a novel VLP framework utilizing objects, sets new state of the art on six vision-and-language tasks: https://aka.ms/AA8flix | Microsoft Research