Lesson plan

Vision-language model

A vision–language model (VLM) is a type of artificial intelligence system that can jointly interpret and generate information from both images and text, extending the capabilities of large language models (LLMs), which are limited to text. It is an example of multimodal learning. Many widely used commercial applications now rely on this ability. OpenAI introduced computer vision capabilities to its GPT-4V variant of the GPT-4 model, enabling users to incorporate uploaded photographs or diagrams into their discussions with ChatGPT. It has since become an integral part of ChatGPT's standard offering. Similar capabilities were added to Google’s Gemini, Anthropic’s Claude 3 Opus, and Microsoft’s Copilot with Vision. Alongside these models, several open-source vision–language models—such as LLaVA, InstructBLIP, and MiniGPT-4 —have been released by the research community, offering smaller-scale alternatives for experimentation and academic study.

Back to article

Article summary

A vision–language model (VLM) is a type of artificial intelligence system that can jointly interpret and generate information from both images and text, extending the capabilities of large language models (LLMs), which are limited to text. It is an example of multimodal learning. Many widely used commercial applications now rely on this ability. OpenAI introduced computer vision capabilities to its GPT-4V variant of the GPT-4 model, enabling users to incorporate uploaded photographs or diagrams into their discussions with ChatGPT. It has since become an integral part of ChatGPT's standard offering. Similar capabilities were added to Google’s Gemini, Anthropic’s Claude 3 Opus, and Microsoft’s Copilot with Vision. Alongside these models, several open-source vision–language models—such as LLaVA, InstructBLIP, and MiniGPT-4 —have been released by the research community, offering smaller-scale alternatives for experimentation and academic study.

Key claims

No claims recorded.

Discussion questions

  1. Summarise the article in your own words. Which single claim would you verify first, and why?
  2. Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
  3. If two sources disagreed about one of these facts, what would you need in order to trust one over the other?

Bibliography

No sources recorded.

Key claims and bibliography are generated from the same tracked claims and sources as the article page.