Multimodal model

A multimodal model handles more than text. Most current frontier models accept images and documents alongside text, some accept audio or video, and some produce audio or images as well as text. Support differs by provider and model, so the label alone tells you little; check exactly which input and output types the model you're considering supports.

For builders, multimodality collapses pipelines. Instead of running OCR, then a layout parser, then a text model, you can send a receipt photo and ask for structured fields. Instead of describing a bug, a user can attach a screenshot. Images and audio are converted into tokens, so they count against the context window and the bill like text does, and resolution choices directly affect cost.

The usual mistake is trusting a model's perception the way you'd trust its reading of plain text. Models still misread small print, digits in poor scans, charts, and spatial relationships, and they can describe details that aren't there. Normalize inputs (size, orientation, format), ask for exact transcription where it matters, verify critical values in code, and build evals from real customer files rather than clean samples.