Vision models

2024-01-30 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

Ollama has updated its LLaVA (Large Language-and-Vision Assistant) model collection to version 1.6, introducing significant enhancements for vision models. These updates include support for up to 4x higher image resolution, enabling the models to process more intricate visual details. The LLaVA 1.6 models also feature improved text recognition and reasoning capabilities, achieved through training on additional document, chart, and diagram datasets. The new models are distributed under more permissive licenses, specifically Apache 2.0 or the LLaMA 2 Community License. Available in 7B, 13B, and a new 34B parameter size, they can be run via the `ollama run` command-line interface or integrated using Ollama's Python and JavaScript libraries, supporting both file paths and base64-encoded images.

Key takeaway

For AI Engineers building multimodal applications, the LLaVA 1.6 update provides significantly improved image resolution and text recognition. You should consider migrating to these new models, especially the 34B version, to enhance detail comprehension and document analysis in your vision-language tasks. Leverage the more permissive Apache 2.0 license for broader deployment flexibility.

Key insights

LLaVA 1.6 models offer enhanced vision capabilities, higher resolution, and improved text recognition under permissive licenses.

Principles

• Higher resolution improves detail comprehension.
• Diverse training data enhances specialized recognition.

Method

Integrate LLaVA 1.6 models via `ollama run` CLI or Ollama's Python/JavaScript libraries, providing image paths or base64-encoded files for vision tasks.

In practice

• Use `ollama run llava:34b` for largest model.
• Supply `./art.jpg` for image input.
• Encode images as base64 for API calls.

Topics

• LLaVA Models
• Multimodal AI
• Vision Models
• Text Recognition
• Ollama

Code references

• ollama/ollama-python
• ollama/ollama-js
• jmorganca/ollama

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• How AI Vision Evolved | Merve Noyan — Vision models have evolved from CNNs to ViTs and now to scalable, open-source Vision-Language Models like LLaVA.
• Llama 3.2 Vision — Llama 3.2 Vision offers local multimodal AI capabilities via Ollama, supporting diverse image-text tasks.
• Data Machina #260 — Small, specialized Vision-Language Models are achieving powerful performance with greater efficiency and openness.
• Qwen 3.7 Max & Plus First Test | Coding, Game Dev, OCR, Image Understanding Testing — Qwen 3.7 Max and Plus models show improved performance and reduced 'overthinking' compared to previous iterations.
• Vision LLMs are PDF Parsers Too: Reading Charts and Diagrams for RAG — Vision LLMs make image content searchable for RAG, complementing text parsers despite trade-offs in cost and exactness.
• International Conference on Learning Representations (ICLR) 2026 — Apple's ICLR 2026 contributions span LLM reasoning, efficient training, computer vision, and AI safety.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.