Windows preview

2024-02-14 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Novice, quick

Summary

Ollama is now available on Windows in a preview release as of February 15, 2024, enabling users to pull, run, and create large language models natively on the platform. This Windows version includes built-in GPU acceleration, supporting NVIDIA GPUs and modern CPU instruction sets like AVX and AVX2 without requiring configuration or virtualization. Users gain full access to the Ollama model library, including vision models such as LLaVA 1.6, which supports drag-and-drop image input. The Ollama API automatically runs in the background on `http://localhost:11434`, offering OpenAI compatibility for integration with existing tools and applications.

Key takeaway

For developers and data scientists building or experimenting with large language models on Windows, Ollama's preview release simplifies local deployment and hardware acceleration. You can now easily run models like LLaVA 1.6 with GPU support and integrate with existing OpenAI-compatible tooling, streamlining your development workflow without complex setup.

Key insights

Ollama's Windows preview enables native, accelerated local LLM execution with full model library and OpenAI-compatible API access.

Principles

• Local LLM execution is simplified.
• Hardware acceleration is automatic.

Method

Download `OllamaSetup.exe`, install, then run `ollama run [model_name]` in a terminal to deploy a model.

In practice

• Run LLaVA 1.6 vision models locally.
• Integrate with OpenAI-compatible tools.

Topics

• Ollama
• Windows Platform
• Large Language Models
• GPU Acceleration
• OpenAI Compatibility

Code references

• ollama/ollama

Best for: NLP Engineer, Computer Vision Engineer, Machine Learning Engineer, AI Engineer, Software Engineer

Related on AIssential

• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.
• Ollama now supports AMD graphics cards — Ollama now supports AMD GPUs on Windows and Linux, expanding local LLM accessibility.
• Improved performance and model support with GGUF — Ollama 0.30 significantly boosts performance and broadens model/hardware compatibility via GGUF and llama.cpp integration.
• Collama - Run Ollama Models on Google Colab (Free, No Local GPU) — Collama simplifies running Ollama LLMs on Google Colab, enabling free cloud-based experimentation without a local GPU.
• Latest open artifacts (#17): NVIDIA, Arcee, Minimax, DeepSeek, Z.ai and others close an eventful year on a high note — The open model ecosystem is rapidly evolving with diverse architectures, data transparency, and specialized performance.
• What is your current local LLM setup? — The community actively explores diverse local LLM setups, balancing hardware, software, and model choices for varied applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.