Python & JavaScript Libraries

2024-01-22 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

Ollama released initial versions of its official Python and JavaScript libraries on January 23, 2024, enabling direct integration of new and existing applications with the Ollama ecosystem. These libraries replicate the features and interface of the Ollama REST API, allowing developers to interact with local large language models. Key functionalities demonstrated include streaming responses, multi-modal interactions with models like LLaVA, text completion using models such as Stable Code, and the creation of custom models from Modelfiles. The release also announced a consolidation of Ollama's GitHub repositories under a new "ollama" organization, aiming to centralize development and community contributions.

Key takeaway

For developers building applications that require local large language model integration, adopting the new official Ollama Python or JavaScript libraries simplifies development. You can quickly implement features like streaming, multi-modal processing, and custom model creation. Consider migrating existing community-driven integrations to these official libraries for enhanced support and feature parity with the core Ollama platform.

Key insights

Ollama's new Python and JavaScript libraries simplify integrating local LLMs into applications, mirroring the REST API.

Principles

• API parity across language bindings
• Community-driven library development

Method

Integrate Ollama via `pip install ollama` or `npm install ollama`, then use `ollama.chat` or `ollama.generate` for model interactions.

In practice

• Stream LLM responses for real-time output
• Process images with multi-modal models
• Define custom models using Modelfiles

Topics

• Ollama Libraries
• Python Library
• JavaScript Library
• LLM Integration
• Multi-modal AI

Code references

• ollama/ollama-python
• ollama/ollama-js
• jmorganca/ollama

Best for: NLP Engineer, Computer Vision Engineer, AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Ollama Python library 0.4 with function calling improvements — Ollama's 0.4 Python library simplifies tool integration by directly accepting Python functions and auto-generating JSON schemas.
• 10 Python Libraries for Building LLM Applications — Building LLM applications requires specialized Python libraries for control over model loading, data integration, serving, and evaluation.
• Llama 3 — Llama 3 offers substantial improvements over Llama 2, now locally runnable via Ollama.
• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.
• Build a free writing assistant with Python and Ollama — Build a local AI typing assistant using Python, Ollama, and keyboard/clipboard control for instant typo correction.
• 10 Python Libraries Every LLM Engineer Should Know — A curated set of Python libraries significantly streamlines LLM application development, fine-tuning, and deployment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.