An entirely open-source AI code assistant inside your editor

2024-05-30 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

Continue, an open-source AI coding assistant, integrates with Ollama to provide local code completion and chat experiences within Visual Studio Code and JetBrains. Users can install both Continue and Ollama to run large language models (LLMs) locally, either on their machine or a remote server. The article details configurations for various models, including Mistral AI's Codestral 22B for both autocomplete and chat, or a split setup using DeepSeek Coder 6.7B for autocomplete and Llama 3 8B for chat, depending on available VRAM. It also covers setting up `nomic-embed-text` embeddings with Ollama to power the `@codebase` context provider for relevant code snippet retrieval and using the `@docs` provider to query documentation. Additionally, the post outlines a process for fine-tuning StarCoder 2 on local development data to improve model suggestions.

Key takeaway

For AI Engineers seeking to enhance their coding workflow with privacy and customization, integrating Continue with Ollama allows for a powerful, local AI assistant. You can configure specific open-source LLMs like Codestral or Llama 3 for tailored autocomplete and chat experiences directly within your IDE. Consider fine-tuning models like StarCoder 2 with your development data to significantly improve suggestion accuracy and relevance for your team's specific codebase.

Key insights

Combine Continue and Ollama for a local, open-source AI coding assistant with customizable LLMs.

Principles

• Local LLMs enhance privacy and control.
• Model choice depends on VRAM and task.
• Fine-tuning improves model relevance.

Method

Install Continue and Ollama, then configure `config.json` to specify LLMs for autocomplete, chat, and embeddings, or fine-tune models using local development data.

In practice

• Use Codestral 22B for combined tasks.
• Split DeepSeek Coder 6.7B and Llama 3 8B.
• Enable `@codebase` with `nomic-embed-text`.

Topics

• AI Code Assistants
• Open-source LLMs
• Ollama Integration
• Code Completion
• Model Fine-tuning

Code references

• ollama/ollama
• dlt-hub/continue-dlt-demo

Best for: Software Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

• Seeing What’s Possible with OpenCode + Ollama + Qwen3-Coder — Combine OpenCode, Ollama, and Qwen3-Coder for a private, offline, and free local AI coding assistant.
• OpenAI compatibility — Ollama now supports the OpenAI Chat Completions API, enabling local LLM use with existing OpenAI-compatible tools.
• Ollama on Steroids with OpenCode: A Headless Multi Agent Coding Workflow — A multi-agent system provides a reproducible, auditable, and private workflow for AI-driven code generation.
• 5 Cool Things I Did with Local Language Models — Local LLMs offer superior privacy, cost, and offline capabilities for sensitive data and proprietary workflows.
• Claude Code + Ollama = FULLY FREE AI Coding FOREVER! (Tutorial) — Integrate Claude Code with Ollama to run open-source LLMs locally, leveraging agentic capabilities for free.
• New coding models & integrations — Ollama expands its coding model offerings and integrations for enhanced developer workflows.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.