Run Code Llama locally

2023-08-23 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Novice, quick

Summary

Meta Platforms, Inc. released Code Llama on August 24, 2023, an open-source code generation model built upon Llama 2. This model offers strong performance for programming tasks, including infilling, large input context support, and zero-shot instruction following. Code Llama is now accessible via Ollama, a platform that allows users to run the model locally. It is available in three parameter sizes: 7 billion, 13 billion (requiring 16GB+ memory), and 34 billion (requiring 32GB+ memory). Users can also access specialized versions, including foundation models for general code tasks and Python-specific models, by using simple `ollama run` or `ollama pull` commands.

Key takeaway

For AI Engineers or developers seeking local code generation capabilities, Code Llama offers a powerful open-source solution. You can easily deploy and experiment with its 7 billion, 13 billion, or 34 billion parameter versions using Ollama, including specialized Python models. Consider your hardware's memory capacity (16GB+ for 13B, 32GB+ for 34B) when selecting a model size to ensure optimal local performance for your programming tasks.

Key insights

Code Llama, based on Llama 2, provides open-source code generation with infilling and large context support.

Principles

• Open models can achieve strong performance.
• Specialized models enhance task-specific capabilities.

Method

Users can run Code Llama locally via Ollama by executing `ollama run codellama:[size]` or `ollama pull codellama:[size]-[specialization]` commands after installing Ollama.

In practice

• Run 7B Code Llama with `ollama run codellama:7b`.
• Access Python-specific models with `ollama pull codellama:7b-python`.

Topics

• Code Llama
• Code Generation
• Large Language Models
• Ollama
• Python Programming

Code references

• jmorganca/ollama

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Llama 3 — Llama 3 offers substantial improvements over Llama 2, now locally runnable via Ollama.
• Collama - Run Ollama Models on Google Colab (Free, No Local GPU) — Collama simplifies running Ollama LLMs on Google Colab, enabling free cloud-based experimentation without a local GPU.
• What Is Llama.cpp? The LLM Inference Engine for Local AI — Llama C++ enables local, private, and cost-effective LLM deployment on consumer hardware via quantization and optimized formats.
• 100% Free Claude Code | Run Claude Code with Local LLM with Ollama and Qwen 3.5 — Local LLMs can be integrated with Quad Code via Ollama for specific coding tasks, offering a viable alternative to API-based solutions.
• Llama 3 on Your Local Computer | Free GPT-4 Alternative — Llama 3 achieves significant performance gains through extensive training data, an improved tokenizer, and advanced fine-tuning.
• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.