How to prompt Code Llama

2023-09-08 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

Meta released the Code Llama model in three variations: Instruct, Code Completion, and Python, with parameter sizes of 7 billion, 13 billion, and 34 billion. This guide demonstrates how to structure prompts for these variations, using the 7 billion parameter model with 4-bit quantization and the open-source Ollama project for execution. The Instruct model excels at human-like answers, including code reviews and unit test generation. The Code Completion model continues code based on a prompt, while its Infill feature allows completing code within existing blocks using a specific prefix/suffix format. A Python-specific variation, fine-tuned on 100B additional Python tokens, is also available for Python-centric tasks. Several tools, including Cody, Continue, Phind, WizardLM, and Open Interpreter, have integrated or fine-tuned Code Llama.

Key takeaway

For AI Engineers and Software Engineers integrating code generation, understanding Code Llama's specialized variations is crucial. You should select the Instruct model for conversational coding tasks like reviews or test generation, and the Code Completion model (especially its Infill feature) for direct code generation within existing structures. This targeted approach will optimize your code assistance workflows and improve output relevance.

Key insights

Code Llama offers specialized models for instruction, completion, and Python, enhancing various coding workflows.

Principles

• Tailor model variation to task
• Infill requires specific prompt format

Method

To prompt Code Llama, use `ollama run codellama:[variation]` followed by the prompt string. For infill, structure the prompt as `{prefix} {suffix}`.

In practice

• Use Instruct for code review
• Generate unit tests with Instruct
• Employ Infill for mid-function code insertion

Topics

• Code Llama
• Prompt Engineering
• Code Generation
• Code Review
• Large Language Models

Code references

• jmorganca/ollama
• sourcegraph/cody

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Vibe Coding and Deploying LlamaIndex Agent Workflows with Claude Code — Claude Code can autonomously develop and deploy LlamaIndex agent workflows for specific tasks like document classification.
• Context Engineering: Foundations, Categories, and Techniques of Prompt Engineering — Prompt engineering is "soft programming" LLMs by crafting inputs to guide their behavior and leverage inherent knowledge.
• Run Code Llama locally — Code Llama, based on Llama 2, provides open-source code generation with infilling and large context support.
• Same Scrutiny, More Time: Eye Tracking Insights into Reviewing LLM-Labelled Code — Explicitly labeling code as LLM-generated increases review time without improving thoroughness, driven by perceived origin.
• Prompt Engineering Is Dead. Here’s What Actually Gets AI to Write Good Code. — Effective AI code generation hinges on providing comprehensive context, not just refined prompts.
• Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing — Proactive, edge-side prompt rewriting with a local LLM significantly reduces token costs for coding agents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.