Build Your Own Local AI Coding Agent with Gemma 4 and OpenCode

2026-06-23 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, medium

Summary

Setting up a local AI coding agent is detailed, offering an alternative to cloud-hosted models for cost control, privacy, and understanding agent stack mechanics. The process involves three core components: Ollama, which serves local language models; Gemma 4, Google's open model released April 2, 2026, used as the local LLM; and OpenCode, an open-source, LLM-agnostic agent interface. The guide outlines installing Ollama, downloading Gemma 4 (specifically the "gemma4:e4b" variant, a 9.6 GB, 4-bit quantized GGUF model with 128K context), installing Node.js and OpenCode, and finally connecting OpenCode to the local Gemma 4 instance via Ollama's "http://localhost:11434/v1" endpoint, ensuring the 128K context window is enabled. This setup allows for tasks like writing READMEs, explaining functions, and creating test scripts, with all data remaining local.

Key takeaway

For AI Engineers concerned about data privacy or cloud costs, building a local coding agent with Ollama, Gemma 4, and OpenCode offers a robust alternative. This setup keeps all code and data on your machine, enabling secure experimentation and development. You should consider using the "gemma4:e4b" variant for enhanced capability and explicitly setting a 128K context window via a "Modelfile" for optimal agent performance in complex coding tasks.

Key insights

Local AI coding agents can be built using Ollama, Gemma 4, and OpenCode for privacy, cost control, and experimentation.

Principles

• Local LLMs enhance privacy and reduce costs.
• Agent interfaces can be LLM-agnostic.
• Explicitly set LLM context for agents.

Method

Install Ollama to serve Gemma 4 locally, then install OpenCode. Configure OpenCode via "opencode.json" to connect to Ollama's local API endpoint, ensuring the Gemma 4 model uses a 128K context window via a "Modelfile".

In practice

• Use "ollama pull gemma4:e4b" for local LLM.
• Create "Modelfile" to set "num_ctx 131072".
• Configure "opencode.json" to "http://localhost:11434/v1".

Topics

• Local AI
• Coding Agents
• Ollama
• Gemma 4
• OpenCode
• LLM Deployment
• Data Privacy

Best for: AI Engineer, Machine Learning Engineer, AI Student

Related on AIssential

• Local Agentic Programming on the Cheap: Claude Code + Ollama + Gemma4 — Local agentic coding with Gemma 4 and Claude Code offers a private, cost-free alternative to cloud LLMs for daily engineering tasks.
• Running Gemma 4 Locally with Ollama on Your PC — Gemma 4 offers open-weight, locally runnable LLMs with multimodal capabilities and diverse architectures for varied hardware.
• How to Implement Tool Calling with Gemma 4 and Python — Gemma 4 and Ollama enable local, privacy-first tool-calling agents for dynamic, real-world interactions.
• Gemma 4 + Ollama = FREE Claude Code Setup! — Gemma 4 models offer high intelligence per parameter, enabling efficient local AI inference across diverse hardware.
• Codex + Ollama = Free Unlimited Coding AI — Ollama's integration with Codex enables free, local execution of open-source LLMs for AI coding tasks.
• Running local models is good now — Local LLMs have matured significantly, now enabling effective agentic coding and deep model introspection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.