Qwen3.6 (Local) with OpenCode & llama.cpp | Build Agentic RAG Template with LangChain | 🔴 Live

2026-04-19 · Source: Venelin Valkov · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, extended

Summary

A live stream demonstrated building a full-stack starter template with a FastAPI backend and Next.js frontend, leveraging the Qwen 3.6 5-bit quantized model within an Open Code environment. The presenter used an M4 Pro with 48 GB of unified memory, noting approximately 31 GB of memory usage for the Q5_K_M quantization. A key configuration change involved setting `preserve-thinking=true` in the `llama.cpp` server to maintain conversational context, which significantly improved the model's performance compared to previous attempts with the Hermes agent. The demonstration focused on developing FastAPI endpoints for markdown file upload, text chunking, and retrieval, adhering to a red-green test-driven development (TDD) methodology. The Qwen 3.6 model successfully generated code, wrote passing tests, updated dependencies, and created a client script for API interaction, showcasing its capabilities for agentic coding on local hardware.

Key takeaway

For AI Engineers building local LLM-powered applications, consider integrating Qwen 3.6 with Open Code and configuring `llama.cpp` for context preservation. This setup enables effective agentic development, allowing the LLM to follow a red-green TDD cycle to build and test backend services, significantly improving code quality and development speed compared to less structured approaches. You should prioritize automated testing to validate agent-generated code and ensure feature stability.

Key insights

Qwen 3.6, with proper context preservation and an agentic framework, can effectively build and test full-stack applications locally.

Principles

• Preserve LLM thinking context for improved conversational coherence.
• Implement red-green TDD for robust agentic code development.
• Automated tests ensure code quality and prevent regressions.

Method

The method involves configuring `llama.cpp` with `preserve-thinking=true` for Qwen 3.6, using Open Code for agentic development, and applying a red-green TDD workflow to build and test FastAPI endpoints for document processing.

In practice

• Use `preserve-thinking=true` for better LLM context retention.
• Integrate Open Code for local LLM-driven development.
• Adopt TDD with agents to ensure functional code.

Topics

• Qwen 3.6
• OpenCode
• Agentic RAG
• FastAPI Backend
• LangChain Text Splitters

Best for: AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Hermes Agent with Qwen3.6 (Local) | AI App Template with LangChain, LangGraph, llama.cpp | 🔴 Live — Local LLMs like Qwen 3.6 face significant challenges in agentic coding tasks, especially with lower quantization.
• Gemma 4 with Pi Coding Agent & llama.cpp | Build LLM Resource Calculator with NextJS | 🔴 Live — PyCoding Agent with local LLMs can build simple apps but struggles with complex UI overhauls and skill integration.
• I Just Built an Agentic RAG System From Scratch — Here’s What I Learned — Agentic RAG systems empower LLMs to autonomously refine searches, significantly improving answer quality and efficiency.
• Run a Local LLM with OpenClaw on Your Mac Mini — Running a local, quantized LLM on a Mac Mini with llama.cpp significantly reduces OpenClaw API costs while maintaining performance for common agent tasks.
• Qwen 3.6 Plus: GREATEST Opensource AI Model EVER! Beats Opus 4.5 and Gemini 3 (Fully Tested) — Qwen 3.6 Plus is a multimodal agentic coding model with a 1 million context window, excelling in complex project generation.
• Run Qwen3.5 on an Old Laptop: A Lightweight Local Agentic AI Setup Guide — Local AI setups are accessible on older hardware using lightweight models and tools like Ollama and OpenCode.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Venelin Valkov.