The Decision Framework: Journey for Choosing LLM Models

2026-07-26 · Source: LLM on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

A decision framework outlines a five-step journey for selecting and implementing Large Language Models (LLMs) in AI applications, prioritizing non-training solutions before considering fine-tuning. The process begins by utilizing a base model such as GPT-4, Claude, Gemini, or an open-source alternative without immediate customization. Subsequently, prompt and context engineering are introduced to refine system prompts and structure inputs effectively. For scenarios requiring fresh or proprietary knowledge, Retrieval Augmented Generation (RAG) is recommended to inject relevant data at query time. If procedural know-how is lacking, agent skills are developed to guide the model through specific workflows or tool usage. Fine-tuning, potentially with LoRA, is reserved as a final consideration, only when specific bottlenecks like latency, consistency, or task failure persist. This framework advocates exhausting all no-training options before engaging in fine-tuning.

Key takeaway

For AI Engineers or ML Architects designing LLM-powered applications, you should systematically explore non-training optimization techniques before committing to fine-tuning. Start with robust base models and refine performance through prompt engineering, RAG for dynamic data, and agent skills for complex workflows. This approach minimizes development overhead and resource expenditure, reserving fine-tuning (like LoRA) only for critical bottlenecks such as latency or consistency that cannot be resolved otherwise.

Key insights

Exhaust non-training LLM optimization methods like RAG and prompt engineering before considering fine-tuning.

Principles

• Prioritize base models and prompt engineering.
• Defer fine-tuning until other options fail.
• Address knowledge gaps with RAG or agents.

Method

The framework proposes a sequential approach: start with a base LLM, then add prompt engineering, RAG, agent skills, and finally consider fine-tuning (e.g., LoRA) for specific bottlenecks.

In practice

• Use GPT-4, Claude, or Gemini first.
• Implement RAG for internal documents.
• Apply LoRA for latency issues.

Topics

• LLM Models
• Decision Framework
• Prompt Engineering
• Retrieval-Augmented Generation
• Agent Skills
• Fine-tuning
• LoRA

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Is Fine-Tuning Still Needed? LLMs, RAG, & LoRA — General LLMs increasingly match fine-tuned models, making weight-based customization a narrower, specialized need.
• Before You Train a New LLM: Two AI Customization Ladders Every Full-Stack Engineer Should Know — Effective LLM customization involves a decision tree across agent and model layers, not a linear upgrade path.
• Environment-Grounded Automated Prompt Optimization for LLM Game Agents — The framework automatically optimizes LLM agent prompts in interactive environments using an evolutionary loop and a multi-agent decomposition.
• Fine-Tuning vs RAG vs Prompt Engineering — Effective LLM deployment requires strategically applying prompt engineering, RAG, or fine-tuning based on the specific problem.
• What We’ve Learned From A Year of Building with LLMs — Successful LLM product development requires a holistic approach beyond demos, focusing on robust systems, data, and strategic iteration.
• Providing Greater Access to LLMs with Brandon Duderstadt, Co-Founder and CEO of Nomic AI — Fine-tuning and prompt engineering are crucial for optimizing LLM performance and accessibility.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by LLM on Medium.