LAI #135: The Useful Part of Graph Engineering Is Not the Graph

2026-07-23 · Source: Towards AI - Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, medium

Summary

LAI #135 introduces "Graph Engineering" as a new term for orchestrating multiple AI agent loops, noting Anthropic's 2024 work already covered these concepts. The article emphasizes that while drawing graphs aids process setup, the core value lies in taking orchestration seriously with models like 5.6 and Fable. A critical AI tip highlights that forcing structured outputs, like JSON, can degrade a model's answer quality, especially for complex reasoning tasks, even if the parse rate is 100%. It advises evaluating answer quality and schema validity separately. The brief also covers a journalist's misuse of p-values to "prove" chocolate aids weight loss, and curates articles on a multi-agent support triage system, DPO fine-tuning in NumPy, an orchestrator pattern for self-managing AI agents, and a self-hosted LLMOps stack using Langfuse. A community poll shows Anthropic as the top dream workplace, with 18% preferring to found their own company.

Key takeaway

For AI Engineers designing LLM applications, carefully evaluate the impact of structured outputs on model reasoning. If you require JSON or other schemas, first test your model's performance in free-form, then with the minimal necessary structure, measuring answer quality and parse rate separately. Do not assume a 100% parse rate means correct answers. Consider simplifying schemas or post-processing free-form responses in code to preserve accuracy, especially for complex tasks or multi-agent orchestrations.

Key insights

Forcing structured outputs can degrade LLM reasoning; evaluate answer quality and schema validity separately.

Principles

• Graph engineering formalizes multi-agent orchestration.
• Complex schemas can reduce LLM reasoning accuracy.
• P-values require careful interpretation beyond statistical significance.

Method

Evaluate structured output impact by comparing free-form task quality against minimal schema performance. Measure answer quality and schema validity independently to avoid misinterpreting parse rates.

In practice

• Simplify schemas if LLM accuracy decreases, or package free-form answers in code.
• Deploy multi-agent systems using local Llama3 or GPT-4o-mini.
• Implement self-hosted LLMOps with Langfuse for tracing and evals.

Topics

• Graph Engineering
• LLM Orchestration
• Structured Outputs
• Multi-Agent Systems
• LLMOps
• DPO Fine-tuning

Code references

• mahmoudmoe84/review-toolkit

Best for: AI Architect, NLP Engineer, AI Scientist, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• LAI #135: The Useful Part of Graph Engineering Is Not the Graph — Structured outputs can degrade LLM answer quality despite perfect parsing, requiring separate quality evaluation.
• DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning — LLMs excel at orchestrating graph algorithms via tools but struggle with compositional multi-hop graph reasoning.
• 🥇Top AI Papers of the Week — Agent systems benefit from structured memory, skill evolution, and robust evaluation beyond simple precision.
• Does the Structured Output schema become part of the prompt? — Structured Outputs ensure LLM responses conform to a predefined schema through dynamic constrained decoding, not by modifying the prompt text.
• StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows — Artifact-centered evaluation reveals workflow failures LLM agents miss in structural engineering.
• Are Large Language Models Suitable for Graph Computation? Progress and Prospects — LLMs show promise for simple graph tasks but struggle with large-scale, exact graph computation, necessitating role-based integration.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.