Fable 5: Does the smartest LLM decompile better?

· Source: LLM on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Gaming & Interactive Media · Depth: Intermediate, medium

Summary

A benchmark using Mizuchi, a tool for guiding LLMs in matching decompilation, evaluated the new Fable 5 model against Claude Sonnet 4.6. The study utilized a dataset of 60 functions from retro games, specifically 30 from Sonic Advance 3 and 30 from Animal Forest. Fable 5 demonstrated superior performance, matching 5 functions that Sonnet consistently failed on and stabilizing nearly all of Sonnet's flaky matches, even under stricter retry (6 vs. 12) and timeout (3:40 vs. 7:00) limits in initial runs. All Fable 5 matches occurred within the first 3 attempts. However, this improved capability comes at a significant cost, with Fable 5 being 3.5-4.6 times more expensive than Sonnet, despite emitting fewer tokens. Its session limit is also 50% smaller, requiring runs to be split.

Key takeaway

For reverse engineers or game developers engaged in matching decompilation, you should consider a tiered LLM strategy. While Fable 5 offers superior accuracy and stability on challenging functions, its 3.5-4.6x higher cost makes it impractical for all tasks. Integrate Fable 5 as a specialized tool for functions that cheaper models like Sonnet consistently fail on, or use it to generate resources that improve less-capable models. Avoid a wholesale replacement to manage operational expenses effectively.

Key insights

Fable 5 excels at matching decompilation for difficult functions but is significantly more expensive than prior models.

Principles

Method

Mizuchi guides LLMs for matching decompilation by converting assembly to C source code that produces byte-for-byte identical machine code. Benchmarking involves multiple runs with varying retry limits and soft timeouts.

In practice

Topics

Code references

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Software Engineer

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by LLM on Medium.