Shopify/liquid: Performance: 53% faster parse+render, 61% fewer allocations

2026-03-13 · Source: Simon Willison's Weblog · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Advanced, quick

Summary

Shopify CEO Tobias Lütke significantly optimized Liquid, Shopify's open-source Ruby template engine, achieving a 53% faster parse and render time and 61% fewer memory allocations. This performance boost resulted from 93 commits across approximately 120 automated experiments, leveraging a variant of Andrej Karpathy's "autoresearch" system. Lütke initiated the process with an `autoresearch.md` prompt and an `autoresearch.sh` script to guide a coding agent, Pi, in identifying and implementing micro-optimizations. Key improvements included replacing `StringScanner` with `String#byteindex` for a ~12% parse time reduction, implementing pure-byte `parse_tag_token` to eliminate costly `StringScanner#string=` resets, and caching small integer `to_s` conversions to avoid 267 allocations per render. This effort demonstrates the power of robust test suites and AI-driven code optimization.

Key takeaway

For engineering leaders evaluating AI-assisted development, this case demonstrates that coding agents, when paired with comprehensive test suites and clear benchmarking, can deliver substantial performance improvements even in legacy systems. Your teams should consider integrating "autoresearch" patterns to identify and implement micro-optimizations, potentially freeing up senior engineers for more strategic tasks while still driving significant technical debt reduction and efficiency gains.

Key insights

AI-driven "autoresearch" can yield significant performance gains even in mature, highly optimized codebases.

Principles

• Robust test suites enable agent-driven optimization.
• Benchmarking scripts make "faster" an actionable goal.
• Coding agents empower high-interruption roles.

Method

An agent brainstorms improvements, experiments one-by-one, and reports benchmark scores, maintaining state in an `autoresearch.jsonl` file.

In practice

• Use `String#byteindex` for faster tokenization.
• Cache small integer `to_s` conversions.
• Implement pure-byte parsing for efficiency.

Topics

• Coding Agents
• Performance Optimization
• Ruby Template Engine
• Autoresearch
• Software Benchmarking

Code references

• Shopify/liquid
• karpathy/autoresearch
• karpathy/nanochat
• badlogic/pi-mono
• davebcn87/pi-autoresearch

Best for: AI Architect, CTO, VP of Engineering/Data, Software Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

• Coding Agents Meet Data Science — AI coding agents boost developer productivity but introduce new bottlenecks and require specialized adaptation for data science workflows.
• How to Improve Claude Code Performance with Automated Testing — Automated and effective testing is crucial for maximizing coding agent efficiency and overcoming development bottlenecks.
• Andrej Karpathy's new open source 'autoresearch' lets you run hundreds of AI experiments a night — with revolutionary implications — AI agents can autonomously conduct and optimize experiments, accelerating research and discovery across diverse fields.
• Introducing the agent quality loop: AgentCore Optimization now in preview — AgentCore Optimization automates AI agent improvement through data-driven recommendations and rigorous validation.
• True Positive Weekly #153 — The AI landscape is rapidly evolving across benchmarks, verification, agent skills, and economic integration.
• Autoresearch, Agent Loops and the Future of Work — AI agentic loops enable autonomous, iterative experimentation and continuous improvement across various domains.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Simon Willison's Weblog.