Self Driving Products: Product Signals to Pull Requests — Joshua Snyder, PostHog

2026-06-10 · Source: AI Engineer · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Advanced, long

Summary

PostHog is developing a "Self Driving Products" pipeline designed to automate the process of identifying and fixing product issues by transforming observability data into actionable pull requests. This system aims to replace the slow, manual cycle of engineers interpreting dashboards, creating issues, and then developing fixes. The pipeline ingests trillions of events monthly from various sources, applying a safety filter and normalizing signals. A key innovation is grouping disparate signals (e.g., error stack traces, Slack messages, session replays) into unified "reports" by generating LLM queries rather than relying on structural embedding similarity. These reports are then processed by a Claude agent SDK-powered research agent in a Modal sandbox, which uses internal and external tools like Linear and Notion to diagnose problems, assign priority, and identify reviewers. The system then determines actionability, either queuing for human input, gathering more data, or automatically executing code to generate and iterate on a pull request until it passes CI.

Key takeaway

For MLOps Engineers or AI Engineers building autonomous systems, consider PostHog's approach to automating product fixes. Your focus should shift from dashboard interpretation to reviewing agent-generated pull requests. Implement robust evaluation frameworks with representative data early on. When designing data pipelines for agents, prioritize query generation over direct signal embedding for diverse data types to ensure accurate problem grouping and reduce noise. This can significantly accelerate your development cycle and free up engineering time.

Key insights

PostHog's pipeline automates product issue resolution by converting observability data into self-correcting pull requests.

Principles

• Evals on representative data are crucial for pipeline iteration.
• Embed queries, not raw signals, to overcome structural similarity bias.
• Ensure problem specificity before agent execution to avoid noise.

Method

The pipeline ingests, normalizes, and groups product signals into reports, which a research agent diagnoses. An actionability step determines if an agent can automatically generate and iterate on a pull request.

In practice

• Use LLM classifiers for safety filtering public data streams.
• Integrate external tools (e.g., Linear, Notion) to ground research agents.
• Snapshot agent sandboxes to rehydrate and iterate on PR feedback.

Topics

• Self-Driving Products
• Observability Automation
• LLM Agents
• Pull Request Automation
• Product Analytics
• Embedding Models

Best for: AI Architect, AI Product Manager, Software Engineer, MLOps Engineer, AI Engineer

Related on AIssential

• How Long Until We Call AI Agents Data Products — Treating AI agents as data products with robust product analytics and feedback loops is crucial for their success.
• Improve Your Agentic AI Trading With a Great Data Pipeline — High-quality, diverse data pipelines are crucial for successful agentic AI trading, outweighing model sophistication.
• Introducing Showboat and Rodney, so agents can demo what they’ve built — Showboat and Rodney enable coding agents to autonomously demonstrate their work through verifiable documentation and browser automation.
• PostHog / posthog — PostHog offers a unified, open-source platform for product development, integrating analytics, experimentation, and user feedback.
• New in Opik: Native OpenClaw Observability, Custom Dashboards, Optimization UI Upgrades, & More! — Opik's updates enhance AI development visibility through agent observability, prompt optimization, and flexible dashboards.
• Let the AI Do the Experimenting — Autonomous AI agents can iteratively optimize complex problems by experimenting, measuring, and adapting code.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.