๐Ÿ˜บ ChatGPT admitted its memory was broken

ยท Source: The Neuron ยท Field: Technology & Digital โ€” Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation ยท Depth: Novice, medium

Summary

OpenAI has significantly upgraded ChatGPT's memory feature, "Dreaming V3," after admitting its previous version, launched in February 2024, had a factual recall accuracy of only 41.5%. The new background process automatically synthesizes conversation history, boosting factual recall to 82.8% and preference adherence from 55.3% to 71.3% in internal tests by 2026. This enhancement also reduced compute costs by 5x, making the feature available to free users, and doubled memory storage for Plus and Pro subscribers. The upgrade allows memory to self-correct over time, adapting to user changes. This improvement is particularly crucial given ChatGPT's recent personal finance features, combining memory with sensitive financial data, highlighting a broader issue of unstated accuracy problems in other AI assistants.

Key takeaway

For AI product managers evaluating user trust and feature reliability, OpenAI's candid disclosure about ChatGPT's past 41.5% memory recall underscores the hidden risks in AI systems. You should proactively audit your own AI assistants' "memory" or personalization features for unstated accuracy issues. This transparency, coupled with the "Dreaming V3" upgrade, sets a precedent for improving user experience and data integration, especially as AI handles more sensitive information like personal finance.

Key insights

OpenAI's memory upgrade for ChatGPT significantly improves recall and adherence, revealing past accuracy issues in AI systems.

Principles

Method

Users can review, edit, or delete their ChatGPT memory summary via Settings โ†’ Personalization โ†’ Memory โ†’ Memory Summary.

In practice

Topics

Best for: CTO, VP of Engineering/Data, Executive, General Interest, Tech Journalist, Director of AI/ML

Related on AIssential

Open in AIssential โ†’

Editorial summary, takeaway, and curation by AIssential. Original article published by The Neuron.