I don’t think we are close to “AI scientists”

2023-07-27 · Source: Understanding AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Advanced, long

Summary

LLMs exhibit an uncanny ability to recognize authors from unpublished prose, as demonstrated by ChatGPT identifying Timothy B. Lee from his 2012 "Great Canadian Maple Syrup Heist" essay after nine paragraphs, despite being unable to articulate specific reasons. This phenomenon highlights that LLMs, like humans, possess implicit knowledge, but unlike humans, their learning capacity is frozen post-training. The article explores the architecture of modern AI agents such as Claude Code and OpenClaw, which utilize external file systems, Unix shells, and cron jobs to manage context and persistent state, leading to increased demand for local hardware like Apple Mac minis. However, this reliance on explicit file storage means agents lose implicit knowledge between sessions, hindering their ability to perform complex, context-dependent tasks requiring continuous learning and deep, evolving understanding, making the prospect of "AI scientists" based on current architectures distant.

Key takeaway

For AI Engineers and Directors of AI/ML evaluating agent capabilities for complex, long-term projects, understand that current LLM-based agents, despite their utility, are fundamentally limited by their reliance on explicit file-based memory and lack of continuous implicit learning. This architecture hinders their ability to develop deep, evolving context crucial for roles like "AI scientists." You should prioritize exploring novel agentic frameworks and transformer architectures that enable persistent implicit knowledge acquisition to overcome these limitations.

Key insights

Current LLM-based AI agents lack continuous implicit learning, limiting their ability to achieve human-level scientific or complex knowledge work.

Principles

• LLMs' implicit knowledge is fixed post-training.
• Agent coherence relies on explicit file-based state.
• Human expertise compounds through implicit knowledge.

Method

The architecture for AI agents combines an LLM, Unix shell, file system (often Markdown), and a cron job for a persistent loop and heartbeat.

In practice

• Use local agents like OpenClaw on Mac minis.
• Agents can self-modify by rewriting their own files.
• Compaction can lead to data loss in chatbots.

Topics

• LLM Limitations
• AI Agent Architectures
• Implicit Knowledge
• Continuous Learning
• Context Management
• OpenClaw Framework

Best for: AI Scientist, AI Engineer, Director of AI/ML

Related on AIssential

• 🥇Top AI Papers of the Week — AI advancements focus on improving reasoning, context management, and personalization for LLMs and agents.
• AI scientists produce results without reasoning scientifically — LLM-based scientific agents execute workflows but lack fundamental scientific reasoning patterns, primarily due to the base model.
• ImportAI 449: LLMs training other LLMs; 72B distributed training run; computer vision is harder than generative text — LLM agents can autonomously fine-tune other LLMs, but human performance remains superior, with rapid AI progress and emerging reward hacking behaviors.
• Intelligence as Simulation: Why LLM Agents Need World Models — LLM agents need explicit world models to anticipate consequences and act coherently over time, beyond linguistic plausibility.
• AI keeps getting smarter, so why does it still fail at obvious things? — AI's apparent intelligence stems from pattern matching, not true comprehension, leading to reliability gaps.
• Our New Agentic AI Engineering Course! — Building production-ready AI agents requires extensive engineering beyond basic LLM calls.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Understanding AI.