LLMs just have more time and energy?

2026-06-22 · Source: AI on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

Andrej Karpathy's insights into Large Language Models (LLMs) highlight their "self-consciousness" and propensity for hallucinations, which stem from their training objective to predict plausible next tokens rather than internalizing uncertainty. LLMs learn to imitate confident human tones and even humility from human-annotated data, making their incorrect assertions sound convincing. The fundamental difference between human and LLM learning lies in the latter's ability to compress vast amounts of text into billions or trillions of parameters through relentless, uninterrupted training, unlike humans who learn continuously through embodied experience, emotions, and physical interaction, and also experience fatigue and forgetting. This intense, fatigue-free repetition enables LLMs to produce outputs like flawless code. Ultimately, LLMs are presented as a "mirror" of human language, reflecting human-shaped intelligence derived from their training material.

Key takeaway

For Machine Learning Engineers developing or deploying LLMs, understand that their "confidence" and "humility" are statistical imitations from training data, not internal states. This insight should inform your approach to mitigating hallucinations and designing robust evaluation metrics. You must account for the model's lack of true uncertainty, focusing on data quality and prompt engineering to guide its pattern-matching capabilities effectively.

Key insights

LLMs imitate human language patterns, including confidence and humility, through relentless, large-scale data compression, acting as a mirror of human communication.

Principles

• LLMs predict plausible next tokens, not internalize uncertainty.
• Human language data shapes LLM "intelligence."
• Relentless training enables LLM pattern recognition.

Topics

• Large Language Models
• LLM Hallucinations
• Model Training
• Human Language Data
• AI Imitation
• Andrej Karpathy

Best for: AI Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• How Large Language Models Actually Work (Explained Simply) — LLMs are statistical prediction engines, not reasoning entities, whose capabilities emerge from massive scale and sophisticated attention mechanisms.
• The AI Answer You Trusted Most Was Probably the Wrong One — AI models are more confident when incorrect, a byproduct of training that rewards human preference for decisive answers.
• Why AI Makes Things Up: Understanding Hallucinations in Language Models — LLM hallucinations are a structural outcome of probabilistic training, not a fixable bug, requiring systemic mitigation.
• Pokémon Built a Robot Brain — LLMs are trained in stages, from broad knowledge acquisition to fine-tuned conversational and reasoning capabilities.
• When large language models are reliable for judging empathic communication — LLMs reliably judge empathic communication, matching experts and outperforming crowdworkers when properly benchmarked.
• LLMs Can Better Capture Human Judgments--With the Right Prompts — Asking better questions to LLMs through specific prompting strategies improves their ability to capture human judgments and response distributions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI on Medium.