What Actually Happens When You Type Into ChatGPT or Claude From Keystroke to Answer?

2026-05-23 · Source: Deep Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

Large Language Models (LLMs) like ChatGPT and Claude process user input through a sophisticated six-step computing pipeline, transforming keystrokes into articulated responses. Initially, text undergoes Tokenization, where it is broken into fragments and assigned numerical IDs, such as "The chef added a pinch of salt." becoming [464, 7392, 2301, 257, 18239, 295, 4122, 13]. These numerical tokens then enter an Embedding Space, converting them into vectors that represent conceptual meanings. The Attention Layer subsequently establishes context by dynamically linking words, resolving ambiguities like "pinch" as a cooking measurement. Next, the Neural Core processes this contextual data, outputting raw prediction scores called Logits for potential next tokens. The Softmax function then converts these Logits into probabilities, influenced by the "Temperature" hyperparameter, which dictates the creativity or safety of the output. Finally, through Autoregression, the highest probability token is selected, translated back to text, and appended to the prompt, restarting the cycle until an End-of-Sequence token is generated.

Key takeaway

For AI Engineers and ML practitioners aiming to optimize LLM behavior, understanding the internal pipeline from tokenization to autoregression is crucial. You gain the ability to control model outputs at a granular level, moving beyond basic prompt engineering. This knowledge allows you to precisely manipulate hyperparameters like Temperature and logit layers, enabling tasks such as enforcing strict JSON schemas, filtering undesirable vocabulary, or fine-tuning specialized chatbots for specific corporate needs.

Key insights

LLMs transform human language into mathematical representations, process context, and iteratively predict the next token to generate responses.

Principles

• Text becomes numerical tokens and vectors.
• Self-attention resolves contextual ambiguities.
• Autoregression iteratively builds responses.

In practice

• Control output via logit layers.
• Force strict JSON schemas.
• Fine-tune corporate chatbots.

Topics

• Large Language Models
• Tokenization
• Vector Embeddings
• Self-Attention Mechanism
• Autoregressive Generation
• LLM Hyperparameters

Best for: AI Engineer, Machine Learning Engineer, AI Student

Related on AIssential

• What Really Happens When You Ask ChatGPT a Question? — LLMs operate as advanced next-token prediction systems, utilizing Transformers for contextual language understanding.
• Why ChatGPT Is More Than Autocomplete — LLMs generate coherent responses by repeatedly rebuilding a rich internal state from growing text, not by simple sequential prediction.
• If you’ve been using ChatGPT or Claude without quite knowing what’s happening under the hood, this… — Large language models fundamentally predict the next chunk of text, with tokens and prompts governing their operation and output quality.
• Building LLM From Scratch: Understanding How Large Language Models Work — Large Language Models fundamentally predict the next token by processing numerical representations of text through attention mechanisms.
• The Sequence AI of the Week #859: Reading Claude’s Mind in English: A Note on Natural Language Autoencoders — Natural Language Autoencoders provide direct, unsupervised English explanations of LLM activations.
• Deep Dive into LLMs like ChatGPT — LLMs are trained in stages, from vast internet data to human-guided refinement, enabling complex, yet fallible, capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Deep Learning on Medium.