Does an AI Think in Circles? I Opened One Up to Find Out

2026-07-12 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

An investigation into how language models represent ideas internally explored whether concepts are stored as linear directions or curved shapes like circles. The author specifically examined Qwen3-1.7B, a small open model, to search for a circular representation using compass bearings. This concept, ranging from 0 degrees (north) to 355 degrees (near north again), was chosen for its inherent circularity and continuity, allowing for the detection of smooth transitions between related ideas within the model's internal structure. This research aims to move beyond small toy models to understand the internal geometry of ideas in real-world language models, challenging the prevailing view of ideas as simple linear directions.

Key takeaway

For AI Scientists investigating neural network interpretability, this exploration into Qwen3-1.7B's internal geometry challenges assumptions about linear idea representations. You should consider probing models for non-linear structures like circles or spirals, especially when dealing with continuous or cyclical concepts. This could lead to more nuanced understanding and potentially novel methods for steering or editing model behavior beyond simple directional pushes.

Key insights

Language models might represent ideas as curved shapes, not just linear directions, as explored in Qwen3-1.7B with compass bearings.

Principles

• Ideas in LMs might reside on curved manifolds.
• Circular concepts test non-linear idea representation.
• Real models may differ from toy model findings.

Method

The method involved analyzing Qwen3-1.7B's internal structure for circular idea representations, specifically using continuous compass bearings (0-360 degrees) to detect smooth paths between related concepts.

In practice

• Analyze internal geometry of LMs.
• Use circular data to probe model structure.
• Investigate Qwen3-1.7B for non-linear embeddings.

Topics

• Language Model Internals
• Qwen3-1.7B
• Neural Network Interpretability
• Idea Representation
• Circular Embeddings
• Model Geometry

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• Trajectory Geometry of Transformer Representations Across Layers — Trajectory geometry offers a principled, probe-free lens to understand Transformer representation evolution across layers.
• Zen and the Art of the Koan and Hallucinations — Reasoning models' "overthinking" and "hallucinations" parallel Zen Koans, revealing AI's inherent syntactic limitations.
• Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry — NLLB-200's internal geometry reflects language-universal conceptual structures, akin to human multilingual cognition.
• Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model — LLMs represent scientific mechanisms more reliably through state transformations than static states.
• How Major Reasoning Models Converge to the Same “Brain” as They Model Reality Increasingly Better — AI models, regardless of modality, converge on a unified internal representation of reality as they scale.
• Verbalizable Representations Form a Global Workspace in Language Models — Large language models develop a "J-space" of verbalizable representations akin to a human global workspace, revealing internal thought processes.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.