Analyze the Thinking Traces of GLM 5.1 & Qwen 3.6plus

2026-04-08 · Source: Discover AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, long

Summary

A live test compared the performance of Qwen 3.6 Plus and GLM 5.1 large language models on an "elevator optimization problem" using the arena.ai platform. The task required finding the shortest path from floor 0 to 50 with specific mathematical button presses, limited energy/token resources, and various traps and time inversions. Initially, Qwen 3.6 Plus produced a 12-button press solution that overshot floor 50, while GLM 5.1 fell into a local minimum with a 2-minute, 42-second, longer sequence. After a hard constraint of 50 floors was enforced, GLM 5.1 unexpectedly found an optimal 8-step solution after nearly 8 minutes, attributed to a "lucky" statistical outcome. Qwen 3.6 Plus, despite self-correction attempts, struggled with validation and eventually entered a chaotic state, losing its reasoning capability and producing invalid, lengthy sequences.

Key takeaway

For AI Engineers evaluating LLMs for complex optimization or logical reasoning tasks, this comparison highlights the unpredictable nature of statistical models. You should implement robust validation steps and consider that optimal solutions might arise from chance, not consistent reasoning. Be prepared to provide explicit constraints and analyze reasoning traces to understand model behavior, especially when models enter chaotic states or local minima.

Key insights

LLM performance on complex optimization tasks can be highly variable and influenced by statistical chance.

Principles

• LLMs can get trapped in local minima.
• Hard constraints improve solution validity.
• Reasoning traces reveal internal processes.

Method

The test involved an elevator optimization problem with mathematical functions, resource limits, and traps, requiring causal reasoning and logic puzzle-solving, with reasoning traces analyzed for internal thought processes.

In practice

• Provide ample execution time for self-correction.
• Enforce hard constraints to guide LLM behavior.
• Analyze reasoning traces for debugging.

Topics

• GLM 5.1
• Qwen 3.6 Plus
• Elevator Optimization Problem
• LLM Reasoning
• Model Performance Analysis

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• TEST GLM-5.2 MAX on Z.ai: Not Perfect - but real Good 👍 — GLM 5.2 MAX demonstrates strong causal reasoning but struggles with multi-layered strategic optimization in complex logic puzzles.
• How reliable are LLMs when it comes to playing dice? — Current LLMs struggle with probabilistic reasoning, especially on counterintuitive problems and when faced with subtle biases or misleading prompts.
• TRACE: Toulmin-based Reasoning Assessment through Constructive Elements for LLM CoT Evaluation — TRACE evaluates LLM Chain-of-Thought reasoning by analyzing argument construction using Toulmin's theory and Flavell's metacognition.
• WHAT? Qwen3.6-A3B Could Solve This? — Instruction tuning and reasoning trace analysis can significantly enhance small MoE model performance on complex tasks.
• Benchmarks in Leipzig — Iterative, multi-stage evaluation reveals impressive mathematical reasoning capabilities in Large Language Models.
• Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning — LLMs generate deep reasoning traces but their decisions are driven by shallow, myopic planning, unlike humans.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Discover AI.