No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot

2026-06-18 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

A mixed-methods think aloud study with 27 professional developers and students explored how cognitive diversity shapes interactions with GitHub Copilot Chat. Conducted between March 27 and May 28, 2025, the study identified 5 distinct "interaction modes" and 10 underlying needs that drive developers' use of the programming assistant. These modes include Navigator, Autopilot, Deputy, Technician, and Scholar, each characterized by specific prompting patterns and goals. The research links these interaction patterns and needs to developers' problem-solving styles and experience profiles, revealing how individual cognitive diversity influences tool usage. The findings form a conceptual model that explains the complex interplay between developer needs, interaction modes, and individual/contextual factors, offering insights for designing more inclusive programming assistants.

Key takeaway

For AI Product Managers designing programming assistants, recognize that your users exhibit diverse problem-solving styles and needs. You should move beyond generic solutions, providing granular support for distinct interaction modes like "Navigator" or "Technician." Consider offering customizable interfaces or adaptive responses that cater to individual preferences for delegation versus control, ensuring your tool effectively supports varied workflows and cognitive profiles.

Key insights

Developer interaction with conversational AI assistants is shaped by diverse cognitive profiles and underlying needs.

Principles

• Cognitive diversity impacts LLM tool adoption.
• Needs for control and delegation often compete.
• One-size-fits-all AI support is insufficient.

Method

A mixed-methods think aloud study with 27 participants, combining topic modeling of prompt annotations with qualitative analysis of transcripts, identified interaction modes and needs.

In practice

• Design programming assistants for varied interaction modes.
• Provide granular support for specific developer needs.
• Clearly communicate AI assistant capabilities.

Topics

• GitHub Copilot Chat
• Cognitive Diversity
• Developer Experience
• Human-AI Interaction
• Problem-Solving Styles
• Programming Assistants

Code references

• features/copilot
• tastejs/todomvc
• features/codespaces
• educational-technology-collective/vscode-telemetry

Best for: Research Scientist, AI Scientist, AI Engineer, AI Product Manager

Related on AIssential

• New ChatGPT Library Explained & More AI News You Can Use — AI platforms are evolving to support document-centric workflows and simplified multiplayer application development.
• Presentation: Choosing Your AI Copilot: Maximizing Developer Productivity — AI copilots offer significant developer productivity gains, especially when integrated into broader workflows.
• I Built a Full AI Engineering Team Inside GitHub Copilot — Here’s How It Works — Structuring AI interactions with agents, prompts, and instructions enhances large language model capabilities for complex workflows.
• Peter Yang on Small Teams, Coding Agents, and Why Human Ambition Has No Ceiling — AI agents are transforming software interaction and development, enabling radically smaller teams and automating knowledge work through conversational interfaces.
• Meet Shapes, the app bringing humans and AI into the same group chats — Integrating AI into group chats with human oversight can mitigate risks associated with isolated AI interactions.
• Designing for tomorrow’s programming workflows — AI-powered tools are transforming programming workflows, necessitating a re-evaluation of API and library design.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.