Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

2026-07-24 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

Tencent WorkBuddy Bench is a multi-domain evaluation suite designed for coding agents, featuring 260 tasks across four distinct work domains: Code (80 tasks), Web (70 tasks), Office (50 tasks), and Security (60 tasks). This benchmark employs a contamination-resistant methodology, reverse-engineering tasks from real commits or business scenarios and rewriting them as colloquial, role-played requests to prevent web-searchable prompts. Its construction is distribution-informed, aligning task categories and difficulty with real usage patterns. The suite is fully open-source, providing task directories, environment images, evaluation harnesses, tests, and reference solutions for auditable and reproducible results. Tasks run under a unified protocol on CodeBuddy Code and Claude Code harnesses, with diverse, domain-specific scoring mechanisms.

Key takeaway

For AI Engineers developing or evaluating coding agents, traditional benchmarks often inflate performance due to data contamination. You should utilize Tencent WorkBuddy Bench to rigorously assess your agent's capabilities across realistic Code, Web, Office, and Security domains. This benchmark's contamination-resistant design and diverse task types will reveal true reasoning and grounding abilities, helping you identify and address critical gaps in real-world application.

Key insights

Tencent WorkBuddy Bench offers a contamination-resistant, multi-domain benchmark for coding agents, informed by real-world usage and openly auditable.

Principles

• Construct tasks from real scenarios, not public issues.
• Match task distributions to actual usage taxonomies.
• Ensure open release for full auditability.

Method

Tasks are reverse-engineered from real commits/scenarios, rewritten as underspecified, colloquial requests, and packaged in self-contained directories with isolated evaluation assets for sandboxed execution.

In practice

• Reverse-engineer tasks from real commits/scenarios.
• Use diverse scoring (hidden tests, rubrics, deterministic).
• Publish all evaluation assets for transparency.

Topics

• Coding Agents
• Benchmark Suites
• Contamination Resistance
• Software Engineering
• Office Automation
• Cybersecurity Evaluation

Code references

• laude-institute/harbor

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction — The Tencent WorkBuddy Bench offers a contamination-resistant, multi-domain benchmark for coding agents, ensuring robust and reproducible evaluation.
• Beyond SWE-Bench Pro - Where do Agents go from Here? — New benchmarks are crucial for advancing AI coding agents beyond simple issue resolution to complex system understanding and improvement.
• DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve — DeepSWE offers a contamination-resistant coding benchmark with original, long-horizon tasks for robust model evaluation.
• Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement — Real-world code generation demands iterative refinement from underspecified user intent, a capability Asuka-Bench effectively measures.
• StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows — Artifact-centered evaluation reveals workflow failures LLM agents miss in structural engineering.
• WANDR Benchmark: Evaluating Research Agents That Must Search Wide and Deep - Perplexity — Research agents face significant challenges in comprehensive, evidence-backed "wide and deep" data collection, with current systems achieving low F1 scores.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.