VietFashion: Benchmarking Sketch-Text Composed Image Retrieval for Cultural Outfits

2026-06-11 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, quick

Summary

VietFashion is a new benchmark dataset designed for sketch-text composed image retrieval, specifically addressing the challenges of cultural garments like the Vietnamese Ao Dai. Standard AI models struggle with the subtle structural and symbolic details of such outfits. This dataset combines 650 initial hand-drawn sketches with over 21,000 photorealistic images generated by generative models, each paired with aligned captions. Textual prompts, derived from fashion magazines for authenticity, describe detailed outfit attributes. VietFashion supports a multi-target retrieval setting, allowing a single query to match multiple valid results, reflecting design intent ambiguity. The benchmark establishes standardized evaluation protocols and reveals significant performance gaps in current state-of-the-art methods for modeling fine-grained cultural semantics and multi-modal composition, making it a challenging resource for fashion retrieval research.

Key takeaway

For computer vision engineers developing fashion retrieval systems, VietFashion highlights critical gaps in handling cultural nuances. You should prioritize models capable of fine-grained semantic understanding and robust multi-modal composition, especially when dealing with diverse global aesthetics. Consider adopting multi-target retrieval to better reflect design ambiguity. This benchmark offers a valuable resource to test and improve your algorithms for culturally sensitive applications.

Key insights

Cultural garment retrieval requires benchmarks that capture subtle structural and semantic details.

Principles

• Cultural outfits demand fine-grained semantic modeling.
• Multi-modal queries improve design intent capture.
• Generative models can expand limited sketch data.

Method

The VietFashion benchmark initializes with hand-drawn sketches, expands with generative models for photorealistic images and captions, and uses fashion magazine text for authentic attributes. It employs a multi-target retrieval setting.

In practice

• Use sketch-text queries for cultural fashion.
• Implement multi-target retrieval for ambiguity.
• Integrate generative models for dataset expansion.

Topics

• Sketch-Text Retrieval
• Cultural Garments
• Ao Dai
• Generative Models
• Multi-modal Composition
• Fashion Benchmarking

Best for: Research Scientist, AI Scientist, Computer Vision Engineer

Related on AIssential

• Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing — Retrieval-augmented vision-language models effectively generate culturally accurate captions for low-resource heritage data.
• Sketch and Text Synergy: Fusing Structural Contours and Descriptive Attributes for Fine-Grained Image Retrieval — Fusing sketch and text modalities enhances fine-grained image retrieval by combining structural and descriptive cues.
• ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China — VLMs demonstrate strong visual recognition but struggle with culturally-grounded reasoning on heritage sites.
• FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learning — A unified MLLM-based framework achieves versatile fashion image retrieval by adapting query representations and balancing task optimization.
• Ukrainian Visual Word Sense Disambiguation Benchmark — Multimodal LLMs significantly underperform in Ukrainian Visual-WSD compared to English, revealing a critical language resource gap.
• From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG — Element-level visual evidence retrieval significantly enhances multimodal RAG verifiability and performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.