FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learning

2026-05-21 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Data Science & Analytics · Depth: Expert, quick

Summary

FashionLens is a unified framework designed for versatile fashion image retrieval, addressing the limitations of existing systems that handle only narrow retrieval tasks. This framework, built upon Multimodal Large Language Models, aims to support diverse query formats and search intentions in e-commerce. To establish a robust data foundation, the authors introduce U-FIRE, a comprehensive benchmark that unifies fragmented fashion datasets and includes two manually curated datasets for generalization testing. FashionLens incorporates a Proposal-Guided Spherical Query Calibrator, which dynamically shifts query representations into task-aligned metric spaces using adaptive spherical linear interpolation. Additionally, it employs a Gradient-Guided Adaptive Sampling strategy to balance optimization by re-weighting tasks based on real-time learning difficulty and data scale. Experiments on U-FIRE demonstrate that FashionLens achieves leading performance across diverse retrieval scenarios and generalizes robustly to unseen tasks. The data and code are publicly available.

Key takeaway

For Machine Learning Engineers developing e-commerce fashion retrieval systems, existing narrow approaches limit versatility. You should evaluate unified frameworks like FashionLens, which leverages Multimodal Large Language Models and adaptive query calibration to handle diverse search intentions. Consider adopting its Proposal-Guided Spherical Query Calibrator and Gradient-Guided Adaptive Sampling strategy to achieve superior performance and robust generalization across varied tasks. Utilizing the U-FIRE benchmark can also provide a comprehensive evaluation foundation for your models.

Key insights

A unified MLLM-based framework achieves versatile fashion image retrieval by adapting query representations and balancing task optimization.

Principles

• Unify fragmented datasets for comprehensive benchmarking.
• Dynamically align query representations to task objectives.
• Adaptively re-weight tasks to balance optimization.

Method

FashionLens uses MLLMs, a Proposal-Guided Spherical Query Calibrator for adaptive spherical linear interpolation of query representations, and a Gradient-Guided Adaptive Sampling strategy to re-weight tasks based on learning difficulty and data scale.

In practice

• Use U-FIRE benchmark for diverse fashion retrieval.
• Implement adaptive query calibration for varied tasks.
• Apply gradient-guided sampling for multi-task learning.

Topics

• Fashion Image Retrieval
• Multimodal Large Language Models
• U-FIRE Benchmark
• Query Calibration
• Adaptive Sampling
• E-commerce AI

Code references

• haokunwen/FashionLens

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval — Training-free zero-shot CIR improves by refining LLM-generated captions and using set-to-set alignment for diverse compositions.
• FASH-iCNN: Making Editorial Fashion Identity Inspectable Through Multimodal CNN Probing — FASH-iCNN reveals fashion's cultural logic by making editorial identity inspectable through multimodal CNN probing.
• LARE: Low-Attention Region Encoding for Text-Image Retrieval — LARE explicitly models low-attention image regions to overcome salience bias in text-image retrieval.
• Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage — CLARA uses visual prototypes and reweighted conformal prediction to resolve CIR ambiguity more effectively than text questions.
• Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking — Adaptive RAG's role shifts from noise mitigation for weaker LLMs to efficiency optimization for stronger ones.
• pplx-embed + Qdrant: Building Production-Grade Semantic Search with Quantization — Perplexity introduced `pplx-embed` on February 26, 2026, a new family of state-of-the-art multilingual text embedding models that utilize diffusion-based pretraining and are built on Qwen3.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.