FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval

2026-07-02 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

FlowCIR introduces a novel paradigm for Zero-shot Composed Image Retrieval (ZS-CIR), a task involving retrieving images by editing a reference image with natural language instructions without domain-specific annotated triplets. Unlike prior methods that rely on lossy textual inversion for composing reference images and instructions, FlowCIR frames ZS-CIR as conditional semantic transport between reference and target embeddings. It employs conditional flow matching to learn a lightweight transport field, which maps the instruction representation to a target-aligned query embedding, conditioned on the reference image. This approach is computationally efficient, requiring approximately 10x fewer training resources than textual-inversion-based methods, as it operates on pre-extracted VLM embeddings and only trains a small transport module. FlowCIR also incorporates an inference-only Multi-Negative Steering strategy to mitigate VLM limitations in handling negation and removal queries, enhancing robustness. The model demonstrates strong and competitive performance across standard CIR benchmarks.

Key takeaway

For Computer Vision Engineers developing zero-shot composed image retrieval systems, FlowCIR offers a significantly more efficient training approach. You should consider adopting conditional flow matching to integrate reference images and natural language instructions, reducing training resources by approximately 10x compared to textual inversion methods. Furthermore, implementing an inference-only Multi-Negative Steering strategy can enhance your system's robustness against complex negation-heavy queries, improving overall retrieval accuracy.

Key insights

FlowCIR redefines ZS-CIR as conditional semantic transport, using flow matching for efficient, robust image retrieval.

Principles

• Semantic transport can unify reference images and instructions.
• Training lightweight modules on VLM embeddings is efficient.
• Inference-only steering improves VLM negation handling.

Method

FlowCIR learns a transport field via conditional flow matching, mapping instruction representations to target-aligned query embeddings conditioned on reference images, then applies Multi-Negative Steering during inference.

In practice

• Use pre-extracted VLM embeddings for efficiency.
• Implement conditional flow matching for semantic alignment.
• Apply Multi-Negative Steering for negation robustness.

Topics

• Zero-Shot Composed Image Retrieval
• Conditional Flow Matching
• Semantic Transport
• VLM Embeddings
• Multi-Negative Steering
• Computer Vision

Best for: Research Scientist, AI Scientist, Computer Vision Engineer

Related on AIssential

• Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasets — Genuine zero-shot image retrieval benchmarks require novel, consistent data and robust evaluation to counter inflated performance from pre-trained models.
• FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching — FlowMaps uses latent flow matching to predict multimodal future object locations, improving robot navigation in dynamic 3D environments.
• STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval — Training-free zero-shot CIR improves by refining LLM-generated captions and using set-to-set alignment for diverse compositions.
• Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study — Zero-shot semantic re-identification using VLMs generates textual descriptions for robust, interpretable identity matching in autonomous driving.
• Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticism — Multi-stage reasoning with self-criticism significantly enhances zero-shot composed image retrieval by reducing generative errors.
• Capacity-Controlled Multi-View Stylization of 3D Gaussian Splatting — Optimal transport with capacity control enables consistent multi-view stylization for 3D Gaussian Splatting by mitigating feature reuse.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.