Cross-Image-Attention for Conditional Embeddings in Deep Metric Learning

Kotovenko, Dmytro; Ma, Pingchuan; Milbich, Timo und Ommer, Björn (2023): Cross-Image-Attention for Conditional Embeddings in Deep Metric Learning. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17-24 June 2023. Institute of Electrical and Electronics Engineers (IEEE) (Hrsg.), In: CR-FIQA: Face Image Quality Assessment by Learning Sample Relative Classifiability, Piscataway, NJ: IEEE. S. 1170-1181

Abstract

Learning compact image embeddings that yield seman-tic similarities between images and that generalize to un-seen test classes, is at the core of deep metric learning (DML). Finding a mapping from a rich, localized image feature map onto a compact embedding vector is challenging: Although similarity emerges between tuples of images, DML approaches marginalize out information in an individ-ual image before considering another image to which simi-larity is to be computed. Instead, we propose during training to condition the em-bedding of an image on the image we want to compare it to. Rather than embedding by a simple pooling as in standard DML, we use cross-attention so that one image can iden-tify relevant features in the other image. Consequently, the attention mechanism establishes a hierarchy of conditional embeddings that gradually incorporates information about the tuple to steer the representation of an individual image. The cross-attention layers bridge the gap between the origi-nal unconditional embedding and the final similarity and al-low backpropagtion to update encodings more directly than through a lossy pooling layer. At test time we use the re-sulting improved unconditional embeddings, thus requiring no additional parameters or computational overhead. Ex-periments on established DML benchmarks show that our cross-attention conditional embedding during training im-proves the underlying standard DML pipeline significantly so that it outperforms the state-of-the-art.

• Dokumententyp: Konferenzbeitrag (Paper)
• Fakultät: Mathematik, Informatik und Statistik ; Mathematik, Informatik und Statistik > Informatik > Künstliche Intelligenz und Maschinelles Lernen
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik ; 700 Künste und Unterhaltung > 770 Fotografie, Computerkunst
• Ort: Piscataway, NJ
• Bemerkung: ISBN 979-8-3503-0129-8 ; 979-8-3503-0130-4 (ISBN der Printausgabe)
• Sprache: Englisch