Tasks Integrated Networks: Joint Detection and Retrieval for Image Search

doi:10.1109/TPAMI.2020.3009758

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	Tasks Integrated Networks: Joint Detection and Retrieval for Image Search
	Zhang, Lei 2; He, Zhenwei 2; Yang, Yi 3; Wang, Liang 4; Gao, X-B 1
刊名	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
	2022
卷号	44 期号:1 页码:456-473
关键词	Image search object detection re-identification retrieval deep learning
ISSN号	0162-8828
DOI	10.1109/TPAMI.2020.3009758
通讯作者	Zhang, Lei(leizhang@cqu.edu.cn)
英文摘要	The traditional object (person) retrieval (re-identification) task aims to learn a discriminative feature representation with intra-similarity and inter-dissimilarity, which supposes that the objects in an image are manually or automatically pre-cropped exactly. However, in many real-world searching scenarios (e.g., video surveillance), the objects (e.g., persons, vehicles, etc.) are seldom accurately detected or annotated. Therefore, object-level retrieval becomes intractable without bounding-box annotation, which leads to a new but challenging topic, i.e., image-level search with multi-task integration of joint detection and retrieval. In this paper, to address the image search issue, we first introduce an end-to-end Integrated Net (I-Net), which has three merits: 1) A Siamese architecture and an on-line pairing strategy for similar and dissimilar objects in the given images are designed. Benefited by the Siamese structure, I-Net learns the shared feature representation, because, on which, both object detection and classification tasks are handled. 2) A novel on-line pairing (OLP) loss is introduced with a dynamic feature dictionary, which alleviates the multi-task training stagnation problem, by automatically generating a number of negative pairs to restrict the positives. 3) A hard example priority (HEP) based softmax loss is proposed to improve the robustness of classification task by selecting hard categories. The shared feature representation of I-Net may restrict the task-specific flexibility and learning capability between detection and retrieval tasks. Therefore, with the philosophy of divide and conquer, we further propose an improved I-Net, called DC-I-Net, which makes two new contributions: 1) two modules are tailored to handle different tasks separately in the integrated framework, such that the task specification is guaranteed. 2) A class-center guided HEP loss (C2HEP) by exploiting the stored class centers is proposed, such that the intra-similarity and inter-dissimilarity can be captured for ultimate retrieval. Extensive experiments on famous image-level search oriented benchmark datasets, such as CUHK-SYSU dataset andPRWdataset for person search and the large-scaleWebTattoo dataset for tattoo search, demonstrate that the proposed DC-I-Net outperforms the state-of-the-art tasks-integrated and tasks-separated image search models.
资助项目	National Science Fund of China[61771079] ; Chongqing Youth Talent Program
WOS关键词	RECOGNITION
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE COMPUTER SOC
WOS记录号	WOS:000728561300033
资助机构	National Science Fund of China ; Chongqing Youth Talent Program
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/46800]
专题	自动化研究所_智能感知与计算研究中心
通讯作者	Zhang, Lei
作者单位	1.Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China 2.Chongqing Univ, Sch Microelect & Commun Engn, Chongqing 400044, Peoples R China 3.Univ Technol Sydney, Ctr Artificial Intelligence, Ultimo, NSW 2007, Australia 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Lei,He, Zhenwei,Yang, Yi,et al. Tasks Integrated Networks: Joint Detection and Retrieval for Image Search[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2022,44(1):456-473.
APA	Zhang, Lei,He, Zhenwei,Yang, Yi,Wang, Liang,&Gao, X-B.(2022).Tasks Integrated Networks: Joint Detection and Retrieval for Image Search.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,44(1),456-473.
MLA	Zhang, Lei,et al."Tasks Integrated Networks: Joint Detection and Retrieval for Image Search".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 44.1(2022):456-473.