Web-scale image search engines mostly rely on surrounding text features. It is difficult for them to interpret users’ search intention only by query keywords and this leads to ambiguous and noisy search results which are far from satisfactory. Image re-ranking, as an effective way to improve the results of web-based image search, has been adopted by current commercial search engines such as Bing and Google. Given a query keyword, a pool of images are ﬁrst retrieved based on textual information. By asking the user to select a query image from the pool, the remaining images are re-ranked based on their visual similarities with the query image. We have developed techniques that are able to provide satisfactory re-ranking results with minimum user effort. Our methods explore two directoins: better capturing user intention and better model the semantic structure of re-ranking image pool. Both directoins have gained significant improvement for image re-ranking.
Web Image Re-ranking Using Query-Specific Semantic Signatures
We propose a novel image re-ranking framework, which automatically ofﬂine learns different semantic spaces for different query keywords. The visual features of images are projected into their related semantic spaces to get semantic signatures. At the online stage, images are re-ranked by comparing their semantic signatures obtained from the semantic space speciﬁed by the query keyword. The proposed query-speciﬁc semantic signatures signiﬁcantly improve both the accuracy and efﬁciency of image re-ranking. The original visual features of thousands of dimensions can be projected to the semantic signatures as short as 25 dimensions. Experimental results show that 25%~40% relative improvement has been achieved on re-ranking precisions compared with the state-of-the-art methods.
X. Wang, S. Qiu, K. Liu, and X. Tang, "Web Image Re-ranking Using Query-Specific Semantic Signatures,"
accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).
PDF Project Page
X. Wang, K. Liu, and X. Tang, "Query-Specific Visual Semantic Spaces for Web Image Re-ranking,"
in Proceedings of IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2011.
IntentSearch: Capturing User Intention for One-Click Internet Image Search
Web-scale image search engines (e.g., Google image search, Bing image search) mostly rely on surrounding text features. It is difficult for them to interpret users’ search intention only by query keywords and this leads to ambiguous and noisy search results which are far from satisfactory. It is important to use visual information in order to solve the ambiguity in text-based image retrieval. We propose a novel Internet image search approach. It only requires the user to click on one query image with minimum effort and images from a pool retrieved by text-based search are reranked based on both visual and textual content.
X. Tang, K. Liu, J. Cui, F. Wen, and X. Wang, "IntentSearch:Capturing User Intention for One-Click Internet Image Search," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 34, pp. 1342-1353, 2012.