Manipulating Bilevel Feature Space
for Category-Aware Image Exploration

Introduction

Recent development of web infrastructure has resulted in the rapid increase in the number of images accessible to common users of the Internet. Nonetheless, seeking a more effective means of retrieving specific images from the image database systems still poses a variety of interesting problems from a technical point of view. One of the common solutions is to encode images in a feature vector format and then plot the respective images in the feature space for estimating the similarity between them. On the other hand, we usually have our own categorization of the images and try to rearrange the images in the feature space to best meet our expectations. Nonetheless, editing local configuration of the image feature space is not still sufficient for image exploration and thus new high-level operations for organizing the overall categorization are required as shown in Figure 1. This paper presents an approach to manipulating both the local and global arrangements of images by projecting the overall feature space onto the 2D screen space as shown in Figure 2.

Figure 1. Gap between application categoris and user categories	Figure 2. The manipulation scheme

Method

This paper presents an approach to manipulating both the local and global arrangements of images by projecting the overall feature space onto the 2D screen space. For this purpose, we incorporated bilevel feature space representation so that we can associate the global categories and local features of the images with the upper- and lower-level graph structures, respectively. Here, the upper-level graph characterizes the relationships among the underlying image categories extracted from the bag-of-features model, while the lower-level graph shows local closeness between a pair of images within the same category.

Figure 3. Method overview

Feature space manipulation is accomplished by updating the distance among vi-sual word nodes and image nodes in the two-layered graph we constructed. In practice, with the local operations we basically update the lengths of edges incident to image nodes, so that we can manually control the correlation among a specific set of images to make them either closer to or further from each other as shown in (a). On the other hand, the global operations enable us to adjust the lengths of edges between visual word nodes and thus modify the image categorization by increasing/decreasing the correlation between particular pairs of categories as shown in (b).

(a) Local graph manipulation	(b) Global graph manipulation

Results

Results images show how we can manipulate the relative positions of images using the proposed approach. Here, global manipulation allows us to edit the closeness between image categories on the screen space, while local manipulation basically controls the distances between images within each category. In our system, different background colors are assigned to the images by referring to their primary categories while those in the same category are tied with edges of the same color. This color-based rendering scheme gives us a perceptually plausible guide to identify the images of the same category, and further helps us perform category-aware manipulation of the entire feature space as well as local manipulation of image arrangement.

(a)　Before	(b)　After

(a)　Before	(b)　After

(a)　Before	(b)　After

(a)　Before	(b)　After

Paper & Video