Distributed Image Search in Camera Sensor Networks
years have seen massive proliferation of networked camera sensor
applications, such as cellphone cameras, surveillance cameras,
low-power imagers for environmental sensing, lifelog devices (e.g.
sensecam), traffic cameras, digicams, etc. These camera sensor network
applications generate tremendous volume of data whiletransmitting
images from battery-powered devices is very expensive. Meanwhile,
privacy concern makes a centralized infrastructure less attractive.
Project Goal: Design a general purpose distributed image storage and search engine across heterogeneous camera sensor sources.
Our study on distributed image search covers the following three aspects: 1) Compact image
representation, 2) Energy efficient search on
flash storage centric sensor nodes, and 3) Distributed image search in sensor networks.
- Efficient image representations to enable high-accuracy, fast, image search.
- Efficient local storage and search across diverse storage media (disk, flash), and platforms with diverse resource constraints.
- Distributed search across diverse image datasets while minimizing communication and processing.
- Privacy-aware search to provide users full control over images that they seek to share.
use visual word/visterm as compact feature for image representation.
Two steps are involved in retrieving visterms. 1) SIFT features are
extracted from raw QVGA images; 2) Hierarchical clustering is used
to convert SIFT features to visterms.|
vocabulary tree, the leaf clusters are used to represent the SIFT
features of database images. Therefore, a 128 dimensional SIFT feature
is now turned into a cluster id (usually 4 Bytes). We can this cluster
id visual words, or visterm. In distributed image search, we use
visterm as our distinguishing feature to represent images.
Energy efficient search on
flash storage centric sensor nodesWe have following two key technique to conduct energy efficient search on sensor nodes.
Buffered Vocabulary Tree
vocabulary tree is the core data structure to map SIFT features to
visterms. Since vocabulary tree is a largedata structure (usually
several MB), it is necessary to be maintained on flash. We can only
load a subset of entire vocabulary tree into memory when converting
SIFT to visterm. Thus, we developped the Buffered Vocabulary tree to
optimize this converting process. The key idea of Buffered Vocabulary
tree is to partition large Vocabulary tree into sub-trees, and buffer
SIFT features for batch processing to reduce times of loading the
same subtrees. |
Inverted File based Scoring and matching
file is used to map a visterm to the set of images in the local
database that contain the visterm. As the number of captured images
increases, the size of inverted file grows to exceed the memory limit.
Therefore, we introduce an additive log-like flash storage to
store inverted file. The longest document list in the inverted
file are stored in this log-like storage to keep the flash writing
time as less as possible. |
Distributed image search in sensor networks
matching is to further refine the local search results for global
optimal. The proxy gathers the top-k locally ranked results and merge
them into the global top-k results. The sensors only need to send back
the correspondent images that are in the global ranking list. |
consists of 6 iMote2 nodes, each of which is equipped with an OmniVision camera
and an 1 GB SD extension card. The OmniVision provides color VGA
(640x480) image aquisition. Our iMote2 nodes forms a multihop metwork
in our evaluation.
PublicationsTingxin Yan, Deepak Ganesan and R. Manmatha, Distributed
Image Search in Camera Sensor Networks. In Proceedings of the
6th ACM Conference on Embedded Networked Sensor Systems (SenSys 2008), Raleigh,
NC, Nov 2008. [pdf]
FundingThis work was supported by NSF grants: CNS-0626873, CNS-0546177 and CNS-052072.
©2008 Sensors Lab, Computer Science Department,
University of Massachusetts Amherst