home | people | research | toolbelt | publications | calendar | photos | downloads | weblinks | news

Last Update: 2013.12.03

VIVA: Visual Media Reasoning (VMR)

Project Info


Building detection, Content based Image retrieval, Segmentation, Feature voting


Develop image analysis techniques to to extract identities and classification of objects.

Funded by:

Information Innovative Office - Defense Advanced Research Projects Agency (DARPA)

VIVA Team Members:

Scott Acton, Sedat Ozer, Rituparna Sarkar


Robust content based image retrieval system based on feature voting algorithms.


The objective of this project is to show that a meta-algorithm consisting of segmentation and classification forms a sound basis for selecting which segments to analyze and the choice of algorithms to apply for the analysis and to show that content-based image retrieval (CBIR) is a necessary ingredient in classification. In the unstructured environment that the VMR program envisions, there remains significant risk that query images will not meet the requirements of any available directed recognition algorithm, producing a high rate of false negatives, or a high volume of unusable, false-positive results that cannot enhance the experience of the analyst. A CBIR approach is instead based on identifying salient properties of the overall image and finding images with the most similar properties. This is what allows CBIR to be helpful for classification.

Fig.1 Segmentation Example

Current state-of-the-art classification and recognition algorithms do not have the flexibility to robustly classify and recognize all potential objects across all imaging scenarios. In short, we hypothesize that CBIR approaches are valuable because they much less likely to suffer from false negatives, despite a higher risk of false positives. (We therefore hypothesize as well that, when other algorithms fail, users will be willing to tolerate reasonable number false positives as long as the system also produces sufficient true positives). The CBIR approach has the powerful advantage of employing multiple algorithms in order to maximize likelihood of identifying different types of features, and will also benefit from a meta-algorithm to direct the set of analysis steps. This leads to a self-nomination paradigm in which, for any analysis task (such as segmenting an image, or classifying or recognizing an object), a large pool of algorithms self-score their suitability. This allows the controller or solution planner for that task to identify the best algorithms without the need to have specific knowledge of each algorithm’s capabilities (which in turn would require modifying the control algorithm each time a new algorithm is added!). Identification of individual buildings forms another important part of the project. Specifically, we hypothesize that a combination of scale-invariant feature detection (i.e., using SIFT and new morphological alternatives proposed here) combined with a novel global-local feature detection algorithm can accurately identify buildings even when the candidates are very similar.



  • Segmentation [.zip]

    Matlab implementation of some segmentation algorithms are included. Example included.

©2013 University of Virginia.  Privacy Statement
Virginia Image and Video Analysis · School of Engineering and Applied Science · University of Virginia
P.O. Box 400743 · Charlottesville, VA · 22904 · E-mail viva.uva@gmail.com