Physical Sciences and Mathematics Commons^™

Content-Based Image Retrieval Using Hierarchical Decomposition Of Feature Descriptors, Eisa Adil

Electronic Theses and Dissertations

Due to modern technological advancements, the pervasiveness and complexity of images have remarkably increased. Searching databases for similar visual content, i.e., Content-Based Image Retrieval (CBIR), remains an open research problem. In this thesis, we propose a novel CBIR approach, in which each symbolic image has a quadtree representation consisting of SIFT-based orientational keypoints. Every quadrant node in the tree represents the dominant orientation of a region in the image. The quadtree image representation is used for bitwise signature indexing and image similarity measurement. Also, we convert each quadtree image representation to a trainable feature vector for use in the K-Nearest …

Improved Scoring Models For Semantic Image Retrieval Using Scene Graphs, Erik Timothy Conser

Dissertations and Theses

Image retrieval via a structured query is explored in Johnson, et al. [7]. The query is structured as a scene graph and a graphical model is generated from the scene graph's object, attribute, and relationship structure. Inference is performed on the graphical model with candidate images and the energy results are used to rank the best matches. In [7], scene graph objects that are not in the set of recognized objects are not represented in the graphical model. This work proposes and tests two approaches for modeling the unrecognized objects in order to leverage the attribute and relationship models to …

Unsupervised Visual Hashing With Semantic Assistant For Content-Based Image Retrieval, Lei Zhu, Jialie Shen, Liang Xie, Zhiyong Cheng

Research Collection School Of Computing and Information Systems

As an emerging technology to support scalable content-based image retrieval (CBIR), hashing has recently received great attention and became a very active research domain. In this study, we propose a novel unsupervised visual hashing approach called semantic-assisted visual hashing (SAVH). Distinguished from semi-supervised and supervised visual hashing, its core idea is to effectively extract the rich semantics latently embedded in auxiliary texts of images to boost the effectiveness of visual hashing without any explicit semantic labels. To achieve the target, a unified unsupervised framework is developed to learn hash codes by simultaneously preserving visual similarities of images, integrating the semantic …

Association-Based Image Retrieval, Arun D. Kulkarni, H. Gunturu, S. Dalta

Arun Kulkarni

With advances in the computer technology and the World Wide Web there has been an explosion in the amount and complexity of multimedia data that are generated, stored, transmitted, analyzed, and accessed. In order to extract useful information from this huge amount of data, many content-based image retrieval (CBIR) systems have been developed in the last decade. A typical CBIR system captures image features that represent image properties such as color, texture, or shape of objects in the query image and try to retrieve images from the database with similar features. Recent advances in CBIR systems include relevance feedback based …

Online Multi-Modal Distance Metric Learning With Application To Image Retrieval, Pengcheng Wu, Steven C. H. Hoi, Peilin Zhao, Chunyan Miao, Zhi-Yong Liu

Research Collection School Of Computing and Information Systems

Distance metric learning (DML) is an important technique to improve similarity search in content-based image retrieval. Despite being studied extensively, most existing DML approaches typically adopt a single-modal learning framework that learns the distance metric on either a single feature type or a combined feature space where multiple types of features are simply concatenated. Such single-modal DML methods suffer from some critical limitations: (i) some type of features may significantly dominate the others in the DML task due to diverse feature representations; and (ii) learning a distance metric on the combined high-dimensional feature space can be extremely time-consuming using the …

Content-Based Texture Image Retrieval By Histogram Of Curvelets, Erkan Uslu, Songül Albayrak

Turkish Journal of Electrical Engineering and Computer Sciences

No abstract provided.

Deep Learning For Content-Based Image Retrieval: A Comprehensive Study, Ji Wan, Dayong Wang, Steven C. H. Hoi, Pengcheng Wu, Jianke Zhu, Yongdong Zhang, Jintao Li

Research Collection School Of Computing and Information Systems

Learning effective feature representations and similarity measures are crucial to the retrieval performance of a content-based image retrieval (CBIR) system. Despite extensive research efforts for decades, it remains one of the most challenging open problems that considerably hinders the successes of real-world CBIR systems. The key challenge has been attributed to the well-known "semantic gap" issue that exists between low-level image pixels captured by machines and high-level semantic concepts perceived by human. Among various techniques, machine learning has been actively investigated as a possible direction to bridge the semantic gap in the long term. Inspired by recent successes of deep …

Online Multi-Modal Distance Metric Learning With Application To Image Retrieval, Pengcheng Wu, Steven C. H. Hoi, Peilin Zhao, Chunyan Miao, Zhi-Yong Liu

Research Collection School Of Computing and Information Systems

See https://ink.library.smu.edu.sg/sis_research/2924/. Distance metric learning (DML) is an important technique to improve similarity search in content-based image retrieval. Despite being studied extensively, most existing DML approaches typically adopt a single-modal learning framework that learns the distance metric on either a single feature type or a combined feature space where multiple types of features are simply concatenated. Such single-modal DML methods suffer from some critical limitations: (i) some type of features may significantly dominate the others in the DML task due to diverse feature representations; and (ii) learning a distance metric on the combined high-dimensional feature space can be extremely …

Retrieval-Based Face Annotation By Weak Label Regularized Local Coordinate Coding, Dayong Wang, Steven C. H. Hoi, Ying He, Jianke Zhu, Mei Tao, Jiebo Luo

Research Collection School Of Computing and Information Systems

Auto face annotation, which aims to detect human faces from a facial image and assign them proper human names, is a fundamental research problem and beneficial to many real-world applications. In this work, we address this problem by investigating a retrieval-based annotation scheme of mining massive web facial images that are freely available over the Internet. In particular, given a facial image, we first retrieve the top n similar instances from a large-scale web facial image database using content-based image retrieval techniques, and then use their labels for auto annotation. Such a scheme has two major challenges: 1) how to …

Mining Weakly Labeled Web Facial Images For Search-Based Face Annotation, Dayong Wang, Steven C. H. Hoi, Ying He, Jianke Zhu

Research Collection School Of Computing and Information Systems

This paper investigates a framework of search-based face annotation (SBFA) by mining weakly labeled facial images that are freely available on the World Wide Web (WWW). One challenging problem for search-based face annotation scheme is how to effectively perform annotation by exploiting the list of most similar facial images and their weak labels that are often noisy and incomplete. To tackle this problem, we propose an effective unsupervised label refinement (ULR) approach for refining the labels of web facial images using machine learning techniques. We formulate the learning problem as a convex optimization and develop effective optimization algorithms to solve …

Online Multiple Kernel Similarity Learning For Visual Search, Hao Xia, Chu Hong Hoi, Rong Jin, Peilin Zhao

Research Collection School Of Computing and Information Systems

Recent years have witnessed a number of studies on distance metric learning to improve visual similarity search in content-based image retrieval (CBIR). Despite their successes, most existing methods on distance metric learning are limited in two aspects. First, they usually assume the target proximity function follows the family of Mahalanobis distances, which limits their capacity of measuring similarity of complex patterns in real applications. Second, they often cannot effectively handle the similarity measure of multimodal data that may originate from multiple resources. To overcome these limitations, this paper investigates an online kernel similarity learning framework for learning kernel-based proximity functions …

Effective Graph-Based Content--Based Image Retrieval Systems For Large-Scale And Small-Scale Image Databases, Ran Chang

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Digital imaging was a great invention in the last century. Since digital cameras became popular in the public, a large amount of digital images emerged in the late of the twentieth century. How to manage the huge amount of images and find desired images among them became an urgent issue during the same period.

Techniques of retrieving a desired image are generally categorized into two basic classes. One relies on text-based key words to retrieve desired images in the image
database. The other one relies on image-based queries to retrieve desired images in the image database. The second technique is …

A Framework For Medical Image Retrieval Using Merging-Based Classification With Dependency Probability-Based Relevance Feedback, Hossein Pourghassem, Sabalan Daneshvar

Turkish Journal of Electrical Engineering and Computer Sciences

Content-based image retrieval (CBIR) systems are used to retrieve relevant images from large-scale databases. In this paper, a framework for the image retrieval of a large-scale database of medical X-ray images is presented. This framework is designed based on query image classification into several prespecified homogeneous classes. Using a merging scheme and an iterative classification, the homogeneous classes are formed from overlapping classes in the database. For this purpose, the shape and texture features, selected using the forward selection algorithm, are optimized by a novel genetic algorithm-based feature reduction and optimization algorithm in the feature space. In this algorithm, using …

Sire: A Social Image Retrieval Engine, Steven C. H. Hoi, Pengcheng Wu

Research Collection School Of Computing and Information Systems

With the explosive growth of social media applications on the internet, billions of social images have been made available in many social media web sites nowadays. This has presented an open challenge of web-scale social image search. Unlike existing commercial web search engines that often adopt text based retrieval, in this demo, we present a novel web-based multimodal paradigm for large-scale social image retrieval, termed "Social Image Retrieval Engine" (SIRE), which effectively exploits both textual and visual contents to narrow down the semantic gap between high-level concepts and low-level visual features. A relevance feedback mechanism is also equipped to learn …

Distance Metric Learning From Uncertain Side Information For Automated Photo Tagging, Lei Wu, Steven C. H. Hoi, Rong Jin, Jianke Zhu, Nenghai Yu

Research Collection School Of Computing and Information Systems

Automated photo tagging is an important technique for many intelligent multimedia information systems, for example, smart photo management system and intelligent digital media library. To attack the challenge, several machine learning techniques have been developed and applied for automated photo tagging. For example, supervised learning techniques have been applied to automated photo tagging by training statistical classifiers from a collection of manually labeled examples. Although the existing approaches work well for small testbeds with relatively small number of annotation words, due to the long-standing challenge of object recognition, they often perform poorly in large-scale problems. Another limitation of the existing …

Semi-Supervised Distance Metric Learning For Collaborative Image Retrieval And Clustering, Steven C. H. Hoi, Wei Liu, Shih-Fu Chang

Research Collection School Of Computing and Information Systems

Learning a good distance metric plays a vital role in many multimedia retrieval and data mining tasks. For example, a typical content-based image retrieval (CBIR) system often relies on an effective distance metric to measure similarity between any two images. Conventional CBIR systems simply adopting Euclidean distance metric often fail to return satisfactory results mainly due to the well-known semantic gap challenge. In this article, we present a novel framework of Semi-Supervised Distance Metric Learning for learning effective distance metrics by exploring the historical relevance feedback log data of a CBIR system and utilizing unlabeled data when log data are …

Semisupervised Svm Batch Mode Active Learning With Applications To Image Retrieval, Steven C. H. Hoi, Rong Jin, Jianke Zhu, Michael R. Lyu

Research Collection School Of Computing and Information Systems

Active learning has been shown as a key technique for improving content-based image retrieval (CBIR) performance. Among various methods, support vector machine (SVM) active learning is popular for its application to relevance feedback in CBIR. However, the regular SVM active learning has two main drawbacks when used for relevance feedback. First, SVM often suffers from learning with a small number of labeled examples, which is the case in relevance feedback. Second, SVM active learning usually does not take into account the redundancy among examples, and therefore could select multiple examples in relevance feedback that are similar (or even identical) to …

Efficient Techniques For Relevance Feedback Processing In Content-Based Image Retrieval, Danzhou Liu

Electronic Theses and Dissertations

In content-based image retrieval (CBIR) systems, there are two general types of search: target search and category search. Unlike queries in traditional database systems, users in most cases cannot specify an ideal query to retrieve the desired results for either target search or category search in multimedia database systems, and have to rely on iterative feedback to refine their query. Efficient evaluation of such iterative queries can be a challenge, especially when the multimedia database contains a large number of entries, and the search needs many iterations, and when the underlying distance measure is computationally expensive. The overall processing costs, …

Semi-Supervised Svm Batch Mode Active Learning For Image Retrieval, Steven Hoi, Rong Jin, Jianke Zhu, Michael R. Lyu

Research Collection School Of Computing and Information Systems

Active learning has been shown as a key technique for improving content-based image retrieval (CBIR) performance. Among various methods, support vector machine (SVM) active learning is popular for its application to relevance feedback in CBIR. However, the regular SVM active learning has two main drawbacks when used for relevance feedback. First, SVM often suffers from learning with a small number of labeled examples, which is the case in relevance feedback. Second, SVM active learning usually does not take into account the redundancy among examples, and therefore could select multiple examples in relevance feedback that are similar (or even identical) to …

Association-Based Image Retrieval, Arun D. Kulkarni, H. Gunturu, S. Dalta

Computer Science Faculty Publications and Presentations

With advances in the computer technology and the World Wide Web there has been an explosion in the amount and complexity of multimedia data that are generated, stored, transmitted, analyzed, and accessed. In order to extract useful information from this huge amount of data, many content-based image retrieval (CBIR) systems have been developed in the last decade. A typical CBIR system captures image features that represent image properties such as color, texture, or shape of objects in the query image and try to retrieve images from the database with similar features. Recent advances in CBIR systems include relevance feedback based …

Collaborative Image Retrieval Via Regularized Metric Learning, Luo Si, Rong Jin, Steven C. H. Hoi, Michael R. Lyu

Research Collection School Of Computing and Information Systems

In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval.” In this paper, we present a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring …

A Unified Log-Based Relevance Feedback Scheme For Image Retrieval, Steven Hoi, Michael R. Lyu, Rong Jin

Research Collection School Of Computing and Information Systems

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback …

Viscors: A Visual-Content Recommender For The Mobile Web, Chan Young Kim, Jae Kyu Lee, Yoon Ho Cho, Deok Hwan Kim

Research Collection School Of Computing and Information Systems

Current search methods for mobile-Web content can be frustrating to use. To shorten searches for cell phone wallpaper images, VISCORS combines collaborative filtering with content-based image retrieval. An increasing selection of content is becoming available in the mobile-Web environment, where users navigate the Web using wireless devices such as cell phones and PDAs. The fast growth and excellent prospects of the mobile-Web content market have attracted many content providers.

Automatic Segmentation And Indexing Of Specialized Databases, Madirakshi Das, R. Manmatha

R. Manmatha

The aim of this work is to index images based on color, in domain specific databases using colors computed from the object of interest only, instead of using the whole image. The main problem in this task is the segmentation of the region of interest from the background. Viewing segmentation as a figure/ground segregation problem leads to a new approach--successful elimination of the background leaves the figure or object of interest. The background elements are eliminated using general observations true for any photograph where there is a single, prominent object of interest. First, we form a hypothesis about possible background …

Video Indexing And Retrieval Techniques Using Novel Approaches To Video Segmentation, Characterization, And Similarity Matching, Waleed Ezzat Farag

Computer Science Theses & Dissertations

Multimedia applications are rapidly spread at an ever-increasing rate introducing a number of challenging problems at the hands of the research community, The most significant and influential problem, among them, is the effective access to stored data. In spite of the popularity of keyword-based search technique in alphanumeric databases, it is inadequate for use with multimedia data due to their unstructured nature. On the other hand, a number of content-based access techniques have been developed in the context of image indexing and retrieval; meanwhile video retrieval systems start to gain wide attention, This work proposes a number of techniques constituting …