Physical Sciences and Mathematics Commons^™

Mixed-Initiative Personal Assistants, Joshua W. Buck, Saverio Perugini

Saverio Perugini

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. Our current research entails incorporating the use of natural language to make the flexibility in communicating user utterances commensurate with that in dialog completion …

Grnsight: A Web Application And Service For Visualizing Models Of Small- To Medium-Scale Gene Regulatory Networks, Kam D. Dahlquist, John David N. Dionisio, Ben G. Fitzpatrick, Nicole A. Anguiano, Anindita Varshneya, Britain J. Southwick, Mihir Samdarshi

John David N. Dionisio

GRNsight is a web application and service for visualizing models of gene regulatory networks (GRNs). A gene regulatory network (GRN) consists of genes, transcription factors, and the regulatory connections between them which govern the level of expression of mRNA and protein from genes. The original motivation came from our efforts to perform parameter estimation and forward simulation of the dynamics of a differential equations model of a small GRN with 21 nodes and 31 edges. We wanted a quick and easy way to visualize the weight parameters from the model which represent the direction and magnitude of the influence of …

What Is Answer Set Programming To Propositional Satisfiability, Yuliya Lierler

Yuliya Lierler

Heat Map Analysis Of Rna-Seq Data Using Rstudio, Ray A. Enke, Ashton Holub

Ray Enke Ph.D.

Intro To Rstudio, Ray A. Enke, Ashton Holub

Ray Enke Ph.D.

Perceptions Of Planned Versus Unplanned Malfunctions: A Human-Robot Interaction Scenario, Theresa T. Kessler, Keith R. Macarthur, Manuel Trujillo-Silva, Thomas Macgillivray, Chris Ripa, Peter A. Hancock

Keith Reid MacArthur

Further Evidence Of The Contribution Of The Ear Canal To Directional Hearing: Design Of A Compensating Filter, Andrea Martelloni, Davide Andrea Mauro Phd, Antonio Mancuso

Davide Andrea Mauro

It has been proven, and it is well documented in literature, that the directional response in HRTFs comes largely from the effect of the pinnae. However, few studies have analysed the contribution given by the remaining part of the external ear, particularly the ear canal. This work investigates the directionally dependent response of the modelled ear canal of a dummy head, assuming that the behaviour of the external ear is sufficiently linear to be approximated by an LTI system. In order to extract the ear canal's transfer function, two critical microphone placements (at the eardrum and at the beginning of …

Self-Organizing The Space Of Vocal Imitations, Davide Rocchesso, Davide Andrea Mauro Phd

Davide Andrea Mauro

The human voice is a powerful instrument for producing sound sketches. The sonic space that can be spanned with the voice is vast and complex and, therefore, it is difficult to organize and explore. In this contribution, we report on our attempts at extracting the principal components from a database of 152 short excerpts of vocal imitations. We describe each excerpt by a set of statistical audio features and by a measure of similarity of the envelope to a small number of prototype envelopes. We apply k-means clustering on a space whose dimensionality has been reduced by singular value decomposition, …

Analyzing And Organizing The Sonic Space Of Vocal Imitation, Davide Andrea Mauro Phd, D. Rocchesso

Davide Andrea Mauro

The sonic space that can be spanned with the voice is vast and complex and, therefore, it is difficult to organize and explore. In order to devise tools that facilitate sound design by vocal sketching we attempt at organizing a database of short excerpts of vocal imitations. By clustering the sound samples on a space whose dimensionality has been reduced to the two principal components, it is experimentally checked how meaningful the resulting clusters are for humans. Eventually, a representative of each cluster, chosen to be close to its centroid, may serve as a landmark in the exploration of the …

Sound And The City: Multi-Layer Representation And Navigation Of Audio Scenarios, Luca A. Ludovico, Davide Andrea Mauro Phd

Davide Andrea Mauro

IEEE 1599-2008 is an XML-based standard originally intended for the multi-layer representation of music information. Nevertheless, it is versatile enough to describe also information different from traditional scores written according to the Common Western Notation (CWN) rules. This paper will discuss the application of IEEE 1599-2008 to the audio description of paths and scenarios from the urban life or other landscapes. The standard we adopt allows the multilayer integration of textual, symbolical, structural, graphical, audio and video contents within a unique synchronized environment. Besides, for each kind of media, a number of digital objects is supported. As a consequence, thanks …

Puremx: Automatic Transcription Of Midi Live Music Performances Into Xml Format, Stefano Baldan, Luca A. Ludovico, Davide Andrea Mauro Phd

Davide Andrea Mauro

This paper addresses the problem of the real-time automatic transcription of a live music performance into a symbolic format based on XML. The source data are given by any music instrument or other device able to communicate with Pure Data by MIDI. Pure Data is a free, multi-platform, real-time programming environment for graphical, audio, and video processing. During a performance, music events are parsed and their parameters are evaluated thanks to rhythm and pitch detection algorithms. The final step is the creation of a well-formed XML document, validated against the new international standard known as IEEE 1599. This work will …

On Binaural Spatialization And The Use Of Gpgpu For Audio Processing, Davide Andrea Mauro Phd

Davide Andrea Mauro

3D recordings and audio, namely techniques that aim to create the perception of sound sources placed anywhere in 3 dimensional space, are becoming an interesting resource for composers, live performances and augmented reality. This thesis focuses on binaural spatialization techniques. We will tackle the problem from three different perspectives. The first one is related to the implementation of an engine for audio convolution, this is a real implementation problem where we will confront with a number of already available systems trying to achieve better results in terms of performances. General Purpose computing on Graphic Processing Units (GPGPU) is a promising …

Analyzing And Organizing The Sonic Space Of Vocal Imitation, Davide Andrea Mauro Phd, D. Rocchesso

Davide Andrea Mauro

The sonic space that can be spanned with the voice is vast and complex and, therefore, it is difficult to organize and explore. In order to devise tools that facilitate sound design by vocal sketching we attempt at organizing a database of short excerpts of vocal imitations. By clustering the sound samples on a space whose dimensionality has been reduced to the two principal components, it is experimentally checked how meaningful the resulting clusters are for humans. Eventually, a representative of each cluster, chosen to be close to its centroid, may serve as a landmark in the exploration of the …

Puremx: Automatic Transcription Of Midi Live Music Performances Into Xml Format, Stefano Baldan, Luca A. Ludovico, Davide Andrea Mauro Phd

Davide Andrea Mauro

This paper addresses the problem of the real-time automatic transcription of a live music performance into a symbolic format based on XML.

The source data are given by any music instrument or other device able to communicate with Pure Data by MIDI. Pure Data is a free, multi-platform, real-time programming environment for graphical, audio, and video processing. During a performance, music events are parsed and their parameters are evaluated thanks to rhythm and pitch detection algorithms. The final step is the creation of a well-formed XML document, validated against the new international standard known as IEEE 1599.

This work will …

Further Evidence Of The Contribution Of The Ear Canal To Directional Hearing: Design Of A Compensating Filter, Andrea Martelloni, Davide Andrea Mauro Phd, Antonio Mancuso

Davide Andrea Mauro

It has been proven, and it is well documented in literature, that the directional response in HRTFs comes largely from the effect of the pinnae. However, few studies have analysed the contribution given by the remaining part of the external ear, particularly the ear canal. This work investigates the directionally dependent response of the modelled ear canal of a dummy head, assuming that the behaviour of the external ear is sufficiently linear to be approximated by an LTI system. In order to extract the ear canal's transfer function, two critical microphone placements (at the eardrum and at the beginning of …

Head In Space : A Head-Tracking Based Binaural Spatialization System, Luca A. Ludovico, Davide Andrea Mauro Phd, Dario Pizzamiglio

Davide Andrea Mauro

This paper discusses a system capable of detecting the position of the listener through a head-tracking system and rendering a 3D audio environment by binaural spatialization. Head tracking is performed through face recognition algorithms which use a standard webcam, and the result is presented over headphones, like in other typical binaural applications. With this system users can choose an audio file to play, provide a virtual position for the source in an euclidean space, and then listen to the sound as if it is coming from that position. If they move their head, the signal provided by the system changes …

On Binaural Spatialization And The Use Of Gpgpu For Audio Processing, Davide Andrea Mauro Phd

Davide Andrea Mauro

3D recordings and audio, namely techniques that aim to create the perception of sound sources placed anywhere in 3 dimensional space, are becoming an interesting resource for composers, live performances and augmented reality. This thesis focuses on binaural spatialization techniques.

We will tackle the problem from three different perspectives. The first one is related to the implementation of an engine for audio convolution, this is a real implementation problem where we will confront with a number of already available systems trying to achieve better results in terms of performances. General Purpose computing on Graphic Processing Units (GPGPU) is a promising …

Growing The Practice Of Vocal Sketching, S. Delle Monache, D. Rocchesso, S. Baldan, Davide Andrea Mauro Phd

Davide Andrea Mauro

Sketch-thinking in the design domain is a complex representational activity, emerging from the reflective conversation with the sketch. A recent line of research on computational support for sound design has been focusing on the exploitation of voice, and especially vocal imitations, as effective representation strategy for the early stage of the design process. A set of introductory exercises on vocal sketching, to probe the communication effectiveness of vocal imitations for design purposes, are presented and discussed, in the scope of the research-through-design workshop activities of the EU project SkAT-VG.

Marim: Mobile Augmented Reality For Interactive Manuals, Tam Nguyen, Dorothy Tan, Bilal Mirza, Jose Sepulveda

Tam Nguyen

In this work, we present a practical system which uses mobile devices for interactive manuals. In particular, there are two modes provided in the system, namely, expert/trainer and trainee modes. Given the expert/trainer editor, experts design the step-by-step interactive manuals. For each step, the experts capture the images by using phones/tablets and provide visual instructions such as interest regions, text, and action animations. In the trainee mode, the system utilizes the existing object detection and tracking algorithms to identify the step scene and retrieve the respective instruction to be displayed on the mobile device. The trainee then follows the displayed …

Static Saliency Vs. Dynamic Saliency: A Comparative Study, Tam Nguyen, Mengdi Xu, Guangyu Gao, Mohan Kankanhalli, Qi Tian, Shuicheng Yan

Tam Nguyen

Recently visual saliency has attracted wide attention of researchers in the computer vision and multimedia field. However, most of the visual saliency-related research was conducted on still images for studying static saliency. In this paper, we give a comprehensive comparative study for the first time of dynamic saliency (video shots) and static saliency (key frames of the corresponding video shots), and two key observations are obtained: 1) video saliency is often different from, yet quite related with, image saliency, and 2) camera motions, such as tilting, panning or zooming, affect dynamic saliency significantly.

Motivated by these observations, we propose a …

Seeing Human Weight From A Single Rgb-D Image, Tam Nguyen, Jiashi Feng, Shuicheng Yan

Tam Nguyen

Human weight estimation is useful in a variety of potential applications, e.g., targeted advertisement, entertainment scenarios and forensic science. However, estimating weight only from color cues is particularly challenging since these cues are quite sensitive to lighting and imaging conditions. In this article, we propose a novel weight estimator based on a single RGB-D image, which utilizes the visual color cues and depth information. Our main contributions are three-fold.

First, we construct the W8-RGBD dataset including RGB-D images of different people with ground truth weight.

Second, the novel sideview shape feature and the feature fusion model are proposed to facilitate …

Salient Object Detection Via Objectness Proposals, Tam Nguyen

Tam Nguyen

Salient object detection has gradually become a popular topic in robotics and computer vision research. This paper presents a real-time system that detects salient objects by integrating objectness, foreground, and compactness measures. Our algorithm consists of four basic steps. First, our method generates the objectness map via object proposals. Based on the objectness map, we estimate the background margin and compute the corresponding foreground map which prefers the foreground objects. From the objectness map and the foreground map, the compactness map is formed to favor the compact objects. We then integrate those cues to form a pixel-accurate saliency map which …

Salient Object Detection Via Augmented Hypotheses, Tam Nguyen, Jose Sepulveda

Tam Nguyen

In this paper, we propose using augmented hypotheses which consider objectness, foreground, and compactness for salient object detection. Our algorithm consists of four basic steps. First, our method generates the objectness map via objectness hypotheses. Based on the objectness map, we estimate the foreground margin and compute the corresponding foreground map which prefers the foreground objects. From the objectness map and the foreground map, the compactness map is formed to favor the compact objects. We then derive a saliency measure that produces a pixel-accurate saliency map which uniformly covers the objects of interest and consistently separates foreground and background.

We …

Towards Decrypting Attractiveness Via Multi-Modality Cue, Tam Nguyen, Si Liu, Bingbing Ni, Jun Tan, Yong Rui, Shuicheng Yan

Tam Nguyen

Decrypting the secret of beauty or attractiveness has been the pursuit of artists and philosophers for centuries. To date, the computational model for attractiveness estimation has been actively explored in the computer vision and multimedia community, yet with the focus mainly on facial features. In this article, we conduct a comprehensive study on female attractiveness conveyed by single/multiple modalities of cues, that is, face, dressing and/or voice; the aim is to discover how different modalities individually and collectively affect the human sense of beauty. To extensively investigate the problem, we collect the Multi-Modality Beauty (M2B) dataset, which is annotated with …

Adaptive Nonparametric Image Parsing, Tam Nguyen, Canyi Lu, Jose Sepulveda, Shuicheng Yan

Tam Nguyen

In this paper, we present an adaptive nonparametric solution to the image parsing task, namely, annotating each image pixel with its corresponding category label. For a given test image, first, a locality-aware retrieval set is extracted from the training data based on superpixel matching similarities, which are augmented with feature extraction for better differentiation of local superpixels. Then, the category of each superpixel is initialized by the majority vote of the k -nearest-neighbor superpixels in the retrieval set. Instead of fixing k as in traditional nonparametric approaches, here, we propose a novel adaptive nonparametric approach that determines the sample-specific k …

Gecka3d: A 3d Game Engine For Commonsense Knowledge Acquisition, Erik Cambria, Tam Nguyen, Brian Cheng, Kenneth Kwok, Jose Sepulveda

Tam Nguyen

Commonsense knowledge representation and reasoning is key for tasks such as artificial intelligence and natural language understanding. Since commonsense consists of information that humans take for granted, gathering it is an extremely difficult task. In this paper, we introduce a novel 3D game engine for commonsense knowledge acquisition (GECKA3D) which aims to collect commonsense from game designers through the development of serious games. GECKA3D integrates the potential of serious games and games with a purpose. This provides a platform for the acquisition of reusable and multi-purpose knowledge and also enables the development of games that can provide entertainment value and …

Hi, Magic Closet, Tell Me What To Wear!, Si Liu, Tam Nguyen, Jiashi Feng, Meng Wang, Shuicheng Yan

Tam Nguyen

In this demo, we present a practical system, "magic closet," for automatic occasion-oriented clothing pairing. Given a user-input occasion, e.g., wedding or shopping, the magic closet intelligently and automatically pairs the user-specified reference clothing (upper body or lower body) with the most suitable one from online shops. Two key criteria are explicitly considered for the magic closet system. One criterion is to dress properly, e.g., compared to suit pants, it is more decent to wear a cocktail dress for a banquet occasion. The other criterion is to dress aesthetically, e.g., a red T-shirt matches better with white pants than with …

Algorithms For An Automatic Transcription Of Live Music Performances Into Symbolic Format, Stefano Baldan, Luca A. Ludovico, Davide Andrea Mauro

Davide Andrea Mauro

This paper addresses the problem of the real-time automatic transcription of a live music performance into a symbolic format. The source data are given by any music instrument or other device able to communicate through a performance protocol. During a performance, music events are parsed and their parameters are evaluated thanks to rhythm and pitch detection algorithms. The final step is the creation of a well-formed XML document, validated against the new international standard known as IEEE 1599. This work will shortly describe both the software environment and the XML format, but the main analysis will involve the real-time recognition …

"Musica Sull'acqua": A Motion Tracking Based Sonification Of An Aquarium In Real Time, Stefano Baldan, Luca A. Ludovico, Davide Andrea Mauro

Davide Andrea Mauro

This paper presents a temporary multimedia installation set up at the Civic Aquarium of Milan. Thanks to four web cameras located in front of the tropical fishpond, fish are tracked and their movements are used to control a number of music-related parameters in real time. In order to process multiple video streams, the open-source programming language Processing has been employed. Then, the sonification is implemented by a Pure Data patch. The communication among the parts of the system has been realized through Open Sound Control (OSC) messages. This paper describes the key concepts, the musical idea, the design phase and …

To "Sketch-A-Scratch", A. Del Piccolo, S. Delle Monache, D. Rocchesso, S. Papetti, Davide Andrea Mauro

Davide Andrea Mauro

A surface can be harsh and raspy, or smooth and silky, and everything in between. We are used to sense these features with our fingertips as well as with our eyes and ears: the exploration of a surface is a multisensory experience. Tools, too, are often employed in the interaction with surfaces, since they augment our manipulation capabilities. “Sketch-a-Scratch” is a tool for the multisensory exploration and sketching of surface textures. The user’s actions drive a physical sound model of real materials’ response to interactions such as scraping, rubbing or rolling. Moreover, different input signals can be converted into 2D …