Physical Sciences and Mathematics Commons^™

The Model 2.0 And Friends: An Interim Report, Garrison W. Cottrell, Martha Gahl, Shubham Kulkarni, Shashank Venkatramani, Yash Shah, Keyu Long, Xuzhe Zhi, Shivaank Agarwal, Cody Li, Jingyuan He, Thomas Fischer

MODVIS Workshop

Last year, I reported on preliminary results of an anatomically-inspired deep learning model of the visual system and its role in explaining the face inversion effect. This year, I will report on new results and some variations on network architectures that we have explored, mainly as a way to generate discussion and get feedback. This is by no means a polished, final presentation!

We look forward to the group’s suggestions for these projects.

Automated Delineation Of Visual Area Boundaries And Eccentricities By A Cnn Using Functional, Anatomical, And Diffusion-Weighted Mri Data, Noah C. Benson, Bogeng Song, Toshikazu Miyata, Hiromasa Takemura, Jonathan Winawer

MODVIS Workshop

Delineating visual field maps and iso-eccentricities from fMRI data is an important but time-consuming task for many neuroimaging studies on the human visual cortex because the traditional methods of doing so using retinotopic mapping experiments require substantial expertise as well as scanner, computer, and human time. Automated methods based on gray-matter anatomy or a combination of anatomy and functional mapping can reduce these requirements but are less accurate than experts. Convolutional Neural Networks (CNNs) are powerful tools for automated medical image segmentation. We hypothesize that CNNs can define visual area boundaries with high accuracy. We trained U-Net CNNs with ResNet18 …

How Object Segmentation And Perceptual Grouping Emerge In Noisy Variational Autoencoders, Ben Lonnqvist, Zhengqing Wu, Michael H. Herzog

MODVIS Workshop

Many animals and humans can recognize and segment objects from their backgrounds. Whether object segmentation is necessary for object recognition has long been a topic of debate. Deep neural networks (DNNs) excel at object recognition, but not at segmentation tasks - this has led to the belief that object recognition and segmentation are separate mechanisms in visual processing. Here, however, we show evidence that in variational autoencoders (VAEs), segmentation and faithful representation of data can be interlinked. VAEs are encoder-decoder models that learn to represent independent generative factors of the data as a distribution in a very small bottleneck layer; …

A Dynamical Model Of Binding In Visual Cortex During Incremental Grouping And Search, Daniel Schmid, Daniel A. Braun, Heiko Neumann

MODVIS Workshop

Binding of visual information is crucial for several perceptual tasks. To incrementally group an object, elements in a space-feature neighborhood need to be bound together starting from an attended location (Roelfsema, TICS, 2005). To perform visual search, candidate locations and cued features must be evaluated conjunctively to retrieve a target (Treisman&Gormican, Psychol Rev, 1988). Despite different requirements on binding, both tasks are solved by the same neural substrate. In a model of perceptual decision-making, we give a mechanistic explanation for how this can be achieved. The architecture consists of a visual cortex module and a higher-order thalamic module. While the …

Smiler: Consistent And Usable Saliency Model Implementations, Toni Kunic, Calden Wloka, John K. Tsotsos

MODVIS Workshop

The Saliency Model Implementation Library for Experimental Research (SMILER) is a new software package which provides an open, standardized, and extensible framework for maintaining and executing computational saliency models. This work drastically reduces the human effort required to apply saliency algorithms to new tasks and datasets, while also ensuring consistency and procedural correctness for results and conclusions produced by different parties. At its launch SMILER already includes twenty three saliency models (fourteen models based in MATLAB and nine supported through containerization), and the open design of SMILER encourages this number to grow with future contributions from the community. The project …

Is The Selective Tuning Model Of Visual Attention Still Relevant?, John K. Tsotsos

MODVIS Workshop

No abstract provided.

Large-Scale Discovery Of Visual Features For Object Recognition, Drew Linsley, Sven Eberhardt, Dan Shiebler, Thomas Serre

MODVIS Workshop

A central goal in vision science is to identify features that are important for object and scene recognition. Reverse correlation methods have been used to uncover features important for recognizing faces and other stimuli with low intra-class variability. However, these methods are less successful when applied to natural scenes with variability in their appearance.

To rectify this, we developed Clicktionary, a web-based game for identifying features for recognizing real-world objects. Pairs of participants play together in different roles to identify objects: A “teacher” reveals image regions diagnostic of the object’s category while a “student” tries to recognize the object. Aggregating …

Texture Modelling Using Convolutional Neural Networks, Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

MODVIS Workshop

We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. Extending this framework to texture transfer, we introduce A Neural Algorithm of Artistic Style that …

Focusing On Selection For Fixation, John K. Tsotsos, Calden Wloka, Yulia Kotseruba

MODVIS Workshop

Building on our presentation at MODVIS 2015, we continue in our quest to discover a functional, computational, explanation of the relationship among visual attention, interpretation of visual stimuli, and eye movements, and how these produce visual behavior. Here, we focus on one component, how selection is accomplished for the next fixation. The popularity of saliency map models drives the inference that this is solved; we suggested otherwise at MODVIS 2015. Here, we provide additional empirical and theoretical arguments. We then develop arguments that a cluster of complementary, conspicuity representations drive selection, modulated by task goals and history, leading to a …

Video Event Understanding With Pattern Theory, Fillipe Souza, Sudeep Sarkar, Anuj Srivastava, Jingyong Su

MODVIS Workshop

We propose a combinatorial approach built on Grenander’s pattern theory to generate semantic interpretations of video events of human activities. The basic units of representations, termed generators, are linked with each other using pairwise connections, termed bonds, that satisfy predefined relations. Different generators are specified for different levels, from (image) features at the bottom level to (human) actions at the highest, providing a rich representation of items in a scene. The resulting configurations of connected generators provide scene interpretations; the inference goal is to parse given video data and generate high-probability configurations. The probabilistic structures are imposed using energies that …

Two Correspondence Problems Easier Than One, Aaron Michaux, Zygmunt Pizlo

MODVIS Workshop

Computer vision research rarely makes use of symmetry in stereo reconstruction despite its established importance in perceptual psychology. Such stereo reconstructions produce visually satisfying figures with precisely located points and lines, even when input images have low or moderate resolution. However, because few invariants exist, there are no known general approaches to solving symmetry correspondence on real images. The problem is significantly easier when combined with the binocular correspondence problem, because each correspondence problem provides strong non-overlapping constraints on the solution space. We demonstrate a system that leverages these constraints to produce accurate stereo models from pairs of binocular images …

Formal Aspects Of Non-Rigid-Shape-From-Motion Perception, Vicky Froyen, Qasim Zaidi

MODVIS Workshop

Our world is full of objects that deform over time, for example animals, trees and clouds. Yet, the human visual system seems to readily disentangle object motions from non-rigid deformations, in order to categorize objects, recognize the nature of actions such as running or jumping, and even to infer intentions. A large body of experimental work has been devoted to extracting rigid structure from motion, but there is little experimental work on the perception of non-rigid 3-D shapes from motion (e.g. Jain, 2011). Similarly, until recently, almost all formal work had concentrated on the rigid case. In the last fifteen …

Object Recognition And Visual Search With A Physiologically Grounded Model Of Visual Attention, Frederik Beuth, Fred H. Hamker

MODVIS Workshop

Visual attention models can explain a rich set of physiological data (Reynolds & Heeger, 2009, Neuron), but can rarely link these findings to real-world tasks. Here, we would like to narrow this gap with a novel, physiologically grounded model of visual attention by demonstrating its objects recognition abilities in noisy scenes.

To base the model on physiological data, we used a recently developed microcircuit model of visual attention (Beuth & Hamker, in revision, Vision Res) which explains a large set of attention experiments, e.g. biased competition, modulation of contrast response functions, tuning curves, and surround suppression. Objects are represented by …

Modeling Visual Features To Recognize Biological Motion: A Developmental Approach, Giulio Sandini, Nicoletta Noceti, Alessia Vignolo, Alessandra Sciutti, Francesco Rea, Alessandro Verri, Francesca Odone

MODVIS Workshop

In this work we deal with the problem of designing and developing computational vision models – comparable to the early stages of the human development – using coarse low-level information.

More specifically, we consider a binary classification setting to characterize biological movements with respect to non-biological dynamic events. To this purpose, our model builds on top of the optical flow estimation, and abstract the representation to simulate the limited amount of visual information available at birth. We take inspiration from known biological motion regularities explained by the Two-Thirds Power Law, and design a motion representation that includes different low-level features, …