Physical Sciences and Mathematics Commons^™

The Model 2.0 And Friends: An Interim Report, Garrison W. Cottrell, Martha Gahl, Shubham Kulkarni, Shashank Venkatramani, Yash Shah, Keyu Long, Xuzhe Zhi, Shivaank Agarwal, Cody Li, Jingyuan He, Thomas Fischer

MODVIS Workshop

Last year, I reported on preliminary results of an anatomically-inspired deep learning model of the visual system and its role in explaining the face inversion effect. This year, I will report on new results and some variations on network architectures that we have explored, mainly as a way to generate discussion and get feedback. This is by no means a polished, final presentation!

We look forward to the group’s suggestions for these projects.

Automated Delineation Of Visual Area Boundaries And Eccentricities By A Cnn Using Functional, Anatomical, And Diffusion-Weighted Mri Data, Noah C. Benson, Bogeng Song, Toshikazu Miyata, Hiromasa Takemura, Jonathan Winawer

MODVIS Workshop

Delineating visual field maps and iso-eccentricities from fMRI data is an important but time-consuming task for many neuroimaging studies on the human visual cortex because the traditional methods of doing so using retinotopic mapping experiments require substantial expertise as well as scanner, computer, and human time. Automated methods based on gray-matter anatomy or a combination of anatomy and functional mapping can reduce these requirements but are less accurate than experts. Convolutional Neural Networks (CNNs) are powerful tools for automated medical image segmentation. We hypothesize that CNNs can define visual area boundaries with high accuracy. We trained U-Net CNNs with ResNet18 …

Toward A Manifold Encoding Neural Responses, Luciano Dyballa, Andra M. Rudzite, Mahmood S. Hoseini, Mishek Thapa, Michael P. Stryker, Greg D. Field, Steven W. Zucker

MODVIS Workshop

Understanding circuit properties from physiological data presents two challenges: (i) recordings do not reveal connectivity, and (ii) stimuli only exercise circuits to a limited extent. We address these challenges for the mouse visual system with a novel neural manifold obtained using unsupervised algorithms. Each point in our manifold is a neuron; nearby neurons respond similarly in time to similar parts of a stimulus ensemble. This ensemble includes drifting gratings and flows, i.e., patterns resembling what a mouse would “see” running through fields.

Regarding (i), our manifold differs from the standard practice in computational neuroscience: embedding trials in neural coordinates. Topology …

How Object Segmentation And Perceptual Grouping Emerge In Noisy Variational Autoencoders, Ben Lonnqvist, Zhengqing Wu, Michael H. Herzog

MODVIS Workshop

Many animals and humans can recognize and segment objects from their backgrounds. Whether object segmentation is necessary for object recognition has long been a topic of debate. Deep neural networks (DNNs) excel at object recognition, but not at segmentation tasks - this has led to the belief that object recognition and segmentation are separate mechanisms in visual processing. Here, however, we show evidence that in variational autoencoders (VAEs), segmentation and faithful representation of data can be interlinked. VAEs are encoder-decoder models that learn to represent independent generative factors of the data as a distribution in a very small bottleneck layer; …

Evaluating Models Of Scanpath Prediction, Matthias Kümmerer, Matthias Bethge

MODVIS Workshop

No abstract provided.

A Dynamical Model Of Binding In Visual Cortex During Incremental Grouping And Search, Daniel Schmid, Daniel A. Braun, Heiko Neumann

MODVIS Workshop

Binding of visual information is crucial for several perceptual tasks. To incrementally group an object, elements in a space-feature neighborhood need to be bound together starting from an attended location (Roelfsema, TICS, 2005). To perform visual search, candidate locations and cued features must be evaluated conjunctively to retrieve a target (Treisman&Gormican, Psychol Rev, 1988). Despite different requirements on binding, both tasks are solved by the same neural substrate. In a model of perceptual decision-making, we give a mechanistic explanation for how this can be achieved. The architecture consists of a visual cortex module and a higher-order thalamic module. While the …

Understanding The Influence Of Perceptual Noise On Visual Flanker Effects Through Bayesian Model Fitting, Jordan Deakin, Dietmar Heinke

MODVIS Workshop

No abstract provided.

A Two-Layer Model Explains Higher-Order Feature Selectivity Of V2 Neurons, Timothy D. Oleskiw, Justin D. Lieber, J. Anthony Movshon, Eero P. Simoncelli

MODVIS Workshop

Neurons in cortical area V2 respond selectively to higher-order visual features, such as the quasi-periodic structure of natural texture. However, a functional account of how V2 neurons build selectivity for complex natural image features from their inputs – V1 neurons locally tuned for orientation and spatial frequency – remains elusive.

We made single-unit recordings in area V2 in two fixating rhesus macaques. We presented stimuli composed of multiple superimposed grating patches that localize contrast energy in space, orientation, and scale. V2 activity is modeled via a two-layer linear-nonlinear network, optimized to use a sparse combination of V1-like outputs to account …

Smiler: Consistent And Usable Saliency Model Implementations, Toni Kunic, Calden Wloka, John K. Tsotsos

MODVIS Workshop

The Saliency Model Implementation Library for Experimental Research (SMILER) is a new software package which provides an open, standardized, and extensible framework for maintaining and executing computational saliency models. This work drastically reduces the human effort required to apply saliency algorithms to new tasks and datasets, while also ensuring consistency and procedural correctness for results and conclusions produced by different parties. At its launch SMILER already includes twenty three saliency models (fourteen models based in MATLAB and nine supported through containerization), and the open design of SMILER encourages this number to grow with future contributions from the community. The project …

Is The Selective Tuning Model Of Visual Attention Still Relevant?, John K. Tsotsos

MODVIS Workshop

No abstract provided.

The Fluid Representations Of Networks Estimating Liquid Viscosity, Jan Jaap R. Van Assen, Shin'ya Nishida, Roland W. Fleming

MODVIS Workshop

No abstract provided.

Towards Human Retinal Cones Spatial Distribution Modeling, Matteo Paolo Lanaro, Hélène Perrier, David Coeurjolly, Victor Ostromoukhov, Alessandro Rizzi

MODVIS Workshop

No abstract provided.

Selecting Maximally-Predictive Deep Features To Explain What Drives Fixations In Free-Viewing, Matthias Kümmerer, Thomas S.A. Wallis, Matthias Bethge

MODVIS Workshop

No abstract provided.

Recovering Depth From Stereo Without Using Any Oculomotor Information, Tadamasa Sawada

MODVIS Workshop

The human visual system uses binocular disparity to perceive depth within 3D scenes. It is commonly assumed that the visual system needs oculomotor information about the relative orientation of the two eyes to perceive depth on the basis of binocular disparity. The necessary oculomotor information can be obtained from an efference copy of the oculomotor signals, or from a 2D distribution of the vertical disparity, specifically, from the vertical component of binocular disparity. It is known that oculomotor information from the efference copy and from the vertical disparity distribution can affect the perception of depth based on binocular disparity. But, …

Mental Geometry For Estimating Relative 3d Size, Akihito Maruya, Qasim Zaidi Dr.

MODVIS Workshop

No abstract provided.

The Challenge For Vision Of Fluctuating Real-World Illumination, David H. Foster

MODVIS Workshop

No abstract provided.

Brightness Perception Involves Local Adaptation Opposed By Lateral Interaction, Qasim Zaidi, Romain Bachy, Jose-Manuel Alonso

MODVIS Workshop

No abstract provided.

Understanding Qualitative 3d Shape From Texture And Shading, Benjamin Kunsberg, Steven W. Zucker

MODVIS Workshop

No abstract provided.

Measuring Symmetry In Real-World Scenes Using Derivatives Of The Medial Axis Radius Function, Morteza Rezanejad, John D. Wilder, Kaleem Siddiqi, Sven Dickinson, Allan Jepson, Dirk B. Walther

MODVIS Workshop

Symmetry has been shown to be an important principle that guides the grouping of scene information. Previously, we have described a method for measuring the local, ribbon symmetry content of line-drawings of real-world scenes (Rezanejad, et al., MODVIS 2017), and we demonstrated that this information has important behavioral consequences (Wilder, et al., MODIVS 2017). Here, we describe a continuous, local version of the symmetry measure, that allows for both ribbon and taper symmetry to be captured. Our original method looked at the difference in the radius between successive maximal discs along a symmetric axis. The number of radii differences in …

Consistent Saliency Benchmarking: How One Model Can Win On All Metrics, Matthias Kümmerer, Thomas S.A. Wallis, Matthias Bethge

MODVIS Workshop

No abstract provided.

Learning To Identify Depth Edges In Real-World Images With 3d Ground Truth, Krista A. Ehinger, Kevin T. Joseph, Wendy J. Adams, Erich W. Graf, James H. Elder

MODVIS Workshop

No abstract provided.

A Single Shape From Multiple Cues: How Local And Global Information Organizes Shape Inference, Benjamin Kunsberg, Steven W. Zucker

MODVIS Workshop

No abstract provided.

Shape Features Underlying The Perception Of Liquids, Jan Jaap R. Van Assen, Pascal Barla, Roland W. Fleming

MODVIS Workshop

No abstract provided.

Scoring Scene Symmetry, Morteza Rezanejad, John D. Wilder, Sven Dickinson, Allan Jepson, Dirk B. Walther, Kaleem Siddiqi

MODVIS Workshop

No abstract provided.

Can Cone Signals In The Wild Be Predicted From The Past?, David H. Foster, Iván Marín-Franch

MODVIS Workshop

In the natural world, the past is usually a good guide to the future. If light from the sun and sky is blue earlier in the day and yellow now, then it is likely to be more yellow later, as the sun's elevation decreases. But is the light reflected from a scene into the eye as predictable as the light incident upon the scene, especially when lighting changes are not just spectral but include changes in local shadows and mutual reflections? The aim of this work was to test the predictability of cone photoreceptor signals in the wild over the …

Role Of The Cost Of Plasticity In Determining The Features Of Fast Vision In Humans., Maria M. Del Viva Phd, Renato Budinich M. Sc, Laura Palmieri M. Sc, Vladimir S Georgiev Phd, Giovanni Punzi Phd

MODVIS Workshop

No abstract provided.

Large-Scale Discovery Of Visual Features For Object Recognition, Drew Linsley, Sven Eberhardt, Dan Shiebler, Thomas Serre

MODVIS Workshop

A central goal in vision science is to identify features that are important for object and scene recognition. Reverse correlation methods have been used to uncover features important for recognizing faces and other stimuli with low intra-class variability. However, these methods are less successful when applied to natural scenes with variability in their appearance.

To rectify this, we developed Clicktionary, a web-based game for identifying features for recognizing real-world objects. Pairs of participants play together in different roles to identify objects: A “teacher” reveals image regions diagnostic of the object’s category while a “student” tries to recognize the object. Aggregating …

Texture Modelling Using Convolutional Neural Networks, Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

MODVIS Workshop

We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. Extending this framework to texture transfer, we introduce A Neural Algorithm of Artistic Style that …

Failure Of Surface Color Cues Under Natural Changes In Lighting, David H. Foster, Iván Marín-Franch

MODVIS Workshop

Color allows us to effortlessly discriminate and identify surfaces and objects by their reflected light. Although the reflected spectrum changes with the illumination spectrum, cone photoreceptor signals can be transformed to give useful cues for surface color. But what happens when both the spectrum and the geometry of the illumination change, as with lighting from the sun and sky? Is it possible, as a matter of principle, to obtain reliable cues by processing cone signals alone? This question was addressed here by estimating the information provided by cone signals from time-lapse hyperspectral radiance images of five outdoor scenes under natural …

Focusing On Selection For Fixation, John K. Tsotsos, Calden Wloka, Yulia Kotseruba

MODVIS Workshop

Building on our presentation at MODVIS 2015, we continue in our quest to discover a functional, computational, explanation of the relationship among visual attention, interpretation of visual stimuli, and eye movements, and how these produce visual behavior. Here, we focus on one component, how selection is accomplished for the next fixation. The popularity of saliency map models drives the inference that this is solved; we suggested otherwise at MODVIS 2015. Here, we provide additional empirical and theoretical arguments. We then develop arguments that a cluster of complementary, conspicuity representations drive selection, modulated by task goals and history, leading to a …