Cognition and Perception Commons^™

Emotion Recognition With Audio, Video, Eeg, And Emg: A Dataset And Baseline Approaches, Jin Chen, Tony Ro, Zhigang Zhu

Publications and Research

This paper describes a new posed multimodal emotional dataset and compares human emotion classification based on four different modalities - audio, video, electromyography (EMG), and electroencephalography (EEG). The results are reported with several baseline approaches using various feature extraction techniques and machine-learning algorithms. First, we collected a dataset from 11 human subjects expressing six basic emotions and one neutral emotion. We then extracted features from each modality using principal component analysis, autoencoder, convolution network, and mel-frequency cepstral coefficient (MFCC), some unique to individual modalities. A number of baseline models have been applied to compare the classification performance in emotion recognition, …

During Natural Viewing, Neural Processing Of Visual Targets Continues Throughout Saccades, Atanas D. Stankov, Jonathan Touryan, Stephen Gordon, Anthony J. Ries, Jason Ki, Lucas C. Parra

Publications and Research

Relatively little is known about visual processing during free-viewing visual search in realistic dynamic environments. Free-viewing is characterized by frequent saccades. During saccades, visual processing is thought to be suppressed, yet we know that the presaccadic visual content can modulate postsaccadic processing. To better understand these processes in a realistic setting, we study here saccades and neural responses elicited by the appearance of visual targets in a realistic virtual environment. While subjects were being driven through a 3D virtual town, they were asked to discriminate between targets that appear on the road. Using a system identification approach, we separated overlapping …

Visual Modulation Of Resting State Α Oscillations, Kelly Webster, Tony Ro

Publications and Research

Once thought to simply reflect passive cortical idling, recent studies have demonstrated that α oscillations play a causal role in cognition and perception. However, whether and how cognitive or sensory processes modulate various components of the α rhythm is poorly understood. Sensory input and resting states were manipulated in human subjects while electroencephalography (EEG) activity was recorded in three conditions: eyes-open fixating on a visual stimulus, eyes-open without visual input (darkness), and eyes-closed without visual input (darkness). We show that α power and peak frequency increase when visual input is reduced compared to the eyes open, fixating condition. These results …

Language Experience With A Native-Language Phoneme Sequence Modulates The Effects Of Attention On Cortical Sensory Processing, Valerie L. Shafer, Monica Wagner, Jungmee Lee, Francesca Mingino, Colleen O'Brien, Adam Constantine, Mitchell Steinschneider

Publications and Research

Auditory evoked potentials (AEP) reflect spectro-temporal feature changes within the spoken word and are sufficiently reliable to probe deficits in auditory processing. The current research assessed whether attentional modulation would alter the morphology of these AEPs and whether native-language experience with phoneme sequences would influence the effects of attention. Native-English and native-Polish adults listened to nonsense word pairs that contained the phoneme sequence onsets /st/, /sət/, /pət/ that occur in both the Polish and English languages and the phoneme sequence onset /pt/ that occurs in the Polish language, but not the English language. Participants listened to word pairs within two …

Combined Mnemonic Strategy Training And High-Definition Transcranial Direct Current Stimulation For Memory Deficits In Mild Cognitive Impairment, Benjamin M. Hampstead, Krishnankutty Sathian, Marom Bikson, Anthony Y. Stringer

Publications and Research

Introduction: Memory deficits characterize Alzheimer’s dementia and the clinical precursor stage known as mild cognitive impairment. Nonpharmacologic interventions hold promise for enhancing functioning in these patients, potentially delaying functional impairment that denotes transition to dementia. Previous findings revealed that mnemonic strategy training (MST) enhances long-term retention of trained stimuli and is accompanied by increased blood oxygen level–dependent signal in the lateral frontal and parietal cortices as well as in the hippocampus. The present study was designed to enhance MST generalization, and the range of patients who benefit, via concurrent delivery of transcranial direct current stimulation (tDCS).

Methods: This protocol describes …

Attention Strongly Modulates Reliability Of Neural Responses To Naturalistic Narrative Stimuli, Jason J. Ki, Simon P. Kelly, Lucas C. Parra

Publications and Research

Attentional engagement is a major determinant of how effectively we gather information through our senses. Alongside the sheer growth in the amount and variety of information content that we are presented with through modern media, there is increased variability in the degree to which we “absorb” that information. Traditional research on attention has illuminated the basic principles of sensory selection to isolated features or locations, but it provides little insight into the neural underpinnings of our attentional engagement with modern naturalistic content. Here, we show inhumansubjects that the reliability of an individual’s neural responses with respect to a larger group …

The Effects Of Changing Attention And Context In An Awake Offline Processing Period On Visual Long-Term Memory, Timothy M. Ellmore, Anna Feng, Kenneth Ng, Luthfunnahar Dewan, James C. Root

Publications and Research

There is accumulating evidence that sleep as well as awake offline processing is important for the transformation of new experiences into long-term memory (LTM). Yet much remains to be understood about how various cognitive factors influence the efficiency of awake offline processing. In the present study we investigated how changes in attention and context in the immediate period after exposure to new visual information influences LTM consolidation. After presentation of multiple naturalistic scenes within a working memory paradigm, recognition was assessed 30 min and 24 h later in three groups of subjects. One group of subjects engaged in a focused …

Direct Control Of Visual Perception With Phase-Specific Modulation Of Posterior Parietal Cortex, Andrew Jaegle, Tony Ro

Publications and Research

We examined the causal relationship between the phase of alpha oscillations (9–12 Hz) and conscious visual perception using rhythmic TMS (rTMS) while simultaneously recording EEG activity. rTMS of posterior parietal cortex at an alpha frequency (10 Hz), but not occipital or sham rTMS, both entrained the phase of subsequent alpha oscillatory activity and produced a phase-dependent change on subsequent visual perception, with lower discrimination accuracy for targets presented at one phase of the alpha oscillatory waveform than for targets presented at the opposite phase. By extrinsically manipulating the phase of alpha before stimulus presentation, we provide direct evidence that the …

Unconscious Priming Requires Early Visual Cortex At Specific Temporal Phases Of Processing, Marjan Persuh, Tony Ro

Publications and Research

Although examples of unconscious shape priming have been well documented, whether such priming requires early visual cortex (V1/V2) has not been established. In the current study, we used TMS of V1/V2 at varying temporal intervals to suppress the visibility of preceding shape primes while the interval between primes and targets was kept constant. Our results show that, although conscious perception requires V1/V2, unconscious priming can occur without V1/V2 at an intermediate temporal interval but not at early (5–25 msec) or later (65–125 msec) stages of processing. Because the later time window of unconscious priming suppression has been proposed to interfere …

Unconscious Processing Of Unattended Features In Human Visual Cortex, Tatiana Aloi Emmanouil, Philip Burton, Tony Ro

Publications and Research

Unconscious processing has been convincingly demonstrated for task-relevant feature dimensions. However, it is possible that the visual system is capable of more complex unconscious operations, extracting visual features even when they are unattended and task irrelevant. In the current study, we addressed this question by measuring unconscious priming using a task in which human participants attended to a target object's shape while ignoring its color. We measured both behavioral priming effects and priming-related fMRI activations from primes that were unconsciously presented using metacontrast masking. The results showed faster RTs and decreases in fMRI activation only when the primes were identical …

Maintenance Of Visual Stability In The Human Posterior Parietal Cortex, Erik Chang, Tony Ro

Publications and Research

Visual stability refers to our stable visuospatial perceptions despite the unstable visual input caused by saccades. Functional neuroimaging results, studies on patients with posterior parietal cortex (PPC) lesions, and single-unit recordings in the lateral intraparietal sulcus of primates indirectly suggest that the PPC might be a potential locus of visual stability through its involvement with spatial remapping. Here we directly explored the role of the PPC in visual stability by applying transcranial magnetic stimulation (TMS) while participants performed a perisaccadic displacement detection task. We show that TMS over the PPC but not a frontal control site alters sensitivity to displacement …